Comments (1)
Just to add context. It also seems to use quite a lot of processes. the job contains only 4 assets, however they do retry forever:
e396d0becea1 hardcore_kepler 10.72% 1.8GiB / 30.38GiB 5.92% 1.28GB / 1.5GB 2.51MB / 24.6kB 158
And it seems like there is a lot of similar looking processes:
UID PID PPID C STIME TTY TIME CMD
root 7652 6557 0 06:27 ? 00:00:00 /usr/local/bin/python -c from multiprocessing.resource_tracker import main;main(6)
root 37880 6557 0 06:41 ? 00:00:41 /usr/local/bin/python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=7, pipe_handle=11) --multiprocessing-fork
root 45723 37880 0 06:45 ? 00:00:00 tail -F -c +0 /opt/dagster/dagster_home/storage/6bce6900-5184-4209-9a4c-906e552e371f/compute_logs/ehexwzcz.out
root 45724 37880 0 06:45 ? 00:00:00 /usr/local/bin/python /usr/local/lib/python3.10/site-packages/dagster/_core/execution/watch_orphans.py 384 520
root 45725 37880 0 06:45 ? 00:00:00 tail -F -c +0 /opt/dagster/dagster_home/storage/6bce6900-5184-4209-9a4c-906e552e371f/compute_logs/ehexwzcz.err
root 45726 37880 0 06:45 ? 00:00:00 /usr/local/bin/python /usr/local/lib/python3.10/site-packages/dagster/_core/execution/watch_orphans.py 384 522
root 312188 6557 2 09:00 ? 00:00:41 /usr/local/bin/python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=7, pipe_handle=21) --multiprocessing-fork
root 315025 312188 0 09:01 ? 00:00:00 tail -F -c +0 /opt/dagster/dagster_home/storage/6bce6900-5184-4209-9a4c-906e552e371f/compute_logs/tgismvfl.out
root 315026 312188 0 09:01 ? 00:00:00 /usr/local/bin/python /usr/local/lib/python3.10/site-packages/dagster/_core/execution/watch_orphans.py 3799 3845
root 315027 312188 0 09:01 ? 00:00:00 tail -F -c +0 /opt/dagster/dagster_home/storage/6bce6900-5184-4209-9a4c-906e552e371f/compute_logs/tgismvfl.err
root 315028 312188 0 09:01 ? 00:00:00 /usr/local/bin/python /usr/local/lib/python3.10/site-packages/dagster/_core/execution/watch_orphans.py 3799 3847
root 347147 6557 7 09:17 ? 00:00:37 /usr/local/bin/python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=7, pipe_handle=18) --multiprocessing-fork
root 347148 6557 8 09:17 ? 00:00:42 /usr/local/bin/python -c from multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=7, pipe_handle=23) --multiprocessing-fork
root 350354 347147 0 09:19 ? 00:00:00 tail -F -c +0 /opt/dagster/dagster_home/storage/6bce6900-5184-4209-9a4c-906e552e371f/compute_logs/vnoocjqe.out
root 350355 347147 0 09:19 ? 00:00:00 /usr/local/bin/python /usr/local/lib/python3.10/site-packages/dagster/_core/execution/watch_orphans.py 4295 4375
root 350356 347147 0 09:19 ? 00:00:00 tail -F -c +0 /opt/dagster/dagster_home/storage/6bce6900-5184-4209-9a4c-906e552e371f/compute_logs/vnoocjqe.err
root 350357 347147 0 09:19 ? 00:00:00 /usr/local/bin/python /usr/local/lib/python3.10/site-packages/dagster/_core/execution/watch_orphans.py 4295 4377
root 350449 347148 0 09:19 ? 00:00:00 tail -F -c +0 /opt/dagster/dagster_home/storage/6bce6900-5184-4209-9a4c-906e552e371f/compute_logs/gsdkpjop.out
root 350450 347148 0 09:19 ? 00:00:00 /usr/local/bin/python /usr/local/lib/python3.10/site-packages/dagster/_core/execution/watch_orphans.py 4296 4385
root 350451 347148 0 09:19 ? 00:00:00 tail -F -c +0 /opt/dagster/dagster_home/storage/6bce6900-5184-4209-9a4c-906e552e371f/compute_logs/gsdkpjop.err
root 350452 347148 0 09:19 ? 00:00:00 /usr/local/bin/python /usr/local/lib/python3.10/site-packages/dagster/_core/execution/watch_orphans.py 4296 4387
Not sure if this is by design.
from dagster.
Related Issues (20)
- Url link in ressource page is incorrect
- Incorrect / persistent new data version flag HOT 1
- Enable batch-writing materialization events during ranged run
- Error on Asset Dependency with Multiple dbt Model Tests after Updating to Dagster 1.6.4 HOT 1
- Unable to add dynamic partition in UI, ok in code
- Auto-layout the asset lineage on the UI
- dbt groups do not update in the UI HOT 1
- gRPC Client does not resolve host name in AWS namespaces HOT 1
- in auto-materialize, handle asset observations with data times but no data versions
- feat: Support multi-line string in run_config HOT 1
- dagster-deltalake S3Config allow_unsafe_rename parameter ignored HOT 1
- `build_sensor_context` does not build a valid context for `asset_sensor` when the sensor has `Resources`
- error in view schedules and scheduled-runs HOT 6
- Dagster Daemon randomly crashes HOT 6
- Can't import DuckDBPolarsIOManager HOT 2
- Log rolling for compute logs based on size:
- gRPC server crashes when running an amd container on arm (mac m1)
- [pipes] problems with use in graph_asset
- Subsetting asset checks in graph_multi_assets
- Graph Asset more Asset-like HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dagster.