Comments (7)
from callr.
In terms of sharing a progress bar from the child back to the parent? I think that would be great.
from callr.
@hadley I was wondering if it should be here or in some higher level parallel package. But yeah, probably makes sense to have it here.
from callr.
Rough plan:
Implementation notes
- The function would throw conditions, and the injected code will catch them, and send them to the
parent process. This way progress events can be generated the same way for local and remote
processes. - Ultimately we want to throttle the thrown conditions, i.e. do not throw more than one in a 200ms
interval, except for urgent events, like termination. Maybe we want to throttle by type, e.g. progress
update, status update, estimate update, etc. - We need to have a C API, but that will be different, because you cannot easily throw conditions
from C. So we'll just write to the pipe from C. The C API will need to be throttled as well. - We need to work out how loops with progress bars can be embedded into each other. Maybe
progress bars need ids, and the conditions will contain the ids.
The API
Not decided yet how the conditions will be generated, but they could roughly correspond to these functions:
add_job(name, status, total, format, estimate, auto_estimate)
set_job_progress(id, progress)
add_job_progress(id, increment)
set_job_status(id, status)
set_job_estimate(id, seconds)
add_job_output(id, output, type)
complete_job(id, succeeded, output, error)
from callr.
FWIW there was a project about parallel progress bars at this years' unconf (https://github.com/ropenscilabs/jobstatus), maybe not really that useful
Throwing conditions from C isn't really that hard, I think it would be best to use the same API for both cases, e.g. if you think the pipe interface is better we should provide R functions to wrap it.
from callr.
@jimhester re C API, I am just worried that for C code throwing a condition is too expensive.
In the end the progress event will be written to a pipe, maybe the C API can just write it w/o a throwing a condition....
from callr.
OK, callr now catches conditions of class callr_message
in the subprocess, transmits them (the whole object) to the parent process, and r_session$run()
re-throws them in the parent process.
So, as long as the progress is signalled via messages, callr does everything it needs to do, to support progress bars from child processes. Of course some new APIs are still needed in the progress package: r-lib/progress#72
from callr.
Related Issues (20)
- Upkeep for callr (2023)
- Add alt-text to pictures, plots, etc; see https://posit.co/blog/knitr-fig-alt/ for examples
- Alternative to `arch` argument with `callr` v3.3.2 HOT 2
- r_bg processes hang when waiting for others to spawn HOT 2
- How to detect failure in `r_bg()` ? HOT 1
- Release callr 3.7.4
- please do not run unsigned dll files with random names from temp folder HOT 7
- Release callr 3.7.5
- callr update from CRAN HOT 1
- callr update from CRAN HOT 2
- Rtmp files left in /tmp directory with VSCode HOT 2
- How to use customize functions in the `func` function? HOT 4
- callr update from CRAN HOT 3
- Release callr 3.7.6 HOT 1
- how to get out of session$attach() HOT 5
- Setting different R_HOME for the callr subprocess HOT 1
- r_process subclass shouldn't be cloneable HOT 1
- Spawn background processes across HPC cluster nodes HOT 1
- Use `testthat::with_mock()` instead of mockery
- `r_bg()` don't finished until run `rp$read_error()` and `rp$read_output()`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from callr.