Add sub process helpers? about quicli

[polarathene]

I've done a Python script(it takes in a bunch of args but doesn't have much interactivity beyond that and outputting information/progress). It would call a third party process for a VFX pipeline process, it'd be helpful catching any output from it like if it spits out an error, when the process exits(but I guess that's sync based logic?), running multiple instances was something I didn't get around to(I just called the script multiple times), that scales up to a certain number of cores(could be all by default, or 3/4th of cores, with user arg, depends on resource usage like RAM required, one of my subprocesses can use up to 60GB due to dataset).

There is a crate called parallel I think that specifically handles the use case of single threaded subprocess being run multiple times, maybe that could be integrated? Ideally I'd want to have them emit some progress information that can be consumed and displayed as total progress information for the actuall process running, alternatively displaying progress bars for each subprocess could be neat too :)

[killercup]

I think we can think of "subprocesses" and "running stuff parallel" as two separate things. The following is more about "running stuff parallel".

---

That seems like a good use-case, @polarathene, and I think using rayon (cf. #4) and std::command::Command/duct can satisfy a lot of your requirements. Let me summarize your use case a bit (correct me if I got it wrong):

• You start a script with a bunch of parameters
• Its job is to n processes that do work independent of each other
• You want to run these processes in parallel
• You want to capture their output.

I recently wrote this, where files is a bunch of inputs basically, and run starts a process and parses its output:

let res = files.par_iter()
    .map(|path| run(path))
    .reduce(TestResults::default, |a, b| a + b);

You can set RAYON_NUM_THREADS to control this a bit, I wouldn't go for making this configurable by default.