rofl0r / jobflow Goto Github PK

Thanks for the update.

Most of your claims about GNU Parallel are easy to verify. (E.g. run time parallel echo ::: 1 to see the startup time and seq 1000 | time -v parallel true to see memory usage).

But claiming that Perl's buggy garbage collector is a disadvantage to GNU Parallel is not easy to verify. So if you want to use correct claims, I will encourage you to link to an example that shows the current version of GNU Parallel being affected by Perl's buggy garbage collector.

Having run GNU Parallel on many different architectures and many different versions of Perl (version 5.8.4..5.26.1 on armv7l, sparc, armv6hl, alpha, mips) I have never seen the current version of GNU Parallel being affected by Perl's buggy garbage collector, so I am quite interested in seeing the exact situation in which that happens.

So may I suggest you include a link to a reproducible example showing this actually happening?

Hi

In your description https://github.com/rofl0r/jobflow/blob/master/README.md you write:

this program is inspired by GNU parallel, but has the following differences [...] does not leak memory

I have been unable to find an example of GNU Parallel leaking memory. If you have, could you be kind and let me know? If you do not have an example, consider removing that bullet point as it can be misinterpreted as if GNU Parallel leaks memory.

$ git clone https://github.com/rofl0r/jobflow.git
$ cd jobflow/
$ CFLAGS=" -Wall -Wextra -std=c99 -D_GNU_SOURCE -g -O0 " rcb --force jobflow.c ""
/bin/sh: 1: rcb: not found
make: *** [debug] Error 127

Fixed rcb issue.

$ git clone https://github.com/rofl0r/rcb.git
$ cd rcb
$ cp rcb2make.pl ~/bin/rcb2make
$ cp rcb.pl ~/bin/rcb

$ cd ~/jobflow
$ make
CFLAGS=" -Wall -Wextra -std=c99 -D_GNU_SOURCE -g -O0 " rcb --force jobflow.c ""
[RcB] $CC not set, defaulting to cc
[RcB] $NM not set, defaulting to nm
[RcB] scanning deps...failed to find dependency /home/iqbala/jobflow/../lib/include/optparser.h referenced from /home/iqbala/jobflow/jobflow.c!
failed to find dependency /home/iqbala/jobflow/../lib/include/stringptr.h referenced from /home/iqbala/jobflow/jobflow.c!
failed to find dependency /home/iqbala/jobflow/../lib/include/stringptrlist.h referenced from /home/iqbala/jobflow/jobflow.c!
failed to find dependency /home/iqbala/jobflow/../lib/include/sblist.h referenced from /home/iqbala/jobflow/jobflow.c!
failed to find dependency /home/iqbala/jobflow/../lib/include/strlib.h referenced from /home/iqbala/jobflow/jobflow.c!
failed to find dependency /home/iqbala/jobflow/../lib/include/timelib.h referenced from /home/iqbala/jobflow/jobflow.c!
failed to find dependency /home/iqbala/jobflow/../lib/include/filelib.h referenced from /home/iqbala/jobflow/jobflow.c!
done
[RcB] compiling main file...
[CC] cc -Wall -Wextra -std=c99 -D_GNU_SOURCE -g -O0 -c jobflow.c -o jobflow.o
jobflow.c:25:38: fatal error: ../lib/include/optparser.h: No such file or directory
#include "../lib/include/optparser.h"
^
compilation terminated.
make: *** [debug] Error 1

when using more than 1 process, and not the -buffered option, the output of some processes gets lost.
piping the output into cat > file instead, everything arrives.
linux bug ?

test:
fail:
seq 100 | ./jobflow.out -threads=100 -exec echo {} > test.tmp ; wc -l test.tmp
should return 100, but does not always

success
seq 100 | ./jobflow.out -threads=100 -exec echo {} | cat > test.tmp ; wc -l test.tmp
always returns 100

note missing space between echo and {}

$ seq 1000 | jobflow -threads=100 -exec echo{}
posix_spawn: No such file or directory
posix_spawn: No such file or directory
posix_spawn: No such file or directory
posix_spawn: No such file or directory
...