j50n / deno-proc Goto Github PK
View Code? Open in Web Editor NEWA better way to work with processes in Deno.
License: Other
A better way to work with processes in Deno.
License: Other
I am currently just inheriting.
Callers may want to suppress, scrape, or redirect. Need to work out various use cases and develop tests for each.
Run some tests and see what is going on.
Inspect the Deno source and make sure we are using the correct buffer size for transferring data.
Simple examples of each input/output type. This needs to be completed.
A good evening project.
Check out some standard signals and write some tests. It probably does not work as expected in its current form.
Chained Errors should be optional, as they are a bit harder to deal with in production code, but can be useful during debugging.
Just extend the parameters so environment variables work like they do in Deno.run.
Currently, you can pass an array of strings to the error handler function, and it will attach it to any error that is raised. This is a design error. It should be any type. We are just passing it along.
I have some ideas for reworking the API so it is a lot more intuitive than the current one - and also covers more use cases. This ticket is for the feature branch, as this is going to break stuff, and I want to make sure it is ready before I push it out.
How would I add color and a timestamp on each line of stderr? This is a nice example.
Something is wrong with string Iterables. I am getting 50,000 to 100,000 lines per second peak performance on very small lines. It seems like the overhead per line should be much lower than 200usec per line.
I need to do some profiling to figure out where the performance hit is happening.
Current error processing is baked in. In a few cases, we don't want to throw an error on non-zero exit codes, and it may be desirable to control the error that is thrown.
This will be implemented as another function you can pass along with the alternate stderr handler function. This does not break backwards compatibility of the API.
Also, note that an error can already be thrown from the stderr handler.
This should all be documented.
Explain bash --logon -c. Find a good example.
Deno has a "tee" mechanism. Let's embed that into this and make it simple to use.
The stderr processing function needs to pass bytes, not strings. This is because passing strings means we are buffering until a linefeed, which works for some output but not others. The default implementation should just write the bytes to stderr as they are read.
I cannot fix this immediately, so I am going to hang on to this and keep an eye on the related ticket for a while.
May be related to Aborting a CompressionStream/DecompressionStream leaks a resource #14212.
I have tried several ways to inject the ExitCodeError
of the process into the
stream, including using flush
of TransformStream
. I settled on converting
the stream to AsyncIterableIterator
, throwing the error after the stream had
been fully processed, and then converting all that back to ReadableStream
-
because that mechanism is conceptually straightforward and should just work.
The error:
ERRORS
Non-zero exit code => ./tests/exitcode-errors/errors-1.test.ts:4:6
error: Leaking resources:
- A text decoder (rid 5) was created during the test, but not finished during the test.
Close the text decoder by calling `textDecoder.decode('')` or `await textDecoderStream.readable.cancel()`.
The error is being suppressed where it occurs using:
sanitizeResources: false,
in the test definition.
Separation of error propagation logic from data pump seems to be causing a race condition where occasionally one of the error tests will fail. I think I need a redesign to use a queuing strategy. That would probably clean up a lot of incidental complexity as well, but is a lot of rework.
When the outputs of a process have been completely consumed, the process and its resources are closed automatically. At this point, we won't need to close them again, so we can remove their references in the associated ProcGroup.
This covers the case where someone uses a single ProcGroup to run lots of processes without closing.
We have support for arrays and AsyncIterables, but not for Iterables. It seems like there should be handlers specifically for Iterable, just for the sake of completeness.
To get stderr to be near real time, it needs to be unbuffered.
Deno has deprecated Deno.run
in favor of a new API, Deno.Command
, which is in many ways a major improvement over the old API. It is, however, quite a disruptive redesign.
I have been meaning to look at simplifying my own API. Perhaps this is the time.
I am going to attempt a redo with the idea that I can make this a lot easier. Let's see how it goes.
I'd like to be able to use map and filter without wrapping with asynciter, and I would also like to be able to run another process from the output of a process.
I need to find a good example for the README. I would like it to demonstrate piping bytes and processing string data, but be simple enough to read. It would be good if there were an analogous bash script to compare it to. Might be good to use stdin for input as well.
I'd like to be able to shadow-run a process, sending stderr to Deno stderr and stdout to Deno stdout, and no stdin. This is a common use case for running CI/CD processes.
I would also like to be able to decorate the outputs with color and time, and perhaps a label.
Add unbuffered options for Byte Iterable. Extend these to unbuffered string options.
This is probably just allowing the caller to provide a function that examines the exit code and throws or doesn't.
Add a documentation section for short-form.
Also write the code-level comments.
Current line separators are hard coded to "\n" in some places. These need to be changed to recognize the OS and use the appropriate line ending.
Modify the API to allow no group to be defined. In this case, a global group will be used. This works fine for processes that clean up correctly and for short runs. With that boilerplate gone, it feels a lot more like an extended bash script.
I fetch the Node.js nightly binary release, extract node
executable and run that extracted node
executable without using npm
or node_modules
whatsoever https://github.com/guest271314/download-node-nightly-executable.
I need a way to support passing messages from the stdin side of the process so that I can implement persistent worker processes.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.