cysharp / processx

Simplify call an external process with the async streams in C# 8.0.

ProcessX

ProcessX simplifies call an external process with the aync streams in C# 8.0 without complex Process code. You can receive standard output results by await foreach, it is completely asynchronous and realtime.

Also provides zx mode to write shell script in C#, details see Zx section.

Table of Contents

• Getting Started
• Cancellation
• Raw Process/StdError Stream
• Read Binary Data
• Change acceptable exit codes
• Zx
• Reference
• Competitor
• License

Getting Started

Install library from NuGet that support from .NET Standard 2.0.

PM> Install-Package ProcessX

Main API is only Cysharp.Diagnostics.ProcessX.StartAsync and throws ProcessErrorException when error detected.

• Simple, only write single string command like the shell script.
• Asynchronous, by C# 8.0 async streams.
• Manage Error, handling exitcode and stderror.

using Cysharp.Diagnostics; // using namespace

// async iterate.
await foreach (string item in ProcessX.StartAsync("dotnet --info"))
{
    Console.WriteLine(item);
}

// receive string result from stdout.
var version = await ProcessX.StartAsync("dotnet --version").FirstAsync();

// receive buffered result(similar as WaitForExit).
string[] result = await ProcessX.StartAsync("dotnet --info").ToTask();

// like the shell exec, write all data to console.
await ProcessX.StartAsync("dotnet --info").WriteLineAllAsync();

// consume all result and wait complete asynchronously(useful to use no result process).
await ProcessX.StartAsync("cmd /c mkdir foo").WaitAsync();

// when ExitCode is not 0 or StandardError is exists, throws ProcessErrorException
try
{
    await foreach (var item in ProcessX.StartAsync("dotnet --foo --bar")) { }
}
catch (ProcessErrorException ex)
{
    // int .ExitCode
    // string[] .ErrorOutput
    Console.WriteLine(ex.ToString());
}

Cancellation

to Cancel, you can use WithCancellation of IAsyncEnumerable.

// when cancel has been called and process still exists, call process kill before exit.
await foreach (var item in ProcessX.StartAsync("dotnet --info").WithCancellation(cancellationToken))
{
    Console.WriteLine(item);
}

timeout, you can use CancellationTokenSource(delay).

using (var cts = new CancellationTokenSource(TimeSpan.FromSeconds(1)))
{
    await foreach (var item in ProcessX.StartAsync("dotnet --info").WithCancellation(cts.Token))
    {
        Console.WriteLine(item);
    }
}

Raw Process/StdError Stream

In default, when stdError is used, buffering error messages and throws ProcessErrorException with error messages after process exited. If you want to use stdError in streaming or avoid throws error when process using stderror as progress, diagnostics, you can use GetDualAsyncEnumerable method. Also GetDualAsyncEnumerable can get raw Process, you can use ProcessID, StandardInput etc.

// first argument is Process, if you want to know ProcessID, use StandardInput, use it.
var (_, stdOut, stdError) = ProcessX.GetDualAsyncEnumerable("dotnet --foo --bar");

var consumeStdOut = Task.Run(async () =>
{
    await foreach (var item in stdOut)
    {
        Console.WriteLine("STDOUT: " + item);
    }
});

var errorBuffered = new List<string>();
var consumeStdError = Task.Run(async () =>
{
    await foreach (var item in stdError)
    {
        Console.WriteLine("STDERROR: " + item);
        errorBuffered.Add(item);
    }
});

try
{
    await Task.WhenAll(consumeStdOut, consumeStdError);
}
catch (ProcessErrorException ex)
{
    // stdout iterator throws exception when exitcode is not 0.
    Console.WriteLine("ERROR, ExitCode: " + ex.ExitCode);

    // ex.ErrorOutput is empty, if you want to use it, buffer yourself.
    // Console.WriteLine(string.Join(Environment.NewLine, errorBuffered));
}

Read Binary Data

If stdout is binary data, you can use StartReadBinaryAsync to read byte[].

byte[] bin = await ProcessX.StartReadBinaryAsync($"...");

Change acceptable exit codes

In default, ExitCode is not 0 throws ProcessErrorException. You can change acceptable exit codes globally by ProcessX.AcceptableExitCodes property. Default is [0].

Zx

like the google/zx, you can write shell script in C#.

// ProcessX and C# 9.0 Top level statement; like google/zx.

using Zx;
using static Zx.Env;

// `await string` execute process like shell
await "cat package.json | grep name";

// receive result msg of stdout
var branch = await "git branch --show-current";
await $"dep deploy --branch={branch}";

// parallel request (similar as Task.WhenAll)
await new[]
{
    "echo 1",
    "echo 2",
    "echo 3",
};

// you can also use cd(chdir)
await "cd ../../";

// run with $"" automatically escaped and quoted
var dir = "foo/foo bar";
await run($"mkdir {dir}"); // mkdir "/foo/foo bar"

// helper for Console.WriteLine and colorize
log("red log.", ConsoleColor.Red);
using (color(ConsoleColor.Blue))
{
    log("blue log");
    Console.WriteLine("also blue");
    await run($"echo {"blue blue blue"}");
}

// helper for web request
var text = await fetchText("http://wttr.in");
log(text);

// helper for ReadLine(stdin)
var bear = await question("What kind of bear is best?");
log($"You answered: {bear}");

// run has some variant(run2, runl, withTimeout, withCancellation)
// runl returns string[](runlist -> runl)
var sdks = await runl($"dotnet --list-sdks");

writing shell script in C# has advantage over bash/cmd/PowerShell

• Static typed
• async/await
• Code formatter
• Clean syntax via C#
• Powerful editor environment(Visual Studio/Code/Rider)

Zx.Env has configure property and utility methods, we recommend to use via using static Zx.Env;.

using Zx;
using static Zx.Env;

// Env.verbose, write all stdout/stderror log to console. default is true.
verbose = false;

// Env.shell, default is Windows -> "cmd /c", Linux -> "(which bash) -c";.
shell = "/bin/sh -c";

// Env.terminateToken, CancellationToken that triggered by SIGTERM(Ctrl + C).
var token = terminateToken;

// Env.fetch(string requestUri), request HTTP/1, return is HttpResponseMessage.
var resp = await fetch("http://wttr.in");
if (resp.IsSuccessStatusCode)
{
    Console.WriteLine(await resp.Content.ReadAsStringAsync());
}

// Env.fetchText(string requestUri), request HTTP/1, return is string.
var text = await fetchText("http://wttr.in");
Console.WriteLine(text);

// Env.sleep(int seconds|TimeSpan timeSpan), wrapper of Task.Delay.
await sleep(5); // wait 5 seconds

// Env.withTimeout(string command, int seconds|TimeSpan timeSpan), execute process with timeout. Require to use with "$".
await withTimeout($"echo foo", 10);

// Env.withCancellation(string command, CancellationToken cancellationToken), execute process with cancellation. Require to use with "$".
await withCancellation($"echo foo", terminateToken);

// Env.run(FormattableString), automatically escaped and quoted. argument string requires to use with "$"
await run($"mkdir {dir}");

// Env.run(FormattableString), automatically escaped and quoted. argument string requires to use with "$"
await run($"mkdir {dir}");

// Env.runl(FormattableString), returns string[], automatically escaped and quoted. argument string requires to use with "$"
var l1 = runl("dotnet --list-sdks");

// Env.process(string command), same as `await string` but returns Task<string>.
var t = process("dotnet info");

// Env.processl(string command), returns Task<string[]>.
var l2 = processl("dotnet --list-sdks");

// Env.ignore(Task), ignore ProcessErrorException
await ignore(run($"dotnet noinfo"));

// ***2 receives tuple of result (StdOut, StdError).
var (stdout, stderror) = run2($"");
var (stdout, stderror) = runl2($"");
var (stdout, stderror) = withTimeout2($"");
var (stdout, stderror) = withCancellation2($"");
var (stdout, stderror) = process2($"");
var (stdout, stderror) = processl2($"");

await string does not escape argument so recommend to use run($"string") when use with argument.

If you want to more colorize like Chalk on JavaScript, Cysharp/Kokuban styler for .NET ConsoleApp will help.

Reference

ProcessX.StartAsync overloads, you can set workingDirectory, environmentVariable, encoding.

// return ProcessAsyncEnumerable
StartAsync(string command, string? workingDirectory = null, IDictionary<string, string>? environmentVariable = null, Encoding? encoding = null)
StartAsync(string fileName, string? arguments, string? workingDirectory = null, IDictionary<string, string>? environmentVariable = null, Encoding? encoding = null)
StartAsync(ProcessStartInfo processStartInfo)

// return (Process, ProcessAsyncEnumerable, ProcessAsyncEnumerable)
GetDualAsyncEnumerable(string command, string? workingDirectory = null, IDictionary<string, string>? environmentVariable = null, Encoding? encoding = null)
GetDualAsyncEnumerable(string fileName, string? arguments, string? workingDirectory = null, IDictionary<string, string>? environmentVariable = null, Encoding? encoding = null)
GetDualAsyncEnumerable(ProcessStartInfo processStartInfo)

// return Task<byte[]>
StartReadBinaryAsync(string command, string? workingDirectory = null, IDictionary<string, string>? environmentVariable = null, Encoding? encoding = null)
StartReadBinaryAsync(string fileName, string? arguments, string? workingDirectory = null, IDictionary<string, string>? environmentVariable = null, Encoding? encoding = null)
StartReadBinaryAsync(ProcessStartInfo processStartInfo)

// return Task<string> ;get the first result(if empty, throws exception) and wait completed
FirstAsync(CancellationToken cancellationToken = default)

// return Task<string?> ;get the first result(if empty, returns null) and wait completed
FirstOrDefaultAsync(CancellationToken cancellationToken = default)

// return Task
WaitAsync(CancellationToken cancellationToken = default)

// return Task<string[]>
ToTask(CancellationToken cancellationToken = default)

// return Task
WriteLineAllAsync(CancellationToken cancellationToken = default)

Competitor

• Tyrrrz/CliWrap - Wrapper for command line interfaces.
• jamesmanning/RunProcessAsTask - Simple wrapper around System.Diagnostics.Process to expose it as a System.Threading.Tasks.Task.
• mayuki/Chell Write scripts with the power of C# and .NET.

License

This library is under the MIT License.

processx's Issues

run process with a CancellationToken

it could be nice if methods could allow passing a CancellationToken so they can be cancelled from the outside.

How to change directory and run a program in that directory

My console app is in E: but I need to run a program in D: from cmd. I have tried a few ways (cd.. , cd /d, etc) but nothing seems to work. Any suggestion

issue with newlines in commands

I'm writing a script to auto-setup a git repo (converted from an old bash script), and I noticed when I create a multi-line commit message, ProcessX messes up newlines.

Repro:

const string path = "debugMultiline";
await $"git init {path}";
await $"cd {path}";
await "git commit --allow-empty -m \"initial commit\"";
await "git commit --allow-empty -m \"await multi\nline\"";
System.Diagnostics.Process.Start("git", "commit --allow-empty -m \"Process.Start multi\nline\"");
await Cysharp.Diagnostics.ProcessX.StartAsync("git commit --allow-empty -m \"ProcessX.Start multi\nline\"").WriteLineAllAsync();

output:

[main b0ec9e6] initial commit
[main e7fdefa] await multi
[main a6db33f] Process.Start multi line
[main 7603f25] ProcessX.Start multi line

You can tell right away in the output that the second commit message is truncated (missing trailing "line"), which can be double-checked in a any git client).
Process.Start doesn't have that issue (3rd commit), nor does ProcessX.Start (4th commit).

I'm using the latest version (1.5.5) of the Nuget package.

Add standard error redirect option

Currently(rev: 09f95cb), ProcessX.StartAsync throws error when process output stderr.
This may be unexpected result in apps using stderr as logging or progress.

So I propose adding option to redirecting stderr.
In particular, add System.IO.Stream or System.Buffers.IBufferWriter<byte>(or char?) to ProcessX.StartAsync API

Is there anyway to set acceptable exit code?

I have a 3rd party executable that returns 1 when it completes successfully.
And because of ExitCode=1, ProcessX throws ProcessErrorException.
I can handle this situation by try-catch, but I'm happy if there is a way to set "acceptable" ExitCode.

Exception when running with ToTask()

When I run this line:

            var psi = CreateStartInfo(...);

            var lines = await Cysharp.Diagnostics.ProcessX
                .StartAsync(psi)
                .ToTask(token);            

            var text = string.Join("\r\n", lines);

I get this exception:

Cysharp.Diagnostics.ProcessErrorException : Process returns error, ExitCode:0

AsyncOperation`1.GetResult(Int16 token)
ProcessAsyncEnumerator.MoveNextAsync()
ProcessAsyncEnumerable.ToTask(CancellationToken cancellationToken)
ProcessAsyncEnumerable.ToTask(CancellationToken cancellationToken)
...

When calling syncronously, everything works fine.

Steps to reproduce:

• clone https://github.com/vpenades/GltfValidator.git
• run test Test1Async

Reading large byte data

I'm trying to call process that for example compresses large data or does some video processing. This process outputs large stream of data that I would like to process it in C# ideally as some sort of a stream.

Is there a way to achieve this in ProcessX or is it something where I have should go to .NET Process class and read standard output myself?