fleipold / jproc Goto Github PK

I am trying to write a code to commit with git but it is working sometimes and not working the other time. Here is the code for it:

ProcResult result = new ProcBuilder("git")
                .withArg("init")
                .withWorkingDirectory(new File(dir))
                .run();

    result = new ProcBuilder("git")
            .withArgs("add",".")
            .run();
    result = new ProcBuilder("git")
            .withArgs("commit","-m").withArg("1st commit")
            .run();
    System.out.println("Successfully commited");

I tried something like the given code below but it still doesn't work:

result = new ProcBuilder("git")
            .withArgs("commit","-m").withArg(""+(char)34+"1st commit"+(char)34+"")
            .run();

It throws the following exception:

Exception in thread "main" org.buildobjects.process.ExternalProcessFailureException: External process 'git commit -m '1st commit'' returned 1 after 126ms

at org.buildobjects.process.Proc.<init>(Proc.java:126)
at org.buildobjects.process.ProcBuilder.run(ProcBuilder.java:205)
at ProjectInitializerAutomation.ProjectInitializerAutomation.Local.command(Local.java:90)
at ProjectInitializerAutomation.ProjectInitializerAutomation.Local.createFolder(Local.java:58)
at ProjectInitializerAutomation.ProjectInitializerAutomation.Local.create(Local.java:39)
at ProjectInitializerAutomation.ProjectInitializerAutomation.Driver.main(Driver.java:21)``

Currently all the convenience methods returning or taking strings as input and output use the JVMs default encoding.
It would be nice to have alternative methods that take an encoding.
Also, it should be considered whether the default should be hard-wired to UTF-8

Describe the bug

When a timeout occurs, the behavior is correct but, sometimes, a race condition happens and an unhandled exception is thrown.

To Reproduce

public class TimeoutTest {

    public static void main(String[] args) {
        ProcBuilder builder = new ProcBuilder("bash",  "-i", "-c", "echo hello").withTimeoutMillis(1);

        try {
            builder.run();
        } catch (TimeoutException ex) {
            //OK
        }
    }
}

Expected behavior

No uncaught exception should be printed in stderr.

Actual Behaviour

These traces are printed by stderr

Exception in thread "Thread-1" java.lang.RuntimeException: ${END}
	at org.buildobjects.process.Proc.dispatch(Proc.java:168)
	at org.buildobjects.process.ByteArrayConsumptionThread$1.run(ByteArrayConsumptionThread.java:36)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.InterruptedException
	at java.base/java.util.concurrent.locks.ReentrantLock$Sync.lockInterruptibly(ReentrantLock.java:159)
	at java.base/java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:372)
	at java.base/java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:332)
	at org.buildobjects.process.Proc.dispatch(Proc.java:166)
	... 2 more
Exception in thread "Thread-0" java.lang.RuntimeException: ${END}
	at org.buildobjects.process.Proc.dispatch(Proc.java:168)
	at org.buildobjects.process.StreamCopyConsumptionThread$1.run(StreamCopyConsumptionThread.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.InterruptedException
	at java.base/java.util.concurrent.locks.ReentrantLock$Sync.lockInterruptibly(ReentrantLock.java:159)
	at java.base/java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:372)
	at java.base/java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:332)
	at org.buildobjects.process.Proc.dispatch(Proc.java:166)
	... 2 more

I didn't expect any uncaught exception.

Context

I did use Ubuntu.

I am trying to migrate some code from java.lang.ProcessBuilder to ProcBuilder and have a few questions regarding env var handling.

1. How can I control whether or not environment variables are inherited from the parent process?
   In java.lang.ProcessBuilder the default is to inherit all env vars and I can prevent this with a single call to:
   processBuilder.environment().clear()

2. How can I set multiple env vars at once. Having to iterate over all vars and using the withVar(key, value) builder method defeats the purpose of fluent builder methods. So I am looking for a withVars(Map<String,String>) builder method or similar.

Thanks in advance!

Currently the message of the ExternalProcessFailureException only contains STDERR.
In the Unix tradition tools tend to print error messages on STDERR, but as one our users recently discovered git returns the error message on STDOUT.

In such a case having only STDERR in the message is quite unhelpful. Also, the exception object doesn't contain STDOUT.

A solution should return the relevant information and in case both STDERR and STDOUT contain output should make it obvious what is what.

There is another feature lurking which is limiting the length of the exception message by cutting off excessive output.

There is a getProcString() method in ProcResult,but how can i log the command before ProcBuilder.run()?

Hi,
I would like to execute some commands with JProc library and meet some problem, after studied the tutorial still can't get the solution, so ask some help here.

My command is simple with some pipes, for example,
the simple command is:

# adb devices
List of devices attached
emulator-5554	device

and then I would like to add one pipe:

#adb devices | tail -n +1
emulator-5554	device

and then finally I would like to add more one pipe:

#adb devices | tail -n +2 | awk '{print $1}'
emulator-5554

so my script is:

ByteArrayOutputStream output = new ByteArrayOutputStream();
        ProcResult result = new ProcBuilder("adb")
                .withArgs("devices")
                .withOutputStream(output).run();
        System.out.println(output.toString());
        new ProcBuilder("tail")
                .withArgs("-n", "+2")
                .withOutputStream(output)
                .run();
        System.out.println(output.toString());
        new ProcBuilder("awk")
                .withArgs("'{print $1}'")
                .withOutputStream(output)
                .run();
        System.out.println(output.toString());

it doesn't work.

and also

ByteArrayOutputStream output = new ByteArrayOutputStream();
        ProcResult result = new ProcBuilder("adb")
                .withArgs("devices")
                .withOutputConsumer(stream -> new ProcBuilder("tail")
                        .withArgs("-n", "+2")
                        .withOutputConsumer(stream1 -> {
                            new ProcBuilder("awk")
                                    .withArgs("'{print $1}'")
                                    .withOutputStream(output).run();
                        }).run())
                .run();
        System.out.println(output.toString());

it also doesn't work.

Could you help to provide some solution for my question?
Thanks.

I have just started using jproc, and for most things I'm finding it easy to use, and trouble-free. However there are some commands that I run that are expected to return a non-zero exit, even when they work correctly. What I'm thinking about is a call to Proc that includes a flag as to whether the exit code should be checked. If this flag is false, then don't even look at the exit code in line 77 of Proc.java.

I would be happy to fork the code and try to deliver a pull request to you if you agree.

Hi,

I was wondering if this library allows to start a process with a command, it stays active, and then you keep interacting with it over time with multiple commands and each time you get the output?

I see .withInputStream option in documentation, will it just use the inputstream as it is at creation time, or I can keep adding to inputstream to keep interacting with the process?

Thank you

It would be nice to have process' stderr available to client like stdout is:

• via ProcResult
• via ProcBuilder setter like ProcBuilder.withErrorStream

Currently we can capture stderr only via ExternalProcessFailureException.

Hello :)
I am using the latest 2.6.2 jproc release.
I am trying to execute command "gpg --import C://path/public.key" to import helm public key in java code using Jproc by this way:

ProcResult pr = new ProcBuilder("gpg").withArgs("--import", "C://path/public.key").run();
Command executing normally with exit code 0, but expected console output instead of being in "output" field of ProcResult object, located in "err" field as byte stream.
When I using pr.getErrorString() to see it, I see normal console output : "gpg: key C17701F951693F1F: public key imported".
I'm sure the console output gets there by mistake.
Thank you!

Just an FYI, I made this library into a modular library, compiled with Java 11. I figured since you have it set for Java 1.7, you clearly have backwards compatibility in mind, so I just forked the project and converted it then published it.

Just thought I'd let you know.

Mike

In my current project I start a long running process and I need to send it data through the standard input sparingly.

However there seems to be no way to do that as you stop reading the InputStream and close the Process's underlying OutputStream as soon as the initial reading is completed.

cannot jump to other directories or drives

Hello,

In pom.xml it is stated that this project is distributed under MIT license:

    <licenses>
        <license>
            <name>MIT License</name>
            <url>http://www.opensource.org/licenses/mit-license.php</url>
            <distribution>repo</distribution>
        </license>
    </licenses>

While project itself contains BSD license in LICENSE file.

Please clarify.

Describe the bug
The build is currently running on .org, which has stopped doing builds and is asking to migrate to .com version of the site.

Automatic builds are thus stalled.

To Reproduce

A description/ code snippet
N/A

Expected behavior
The build should happen

Actual Behaviour

• What happened actually?
• How was it different from what you expected to happen?

Explained above

Context

• Is there an exception/ output? Please paste the relevant bits into the ticket.
• What operating system did you use?
  N/A

implementation 'org.buildobjects.jproc:2.6.2'

import org.buildobjects.jproc;
wrong

https://github.com/fleipold/jproc/blob/master/src/main/java/org/buildobjects/process/ByteArrayConsumptionThread.java#L1
import org.buildobjects.process;
wrong

I just don't understand java, why are things so needlessly complicated.

This problem has been elusive for me using the default Java ProcessBuilder and after considering alternate libraries, this one looked to be the most promising, but this library expresses the problem a little differently and I am looking for any insights where my knowledge falls short.

I'm working on a wrapper for the popular command line tool, yt-dlp. The program offers an array of output, including file download progress which I am hoping to extract from the output of the program and apply it to a JavaFX ProgressBar.

When I use the Java ProcessBuilder / Process classes, attempting to get the output stream from the command that is executed ends up blowing straight past the loop that monitors the output stream but will block with with process.waitFor() method.

Using jproc, I set up the process like this:

System.out.println("Building Downloader");
String[] arguments = new String[]{"--progress", link, "-o", outputFileName};
ProcBuilder proc = new ProcBuilder(dirOfYt_dlp + yt_dlpProgramName)
    .withArgs(arguments)
    .withOutputConsumer(new StreamConsumer() {
        @Override
        public void consume(InputStream stream) throws IOException {
            try (BufferedReader reader = new BufferedReader(new InputStreamReader(stream))) {
                String line;
                Pattern p = Pattern.compile("(\\[download]\\s+)(\\d+\\.\\d+)(%)");
                System.out.println("Watching Stream");
                while((line = reader.readLine()) != null) {
                    System.out.println(line);
                    Matcher m = p.matcher(line);
                    if (m.matches()) {
                        String progressPercent = m.group(2);
                        double progress = Double.parseDouble(progressPercent) / 100.0;
                        GUI.setDownloadProgress(progress);
                    }
                }
                System.out.println("Stream Done!");
            }
        }
    });
System.out.println("Downloading");
System.out.println(proc.getCommandLine());
ProcResult result = proc.run();
System.out.println("Finished!");

Here is the output from that code:

Building Downloader
Downloading
./src/main/resources/yt-dlp_macos --progress -P /Users/michael/Downloads/Video/ https://www.youtube.com/watch?v=tXsQJhoauxc -o 'The Lumineers - The Ballad Of Cleopatra.mp4'
Watching Stream
[youtube] Extracting URL: https://www.youtube.com/watch?v=tXsQJhoauxc
[youtube] tXsQJhoauxc: Downloading webpage
Stream Done!

At that point, it continues to block indefinitely and the file never gets downloaded.

However, using the standard ProcessBuilder and Process classes, implementing the process.waitFor() method does block at the same point after blowing past the attempt to get the output from the command, which does not yield any output though the program does download the file while it's blocking.

I have also tried setting up the process like this:

proc = new ProcBuilder(dirOfYt_dlp + yt_dlpProgramName).withArgs(arguments);

And all that does is block when it is .run() and it blocks indefinitely without finishing or downloading anything.

So the differences I am seeing between Javas Process and jproc:

jproc gets output from the program but the output stream clearly receives a null which causes that loop to stop, but that doesn't stop the process from executing because proc.run() continues to block indefinitely.

Process gets nothing from the programs output and it just blows past any attempts to get output but then using process.waitFor() blocks until the program is done downloading the file where it then properly exits and yields an exit level that can be read.

I have no idea what the program is doing that it kicks out a null in the output stream yet continues doing SOMETHING. If I were to make a guess, I would say that it spawns another process to complete the download. But if that were the case, then one would expect the process to no longer block since that thread would be destroyed in favor of a new thread that it spawned, but I can't be 100% on that ... I suppose its possible that it spawns a new process within the same thread, but if that is the case, how do we latch onto that process in order to get its output?

I made a screen recording using the output from proc.getCommandLine() which shows the timing of events when this program runs ... I don't know if that timing is relevant, but seeing the command in action might be useful.

Here is the screen capture

Any insight or thoughts about this would be much appreciated.

Currently the jproc .jar file is 23kb, and the commons-io one is 209kb. That single dependency increases the size by 10x, and that's all for a single function call in jproc:

This function isn't even that long, as it's a total of about 10 function lines of java spread across 5 methods:

public static byte[] toByteArray(final InputStream input) throws IOException {
        try (final ByteArrayOutputStream output = new ByteArrayOutputStream()) {
            copy(input, output);
            return output.toByteArray();
        }
}
public static int copy(final InputStream input, final OutputStream output) throws IOException {
        final long count = copyLarge(input, output);
        if (count > Integer.MAX_VALUE) {
            return -1;
        }
        return (int) count;
}
public static long copyLarge(final InputStream input, final OutputStream output)
            throws IOException {
        return copy(input, output, DEFAULT_BUFFER_SIZE);
}
public static long copy(final InputStream input, final OutputStream output, final int bufferSize)
            throws IOException {
        return copyLarge(input, output, new byte[bufferSize]);
}
public static long copyLarge(final InputStream input, final OutputStream output, final byte[] buffer)
            throws IOException {
        long count = 0;
        int n;
        while (EOF != (n = input.read(buffer))) {
            output.write(buffer, 0, n);
            count += n;
        }
        return count;
}

Would you consider simply copying these methods (or a compact version) from IOUtils to jproc?

Dear fleipold,

Very thank you for creating this lib. It is very helpful.

There is one issue which may need to be fixed.

After running a simple example code:

String output = ProcBuilder.run("echo", "Hello World!");
System.out.println("output = " + output);

I got exception:

Exception in thread "main" org.buildobjects.process.StartupException: Could not startup process 'echo "Hello World!"'.
	at org.buildobjects.process.Proc.<init>(Proc.java:91)
	at org.buildobjects.process.ProcBuilder.run(ProcBuilder.java:205)
	at org.buildobjects.process.ProcBuilder.run(ProcBuilder.java:226)
	at com.philips.iscm.workutils.collins.epub.WordListMdFile.main(WordListMdFile.java:23)
Caused by: java.io.IOException: Cannot run program "echo": CreateProcess error=2, 系统找不到指定的文件(which means system can't find the file or directory)。
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at java.lang.Runtime.exec(Runtime.java:620)
	at org.buildobjects.process.Proc.<init>(Proc.java:77)
	... 3 more
Caused by: java.io.IOException: CreateProcess error=2, 系统找不到指定的文件(which means system can't find the file or directory)。
	at java.lang.ProcessImpl.create(Native Method)
	at java.lang.ProcessImpl.<init>(ProcessImpl.java:386)
	at java.lang.ProcessImpl.start(ProcessImpl.java:137)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 5 more

My env:
OS: windows 10
IDE: IDEA
JDK: 8

Setting a timeout interval of 0ms fails immediately rather that running for as long as necessary.

I need to ensure the running program is killed if the Thread the Proc is called in is Interrupted. I expected it to be the case as the call is blocking, but it is not.

Hi. I use jproc to create a user and I want to set a password.
My code is like
new ProBuilder("passwd").withInput("XXXXXXX").withInput("XXXXXXX").run()
and I got this
STDERR: New password: Retype new password: Password change aborted. STDERR: New password: Password change aborted. STDERR: New password: Password change aborted. STDERR: passwd: Have exhausted maximum number of retries for service
So, Is there a way to do this with jproc ?
Thank you.

Describe the bug

Maven repo contains sources en javadoc, but doesn't contain the jar file or the pom file.

To Reproduce

You can see the files in https://repo1.maven.org/maven2/org/buildobjects/jproc/2.8.1/

Expected behavior

Actual Behaviour

• What happened actually?
• How was it different from what you expected to happen?

Context

• Is there an exception/ output? Please paste the relevant bits into the ticket.
• What operating system did you use?