Parametrized benchmarks about hyperfine

[sharkdp]

Okay, thanks for the clarification. I would like to keep this out for now. I'm happy to discuss this in a separate ticket though, if there is a real use case for this.

[sharkdp]

Another option could be to export an environment variable:

> hyperfine --parameter num_threads 1..8 'make -j $num_threads'

[stevepentland]

This seems like an interesting idea. I personally prefer the first style to the second, but since we'd have to parse apart the input anyway, why not just add --range-min and --range-max values to make it more explicit. For the second example, it may offer some benefit to at least offer a parameter that will set/unset environment variables during runs so that they can be used without users having to remember to manually add and remove them.

[sharkdp]

I personally prefer the first style to the second, but since we'd have to parse apart the input anyway

I think with the second style we would not have to parse the command. We would simply set the environment variable in the current environment and then call the command (via the shell, as usual). Since we spawn a sub-process, the environment variable would be available in the child (the shell and all children further down). This would work in a similar way to:

> export sleep_time=0.2
> hyperfine 'sleep $sleep_time'  # note the single quotes.
                                 # $sleep_time is not expanded when we call hyperfine.

why not just add --range-min and --range-max values to make it more explicit.

That sounds reasonable.

For the second example, it may offer some benefit to at least offer a parameter that will set/unset environment variables during runs so that they can be used without users having to remember to manually add and remove them.

I'm not sure I understand.

[sharkdp]

I'm currently working on this...

[stevepentland]

I'm not sure I understand.

I was just thinking that it may be beneficial to offer user's the opportunity to pass in environment variables they want to have set in the child processes we run their applications in. In many cases these could be given as part of the command, but that both makes the original command argument longer and harder for the user to quickly change between runs, and also is not as portable between platforms. On PowerShell, setting environment variables isn't quite as easy as on Linux/macOS where VAR=val <cmd> will work.

[sharkdp]

I was just thinking that it may be beneficial to offer user's the opportunity to pass in environment variables they want to have set in the child processes we run their applications in.

To be honest, I'm not sure why we should add this to hyperfine. As you said, this will work just fine when done from the shell (because the environment will be inherited by the child processes). Even in PowerShell, it would only be

$env:MyVariable = "SomeValue"
hyperfine ...

[stevepentland]

True, but it offers the ability to test a particular env solely for the command under test, without polluting the user's own environment. It's not a huge thing, more of a creature comfort.