tekwizely / run Goto Github PK

I just upgraded to run version v0.11.0 on Mac Ventura 13.1 installed with brew and started getting errors for command names containing a dash (-) or underscore (_).

If I am reading the documentation correctly this should work and did in previous versions.

$ run version
run v0.11.0

With the following sample configuration:

# Set default shell for all actions
.SHELL = bash

EXPORT RUNFILE := "${.RUNFILE}"

## Check configuration
config-check:
    echo "RUNFILE: $RUNFILE"

I now get the following error:

$ run list
run: Runfile:7.1: expecting runfile statement

If I remove the dash:

# Set default shell for all actions
.SHELL = bash

EXPORT RUNFILE := "${.RUNFILE}"

## Check configuration
configcheck:
    echo "RUNFILE: $RUNFILE"

I get the correct result:

$ run list
Commands:
  list           (builtin) List available commands
  help           (builtin) Show help for a command
  version        (builtin) Show run version
  configcheck    Check configuration

Having the built-in version command available when a Runfile is executed in shebang mode feels awkward:

• The user may not be aware that they are executing run under the hood so displaying "run v0.0.0" might be conusing.

• The presence of the built in command means that the Runfile itself cannot implement its own version command.

Consider this Runfile

# justprep/test/Runfile

module_aaa := "modules/aaa.run"
module_bbb := "modules/bbb.run"

EXPORT module_aaa, module_bbb

##
# Say hi
hello:
  echo "Hello World"

##
# List all tasks including the modules
list_all:
  run list
  run aaa
  run bbb

##
# Module aaa
aaa:
  run -r ${module_aaa} ${@}

##
# Module bbb
bbb:
  run -r ${module_bbb} ${@}

and two others aaa.run and bbb.run both in a modules directory.

# .../modules/aaa.run

##
# Module aaa magic
magic:
  echo 'Do you believe in Magic?'

##
# Do something magical in aaa
xyzzy:
  echo "Magic from aaa ..."
  echo "... watch me pull a ${1} out of my hat."

... and ...

# .../modules/bbb.run

##
# Module bbb magic
magic:
  echo 'Do you believe in Magic?'

##
# Do something magical in bbb
xyzzy:
  echo "Magic from bbb ..."
  echo "... watch me pull a ${1} out of my hat."

Here is a console session using those three files:

13:51:20 3.1.2p20 support_run s5:test $ run
using runfile: /Users/dewayne/Documents/sandbox/git_repos/madbomber/justprep/working/support_run/test/Runfile

Commands:
  list        (builtin) List available commands
  help        (builtin) Show help for a command
  version     (builtin) Show run version
  hello       Say hi
  list_all    List all tasks including the modules
  aaa         Module aaa
  bbb         Module bbb

Usage:
       run <command> [option ...]
          (run <command>)
  or   run help <command>
          (show help for <command>)

see 'run --help' for more information
13:51:30 3.1.2p20 support_run s5:test $ run list_all
Commands:
  list        (builtin) List available commands
  help        (builtin) Show help for a command
  version     (builtin) Show run version
  hello       Say hi
  list_all    List all tasks including the modules
  aaa         Module aaa
  bbb         Module bbb
using runfile: /Users/dewayne/Documents/sandbox/git_repos/madbomber/justprep/working/support_run/test/modules/aaa.run

Commands:
  list       (builtin) List available commands
  help       (builtin) Show help for a command
  version    (builtin) Show run version
  magic      Module aaa magic
  xyzzy      Do something magical in aaa

Usage:
       run <command> [option ...]
          (run <command>)
  or   run help <command>
          (show help for <command>)

see 'run --help' for more information
using runfile: /Users/dewayne/Documents/sandbox/git_repos/madbomber/justprep/working/support_run/test/modules/bbb.run

Commands:
  list       (builtin) List available commands
  help       (builtin) Show help for a command
  version    (builtin) Show run version
  magic      Module bbb magic
  xyzzy      Do something magical in bbb

Usage:
       run <command> [option ...]
          (run <command>)
  or   run help <command>
          (show help for <command>)

see 'run --help' for more information
13:51:41 3.1.2p20 support_run s5:test $ run aaa
using runfile: /Users/dewayne/Documents/sandbox/git_repos/madbomber/justprep/working/support_run/test/modules/aaa.run

Commands:
  list       (builtin) List available commands
  help       (builtin) Show help for a command
  version    (builtin) Show run version
  magic      Module aaa magic
  xyzzy      Do something magical in aaa

Usage:
       run <command> [option ...]
          (run <command>)
  or   run help <command>
          (show help for <command>)

see 'run --help' for more information
13:51:45 3.1.2p20 support_run s5:test $ run bbb
using runfile: /Users/dewayne/Documents/sandbox/git_repos/madbomber/justprep/working/support_run/test/modules/bbb.run

Commands:
  list       (builtin) List available commands
  help       (builtin) Show help for a command
  version    (builtin) Show run version
  magic      Module bbb magic
  xyzzy      Do something magical in bbb

Usage:
       run <command> [option ...]
          (run <command>)
  or   run help <command>
          (show help for <command>)

see 'run --help' for more information
13:51:52 3.1.2p20 support_run s5:test $ run aaa magic
Do you believe in Magic?
13:51:58 3.1.2p20 support_run s5:test $ run bbb magic
Do you believe in Magic?
13:52:04 3.1.2p20 support_run s5:test $ run aaa xyzzy rabbit
Magic from aaa ...
... watch me pull a rabbit out of my hat.
13:52:14 3.1.2p20 support_run s5:test $ run bbb xyzzy dinosaur
Magic from bbb ...
... watch me pull a dinosaur out of my hat.
13:52:33 3.1.2p20 support_run s5:test $

What I do in justprep is to turn a line that looks like this ...

module aaa := "modules/aaa.run

into lines that look like this ...

module_aaa := "modules/aaa.run
EXPORT module_aaa

##
# Module aaa
aaa:
  run -r ${module_aaa} ${@}

Originally posted by @MadBomber in #47 (comment)

Often, it's convenient to have a Runfile in a central place, like the root folder of a repository. I think it would be nice if run could search upwards to try find a Runfile if one is not found in the present directory, to make working in a subdirectory more convenient in that case.

Suggested workflow:

• if -r is found, then require that file to exist. Don't search any other path.
• if not, then search for Runfile in the current directory, and if it does not exist, keep searching upwards until a Runfile is found

We could also apply some other bounding here, like not continuing upwards if a VCS directory (.git, etc) is found, but perhaps that isn't really needed?

Great project. It will be awesome to have have syntax rules for Vim to enjoy highlighting for Runfile.

As I'm already using github.com/Shougo/context_filetype.vim vim plugin, it will be interesting to implement some regular-expressions to automatically highlight nested languages syntax within Runfiles.

Having to specify '#!' as the shell type shebang scripts feels awkward.

It should possible to check the first line of the command script and set the shell type accordingly.

NOTE: This check would only apply if no shell was explicitly given.

There are likely many ways to invision calling between commands within a runfile.

While the nuances of a built-in syntax are being considered, perhaps a simple addition would be to provide access to the Run binary and Runfile currently in use, allowing the user to invoke the runfile direclty within their command script.

Something like:

Runfile

## invokes test2
test1:
    ${RUN} -r ${RUNFILE} test2

## Printls Hello, world
test2:
    echo "Hello, world"

I was under the impression that RUN.BEFORE and RUN.AFTER worked differently. In fact, I thought that I could do the following (especially valid when you have multiple Runfiles):

The contents of ./commands/homebrew/Runfile:

# RUN.AFTER upgrade
upgrade-homebrew:
  echo "Upgrading Homebrew"

The contents of ./commands/aqua/Runfile:

# RUN.AFTER upgrade
upgrade-aqua:
  echo "Upgrading aqua"

The contents of ./Runfile:

INCLUDE **/Runfile

upgrade:
  echo "Upgrading..."

So, I naively hoped that I could wire in more commands in various ./commands/*/Runfile files without having to wire them in ./Runfile, so, if I would get the following result in the setup above:

$ run upgrade
Upgrading...
Upgrading Homebrew
Upgrading aqua

It came out I was wrong and that RUN.BEFORE and RUN.AFTER are just syntax sugar for "${RUN}" <command>, which is not enormously useful.

So, I propose adding the HOOK.BEFORE and HOOK.AFTER, so that the above could work if I replace RUN.AFTER with HOOK.AFTER.

Another option is to make upgrade call other commands using a wildcard, i.e.

# RUN.AFTER upgrade-*
upgrade:
  echo "Upgrading"

A couple of packages to look at for inspiration:

• https://github.com/avh4/binwrap
• https://github.com/sanathkr/go-npm

It would be nice if we could specify enumerated values for an option. This would keep the code DRY as it could save us the validation and keep it in sync with the help text.

Hi!

I wanted to use Bingo. It required Run. So I compiled it. Compilation works, but any Runfile with fail with this error:

exec: "/usr/bin/env": file does not exist

The culprit is this line

Possible solution: Check if GOOS == 'windows' and execute shell directly, without using /usr/bin/env.

Add supoprt for the dash (-) character in command names.

This would allow creation of commands like list-all, etc. which feels natural.

Originally posted by @xkcd386at in #67 (comment)

Hi

Apologies for the huge delay!

I tried this just now, as below, and I clearly misunderstood how RUNFILE_ROOTS is supposed to work. I'll read up a little more and try again.

Anyway here's my failed attempt to get RUNFILE_ROOTS to work:

$ run version
run 0.11.1 (build=v0.11.1 date=2023-01-24T17:09:23Z builder=goreleaser)
$ pwd
/home/x3/x/gits/run
$ cat ~/.runfile
hello:
    echo hi there
$ env|grep -i run
PWD=/home/x3/x/gits/run
$ RUNFILE_ROOTS=$HOME RUNFILE=.runfile run hello
run: ERROR: runfile '.runfile' not found: please create the file or specify an alternative

$ RUNFILE_ROOTS=$HOME run -r .runfile hello
run: ERROR: runfile '.runfile' not found: please create the file or specify an alternative

$ cd
$ RUNFILE=.runfile run hello
hi there

In order to actually use attributes in scripts, commands need to first convert them into environment variables, which often leads to verbose assignments like:

(assumes #55 merged)

export RUNFILE_DIR := ${.RUNFILE.DIR}

Perhaps we can make exporting attributes easier for the user:

Quick Export

A quick export could be as easy as:

##
# export .RUNFILE.DIR
#
main:
    echo $RUNFILE_DIR

ie. the quick export rules could be:

• Ignore the leading .
• Replace . with _

Export With Rename

We could also support an easy way to export with a different name:

##
# export .RUNFILE.DIR as RFD
#
main:
    echo $RFD

Support Variables Too?

The as syntax could also be used as a way to export standard variables with a different name.

Although I'm not sure how useful this would be?

Even if its not overly useful, we might want to support it just for symmetry with attributes

I cant find the project using the github search and need to scroll down through my github stars.

Might i suggest adding the following search tags to improve searchability;

run
runfile
makefile
make

I like run approach to scripts management.
I find it clever, and I think it can be much more understandable in a lot of situations.
However, what I would miss is something similar to prerequisities.
Since run does not "produce files", but instead "runs scripts", I would assume this would be a list of commands to be run before the choosen one.

For example:

hello-world: get-name
    echo "Hello, ${NAME}"

get-name:
    read _NAME
    export NAME=${_NAME}

Of course, this example makes no sense, but shows the idea.

I believe I have found an issue with version 0.11.1. Given the following Runfile, see the output for version 0.10.0 and 0.11.1

• Version 0.10.0 shows the help text and PHP output of the command.
• Version 0.11.1 does not show the help text or correct run PHP embedded code.

Runfile

# Set default shell for all actions
.SHELL = bash

EXPORT .RUNFILE


## 
# Check configuration
config-check:
    echo "RUNFILE: $RUNFILE"


##
# Run php embedded
# OPTION FLAG_contrived -a <arg> Contrived argument
run-php-embedded (php):
    <?= "PHP_VERSION_ID: " . PHP_VERSION_ID . "\n" ?>
    <?= "FLAG_contrived: {$_SERVER['FLAG_contrived']}\n" ?>

run version 0.10.0

$ run version
run 0.10.0 (build=v0.10.0 date=2022-11-10T20:57:42Z builder=goreleaser)
$ run list
Commands:
  list                (builtin) List available commands
  help                (builtin) Show help for a command
  version             (builtin) Show run version
  config-check        Check configuration
  run-php-embedded    Run php embedded
$ run config-check
RUNFILE: /Users/brunk/projects/testing/run/Runfile
$ run run-php-embedded -a asdf
PHP_VERSION_ID: 80201
FLAG_contrived: asdf

run version 0.11.1

$ /opt/homebrew/bin/run version
run v0.11.1
$ /opt/homebrew/bin/run list
Commands:
  list                (builtin) List available commands
  help                (builtin) Show help for a command
  version             (builtin) Show run version
  config-check        Check configuration
  run-php-embedded
$ /opt/homebrew/bin/run run-php-embedded -a asdf
run: ERROR: Runfile:16: command not found: php

Just a reminder to incorporate updating Run's Nix Package as part of our release cycle.

From discussion here:

• NixOS/nixpkgs#77081 (comment)

To contribute to nixpkgs (update a package for instance), we submit PR, such as the current one. You could find more details in the nixpkgs manual: https://nixos.org/manual/nixpkgs/stable/#submitting-changes-making-patches

Looks like command export statements don't have access to attributes.

Example:

Runfile

.TEST := "Hello, world"

##
# EXPORT test := ${.TEST}
test:
	echo $test

output

$ run test

expected output

$ run test

Hello, world

A script with an error at line 9

##
# Invokes hello
# EXPORT RUN := ${.RUN}
# EXPORT RUNFILE := ${.RUNFILE}
test:
# syntaxe error here <------
    "${RUN}" -r "${RUNFILE}" hello

hello:
    echo "Hello, World"

sh-4.4$ run test

Give this hard to understand message
run: non-EOF error returned from lexer, treating as EOF: 10:6: unexpected rune '"'

which is not easy to understand

If the last line of my Runfile isn't empty, run throws the following error:

panic: Lexer.Next: No rune available

goroutine 1 [running]:
github.com/tekwizely/go-parsing/lexer.(*Lexer).Next(0xc0000cc1e0, 0x521c00)
        /workspace/go/pkg/mod/github.com/tekwizely/go-parsing/[email protected]/lexer.go:159 +0x102
github.com/tekwizely/run/internal/lexer.lexCmdScriptLine(0xc0000bc020, 0xc0000cc1e0, 0x4f31e0)
        /workspace/run/internal/lexer/lexer.go:690 +0xe4
github.com/tekwizely/run/internal/lexer.(*LexContext).lex(0xc0000bc020, 0xc0000cc1e0, 0x4e91c0)
        /workspace/run/internal/lexer/lexer.go:41 +0x7e
github.com/tekwizely/go-parsing/lexer.(*tokenNexter).hasNext(0xc0000bc040, 0x7f6291d24460)
        /workspace/go/pkg/mod/github.com/tekwizely/go-parsing/[email protected]/tokennexter.go:55 +0x9d
github.com/tekwizely/go-parsing/lexer.(*tokenNexter).Next(0xc0000bc040, 0x0, 0x0, 0xc0000a43d0, 0xc0000aca20)
        /workspace/go/pkg/mod/github.com/tekwizely/go-parsing/[email protected]/tokennexter.go:22 +0x2f
github.com/tekwizely/go-parsing/parser.(*Parser).growPeek(0xc0000d0050, 0x1, 0x0)
        /workspace/go/pkg/mod/github.com/tekwizely/go-parsing/[email protected]/parser.go:199 +0x75
github.com/tekwizely/go-parsing/parser.(*Parser).CanPeek(...)
        /workspace/go/pkg/mod/github.com/tekwizely/go-parsing/[email protected]/parser.go:59
github.com/tekwizely/run/internal/parser.expectCmdScript(0xc0000bc080, 0xc0000d0050, 0xc0000a6540, 0x5, 0x0)
        /workspace/run/internal/parser/parser.go:630 +0x41c
github.com/tekwizely/run/internal/parser.tryMatchCmd(0xc0000bc080, 0xc0000d0050, 0xc0000d6000, 0xc0000c6000)
        /workspace/run/internal/parser/parser.go:238 +0x149
github.com/tekwizely/run/internal/parser.parseMain(0xc0000bc080, 0xc0000d0050, 0x4f3240)
        /workspace/run/internal/parser/parser.go:200 +0xa45
github.com/tekwizely/run/internal/parser.(*parseContext).parse(0xc0000bc080, 0xc0000d0050, 0x4ff720)
        /workspace/run/internal/parser/parser.go:46 +0x7f
github.com/tekwizely/go-parsing/parser.(*astNexter).hasNext(0xc0000bc0a0, 0xc0000d0050)
        /workspace/go/pkg/mod/github.com/tekwizely/go-parsing/[email protected]/astnexter.go:58 +0x97
github.com/tekwizely/go-parsing/parser.(*astNexter).Next(0xc0000bc0a0, 0xc0000bc040, 0xc0000a4370, 0x53c700, 0xc0000bc0a0)
        /workspace/go/pkg/mod/github.com/tekwizely/go-parsing/[email protected]/astnexter.go:30 +0x2f
github.com/tekwizely/run/internal/parser.Parse(0xc0000bc020, 0x1e)
        /workspace/run/internal/parser/parser.go:79 +0x133
main.main()
        /workspace/run/main.go:125 +0x507

If Run had a proper man page, it would feel more like an official *nix tool.

Support something like

run init docker

Which could fetch a Runfile from a repository.

This would allow the community to create high-quality Runfiles for specific use cases.

With the use of run when not in a directory next to the Runfile, it would be nice to have a convenient way to make paths relative to the primary Runfile.

It's possible to do this already, but it's awkward in that it requires shell code, and it needs to be duplicated into each step that requires it, e.g:

deploy-reviewer:
    RUNFILEDIR=$(dirname "${RUNFILE}")
    cd "${RUNFILEDIR}/foo"
    # ... actually do useful things

If, instead, we could do this, IMHO it becomes quite a bit nicer with many different commands:

EXPORT RUNFILEDIR := ${.RUNFILEDIR}

deploy-reviewer:
    cd "${RUNFILEDIR}/foo"
    # ... actually do useful things

On the subject of attributes, with includes being a thing now, it might also be convenient to have attributes for the paths relevant for the imported Runfile, to be able to more easily make reusable things. The included Runfile may be in many directories all over the place, and want to make use of some resources right next to it: the included Runfile is in a better position to be able to define where that is, than the primary Runfile is, in my opinion.

This can be split off to a separate issue, if preferred.

In order to get #55 out the door, I added a very basic level of documentation of Run's attributes.

I think the documentation should be improved, hence this ticket.

Let's firm up attribute documentation !

It would be nice to be able to include other runfiles, including using wildcards with * and **. This way, you can have top-level commands at the root of the project, and it can include commands for modules that are nested deeply. For example:

IMPORT **/Runfile

or:

IMPORT "component1/Runfile"
...
IMPORT "componentN/Runfile"

Hello

I like the Runfile to be called ".runfile"; this is achieved by using RUNFILE env var.

I also like the runfile to be searched up in the path till one is found. This is achieved by using RUNFILE_ROOTS.

Unfortunately it is not possible to docombine both these thingsfeatures.

Could you make RUNFILE_ROOTS search for $RUNFILE if it is supplied, instead of "Runfile" ?

Thank you!

This is what I do but it doesn't work :

# OPTION SCRIPT -s,--script <script> JS codeintegration  
hellonode (node):
  console.log(${SCRIPT})
  eval(${SCRIPT})

I just discovered that Run has had a Nix package for over 2 years:

• NixOS/nixpkgs#77081 (comment)

Let's add this information to the Installation section of the Readme.

I see the note on the README re: the package manager support upcoming.

Is this something you need help with? I've never created a homebrew formula, but since my first thought on reading the HN article was whether I could brew install it, I guess it might be important.

If you're not already working on that in an outstanding branch, it sounds straightforward enough that I could maybe give it a shot and give you a PR for one hosted in your own repo as a tap.

With #55 adding .RUNFILE.DIR to track the primary runfile's parent directory, and .SELF.DIR to track the current (primary or otherwise) runfile's parent directory, it opens the door to possibly adding attributes for other useful directories.

One that comes to mind is the current processes' working directory (PWD / CWD).

NOTE: The CWD is not always the same as the Runfile's parent directory, as there are many ways to invoke a runfile from outside the user's current directory (shebang mode, runfile roots, etc).

Easy to get to already

As with .RUNFILE.DIR, the user's PWD is already easily achievable without an attribute:

Runfile

.DIR := ${PWD}

##
# export MY_DIR := ${.DIR}
#
pwd:
	echo $MY_DIR

Possibly Different Value

As the code would use Go's os.GetWd() method to retrieve the value for the attribute, there's a chance that the attribute value could differ from $PWD

However, Go does check and use $PWD if it appears to be correct.

Why would duplicate tasks be an interesting thing? Well, I'll tell you. I work on many projects at the same time. Each project has both common and specific needs with regard to tasks so I created the CLI utility "justprep" as a pre-processor to add the ability to include common content and add modularization to my taskfiles. When you do that, you then have the ability to include a common Runfile with its tasks into a project specific Runfile which may over-ride one or more of those common tasks.

So by definition, you could warn about duplicates but not fail. That last task defined in the Runfile should be the one that gets executed.

FIrst, the documentation section is called "arguments", but the keyword used is "option", which is a bit inconsistent. Second, it's too much extra work just to enforce required (usually non-boolean) options. It would be nice to have OPTION? as an alternative to OPTION and OPTION! for the required ones. Parsing a default value from the docstring would also make the usa DRY and ease up development.

The .SHELL attribute should be available for read by the Runfile.

Feels like we're not using log.SetPrefix() consistently.

Sometimes we want it to represent the tool (run, +x runfile) and sometimes we want it to represent the Runfile currently being parsed (for file+line+col error reporting)

Let's take a closer look at its usage and make sure its values represent our intentions during the life the command.

One tradeoff I have to do using run is that I can't autoformat scripts, make linters run properly, etc. It would be pretty bad just to put all commands in separate script files and just declare them in Runfile as this way, parameters, and code will be in two different files.

I suggest adding a way to include/import scripts by using wildcards or directories. Ideally, the folder hierarchy could be used to prefix the commands, i.e., scripts in the terraform folder will automatically prefix with contained commands with terraform-, so ./terraform/lock.sh would be terraform-lock. The manifest in comments could be searched for after the optional shebang.

This way, not having a grammar for Runfile files won't be such a big deal as the vast majority of the code will be simply shell scripts.

Currently, the parser treats a single title line doc as a simple string and does not try to parse it as a dynamic string:

Runfile

## Doc line in ${.RUNFILE}
line:
	echo hello

##
# Doc Block in ${.RUNFILE}
block:
	echo hello

testing

$ run list

Commands:
  list       (builtin) List available commands
  help       (builtin) Show help for a command
  version    (builtin) Show run version
  line       Doc line in ${.RUNFILE}
  block      Doc Block in /tmp/Runfile

expected

...
  line       Doc line in /tmp/Runfile
...

Hi!

I want to run react-native run-ios
And sometimes react-native run-ios --device or react-native run-ios --device MyPhone

So I defined this Runfile:

ios:
  react-native run-ios ${1} ${2}

But then I get an error when I type: run ios --device
Error: flag provided but not defined: -device

A workaround might be to define possible arguments as commandline options in run. But that seems a little cumbersome.
Or I could define a different command for each possible parameter (and combination). Also not ideal I think.

Related question: is ${*} also supported, to pass all arguments as one?