wryun / es-shell Goto Github PK

This seems to be all GNU's fault. They decided to be "helpful" by making GNU getopt parse ALL options in argv, rather than stopping at the first non-option arg as the POSIX spec states. Several libcs have imitated GNU in doing this. Several others follow POSIX.

This is 1. inconsistent with every other shell, 2. inconsistent with es' own man page, and 3. badly surprising in certain cases. See this example:

; ./coolscript foo bar
My cool script! argv: foo bar
; ./coolscript -x
{echo My cool script^! argv: $*}
My cool script! argv:
; cat coolscript 
#!/usr/local/bin/es
echo My cool script! argv: $*
;

This can be worked around by adding "+" to the front of the optstring arg to getopt(), but that's an imperfect solution because some implementations will barf if the "+" is present. (You can also set POSIXLY_CORRECT=1 in the env when invoking es, but ... ew.)

I see two good options here: 1. roll es' own opt-getting logic (it's not terribly hard to do); or 2. set up some autoconf magic to detect whether the local getopt understands "+".

Quoting an email:

There was a shift to a different editline library than that of Salz and Turner in Rakitzis's more recent rc. Looking at es-0.9 might you add to the documentation which editline is being referred to as an option to GNU readline.

Factually, the README does not contain information on obtaining editline, but does include an ftp link (ftp://ftp.sys.utoronto.ca/pub/es/) to where the editline.shar file may be downloaded.

error:

closure.c:72: assertion failed (k == nWord || k == nQword || k == nPrim)

Produced by my somewhat psychotic startup file after two 'es -l':

let (ls = `{which ls}) fn ls {`{which ls} --color\=auto $*}

fn pwd {
    if {~ $#cwd 0} {
        noexport = $noexport cwd
        cwd = `` \n /bin/pwd
    }
    echo $cwd
}

# symlink cd ..
let (cd = $fn-cd) fn cd dir {
    if {~ $#cwd 0} {
        noexport = $noexport cwd
    }
    if {~ $#dir 0} {
        $cd
        cwd = ~
    } {
        let (current = <={
            if {~ $dir /*} {
                result
            } {
                if {~ $#cwd 0} {
                    cwd = `` \n /bin/pwd
                }
                %split / $cwd
            }
        }) {
            for (name = <={%split / $dir}) {
                if {~ $name ..} {
                    if {!~ $#current 0} {
                        let (x = 1 $current) current = $x(2 ... $#current)
                    }
                } {!~ $name . ''} {
                    current = $current $name
                }
            }
            let (path = / ^ <={ %flatten / $current }) {
                $cd $path
                cwd = $path
            }
        }
    }
}

# go back -N directories in cd (cd -1 prints stack, cd - goes to previous)
#
let (cd = $fn-cd; cd-stack = (. . . . . . . . . .)) fn cd dir { 
    if {~ $dir -*} {
        let (index = <={%split - $dir}) {
            if {~ $#index 0} {
                index = 2
            }
            if {~ $index [2-9]} {
                dir = $cd-stack($index)
            } {~ $index 1} {
                echo $cd-stack >[1=2]
                return 0
            } {
                throw error cd 'cd: invalid argument'
            }
        }
    }
    $cd $dir
    cd-stack = (`pwd $cd-stack(1 ... 9))
}

# colourful prompt
let (cd = $fn-cd; c = \1\033; z=\2) fn cd {
    $cd $*;
    let (w = `pwd) {
        if {~ $^w $home^*} {
            w = '~'^<={~~ $^w $home^*}
        }
        prompt = $c[4\;35m$z`{hostname}^$c[0m$z:$c[1\;34m$z$^w$c[0m$z^'; '
    }
}

# when we start, we should 'cd .' to set the colourful prompt
fn %prompt {
    cd .
    fn %prompt # now lose the prompt function
}

CC=colorgcc

Is it possible to treat code blocks as structures and manipulate them as such? I'm poking at implementing something like the multipipe blocks from dagsh and figured a basic version could be doable with a function that takes a block and just rewires the pipes.

I tried to compile es on cygwin (64bit) by using the tarball for the 0.9 release.
I configured the source (from an other build directory) and ran make:

[...]
gcc  -I. -I../es-0.9 -g -O2   -c -o prim-sys.o ../es-0.9/prim-sys.c
gcc  -I. -I../es-0.9 -g -O2   -c -o prim.o ../es-0.9/prim.c
gcc  -I. -I../es-0.9 -g -O2   -c -o print.o ../es-0.9/print.c
../es-0.9/print.c:66:14: error: conflicting types for ‘utoa’
 static char *utoa(unsigned long u, char *t, unsigned int radix, char *digit) {
              ^
In file included from /usr/include/sys/unistd.h:8:0,
                 from /usr/include/unistd.h:4,
                 from ../es-0.9/stdenv.h:24,
                 from ../es-0.9/es.h:4,
                 from ../es-0.9/print.c:3:
/usr/include/stdlib.h:184:8: note: previous declaration of ‘utoa’ was here
 char * _EXFUN(utoa,(unsigned, char *, int));
        ^
<builtin>: recipe for target 'print.o' failed
make: *** [print.o] Error 1
$ 

It seems that the function utoa defined in print.c clashes with a function of the same name defined in stdlib.h.

$ grep -r 'utoa' .                                                  /opt/es-0.9
./print.c:static char *utoa(unsigned long u, char *t, unsigned int radix, char *digit) {
./print.c:              t = utoa(u / radix, t, radix, digit);
./print.c:      len = utoa(u, number, radix, table[upper]) - number;

This function is only used two times. Let's just rename it?

$ sed -ri 's/utoa/another_utoa/g' **/**.c                           /opt/es-0.9
$ grep -r 'utoa' .                                                  /opt/es-0.9
./print.c:static char *another_utoa(unsigned long u, char *t, unsigned int radix, char *digit) {
./print.c:              t = another_utoa(u / radix, t, radix, digit);
./print.c:      len = another_utoa(u, number, radix, table[upper]) - number;

I could now compile the es from the source directory by using ./configure && make. (I needed to install the bison package)

When exception is thrown from inside of any command with redirected I/O ($&openfile), the opened file is left open with no possibility to close it.

ls -l /dev/fd/
catch @ {} {
  let (fd = <=%newfd)
  %open $fd /dev/zero { $&throw anything }
}
ls -l /dev/fd/

As a simple workaround, one could catch all the exceptions and manually pass them ‘outside’ the $&openfile for example this way:

let (openfile_real = $fn-%openfile)
fn-%openfile = @ mode fd file code {
  let (exception = ) {
    let (
      result = <={
        $openfile_real $mode $fd $file {
          catch @ { exception = $* } { $code }
        }
      }
    ) {
      ~ $exception || throw $exception
      result $result
    }
  }
}

http://paganbooks.eu/software/article/rc

Enabling the GCDEBUG flag (equivalent to both GCALWAYS and GCPROTECT) causes two segfaults to occur, one in $&fsplit, and one in exception-handling code.

The $&fsplit segfault happens at split.c:99, because with GCALWAYS, the mkstr and mklist calls start a GC, which makes s point to invalid memory, which causes a segfault when GCPROTECT is enabled. I have a fix implemented for this (essentially, a re-entrant implementation of splitstring) which I'll create a PR with later.

The exception-handling segfault happens at prim-etc.c:254 when GCALWAYS is enabled (this case does not require GCPROTECT to be enabled to happen, though it's a fatal failure either way). I suspect one of the GCs that get triggered during the chained mklist and mkstrs around prim-ctl.c:70 are to blame, but I'm not confident enough with the Refs to know quite how to fix it.

For posterity.

Hello,

I tried to build es-shell, following the instructions in INSTALL and found out that configure script is not available. Would be great to try out this shell.

Kind regards,

reirob

The current behavior of $&if does not catch exceptions in its condition/test parameters:

# throws error $&access 'missing-cmd: No such file or directory'
if missing-cmd {
    echo command found
} {
    echo command not found
}

The description of the -e flag in the man page seems to suggest that this is actually an oversight/bug since the exception will cause the shell to exit in tests of conditional statements.
Wrapping every single condition in a catcher works around this issue, but nobody wants to do that, right?

Many missing macros, etc.

I am used to CTRL-A in a shell moving to the beginning of the line. In es it doesn't; Indeed I'm not sure what it does.

1. What is it doing?
2. How do I move to the beginning of the line?
3. Is there documentation of all keybindings in es?

One of the co-authors of es wrote me historically he had a difficult time locating maintainers for the shell.

xs appears to be abandonware

@wryun

I'd enquire if you might consider suggesting to the xs author that the repo could be moved to wyrun/XS?

running ./buildscan.sh outputs warnings

the autotools build was converted to a meson build

not all meson versions work for the build (a python3 virtual environment with meson-0.54.x or 0.56.x works)

$ python3 -m venv xs_env
$ ~/xs_env/bin/python3 -m pip install --upgrade pip
Requirement already satisfied: pip in ./xs_env/lib/python3.9/site-packages (22.0.4)
$ ~/xs_env/bin/python3 -m pip install meson==0.56.2
Collecting meson==0.56.2
  Downloading meson-0.56.2.tar.gz (1.8 MB)
    ---------------------------------------- 1.8/1.8 MB 358.5 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: meson
  Building wheel for meson (pyproject.toml) ... done
  Created wheel for meson: filename=meson-0.56.2-py3-none-any.whl size=698109 sha256=3262eb59133294b2cc2af3da245c12ed05ce1a8cb1130f7269ad2ba22c83f845
  Stored in directory: /home/eric/.cache/pip/wheels/b8/1a/46/6990fd0de22ecf99596a8ee2d65e0c2e8bc3403872b57f33a5
Successfully built meson
Installing collected packages: meson
Successfully installed meson-0.56.2
$ source ~/xs_env/bin/activate
(xs_env) $ which meson
/home/eric/xs_env/bin/meson
(xs_env) $ 

A FSF associate wrote me they looked at math.xs, describing it as ugly.

In addition, it seems to mean using functional programming to
decompose a list. That is a pain in the neck.

I don't know whether it is more bad or less bad than programming in
shell syntax. But at least there' an excuse in that case: programming
is built around a command line syntax. I see no excuse for making the
math.xs syntax so inconvenient. If they're designing a new language
from zero, why take such a big step backard?

http://wryun.github.com/es-shell/mail-archive/msg01026.html

easy way to reproduce (~25%?) is to create a self-referential file:

#!/usr/bin/env es
./a

related (?): seems to miss a Ctrl-C occasionally as well.

rc changelog suggests they rewrote signal handling code. Worth a look.

My specific encounter was implementing a ghetto switch-case #31 like so:

let( match = something
     cases = some* thing onething $match #last one is 'default'
     bodies = ( {echo this is something}
                {echo this is thing}
                {echo this is onething}
                {echo reached default} ))
for (case = $cases; body = $bodies) if {~ $match $case} $body

but this will echo reached default because 'some*' won't be expanded inside the tilde-match statement.
this is good behaviour of course, but my question is do we have an idiomatic way to override it?

AFAIK there's no expand primitive, and eval would try to run the pattern as a command.
Perhaps the only answer is eval '{~ $match '$case'}', but I wonder if there's something more general to expand quoted patterns without necessarily running a command on them under eval..

http://wryun.github.com/es-shell/mail-archive/msg00987.html

This dumps core:

; echo <={@ x{x=@{$*}; $&result $x}}

why? check this one first:

; echo <={@ x{$&result @{$}} X}
%closure(x=X)@ * {$}

Now, imagine replacing X by the entire closure, and you have a closure
containing a reference to itself. That can never be printed, of
course.

Just because we can.

Hello, thanks for maintaining the es shell.

I recently packaged es for KISS Linux and I am wondering if it is reasonable enough to release the next version soon. A handful of sweet commits have been made since the release of 0.9.1 and I personally have used the master version for a few months without any major problems.

Have a good day.

With readline enabled, es is appending to my ~/.history file but doesn't load it when it starts, so the only history I can search is the one since the session start. Is there a way of making it load the rest of the history?

This is what I get when trying to make Es on Adélie Linux, which uses musl libc:

gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o closure.o closure.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o conv.o conv.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o dict.o dict.c
dict.c: In function ‘put’:
dict.c:126:19: warning: cast between incompatible function types from ‘Dict * (*)(Dict *, char *, void *)’ {aka ‘struct Dict * (*)(struct Dict *, char *, void *)’} to ‘void (*)(void *, char *, void *)’ [-Wcast-function-type]
   dictforall(old, (void (*)(void *, char *, void *)) put, new);
                   ^
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o eval.o eval.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o except.o except.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o fd.o fd.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o gc.o gc.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o glob.o glob.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o glom.o glom.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o input.o input.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o heredoc.o heredoc.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o list.o list.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o main.o main.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o match.o match.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o open.o open.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o opt.o opt.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o prim-ctl.o prim-ctl.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o prim-etc.o prim-etc.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o prim-io.o prim-io.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o prim-sys.o prim-sys.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o prim.o prim.c
gcc  -I. -I. -W -Wall -Wno-missing-field-initializers -Wno-unused-parameter -Wno-clobbered -g -O2   -c -o print.o print.c
print.c: In function ‘fmtprint’:
print.c:281:15: error: assignment to expression with array type
  format->args = saveargs;
               ^
make: *** [<builtin>: print.o] Error 1

Adélie developers told me that the error is indeed related to musl vs. glibc. The problem is with __va_copy, which presumably is an internal glibc macro. Replacing it with the standard va_copy fixes the issue. (Thanks to aranea.)

After fixing that, make still fails, but for other reasons.

It may be useful in many applications where you want to find a command's return status after the fact, not while executing it.
usually to get a command's status we wrap it in <={ ... }, but currently I know of no way to get a command's return status after it had executed. exposing $result by not making its scope local to %interactive-loop (and maybe renaming it to $status so that there's less confusion with the result keyword) would allow for that.

I get the following (or a related) error when I attempt a make install to a directory that does not exist:

$ make install
/usr/bin/install -c -s ./es /tmp/es/bin
/usr/bin/install: cannot create regular file ‘/tmp/es/bin’: No such file or directory
Makefile:85: recipe for target 'install' failed
make: *** [install] Error 1

Weirdly, the first time it failed when installing the man page. Possibly /tmp/es existed but /tmp/es/man/man1 did not.

See the following command:

; let (n = '%closure (x = y) echo $x') {$n}
y y

The correct behavior (and the behavior that comes from variations on the command, like putting {} around the echo $x, is to only print one y.

Digging around with gdb I see the bogus extra y gets added in eval.c:418 (list = append(list, list->next);). Commenting out that line fixes the issue, and strangely seems to be a no-op in most other cases -- trip.es passes, and I can do at least some basic navigating in an interactive shell. It looks like list->next at this point in the code is typically null.

Is it reasonable to delete the line? Can anyone find something that would break?

Once I figured out that the OpenBSD version of es is the original one, I attempted to build your version instead.
The problem is, after executing

libtoolize -qi
autoreconf

the configure script does not magically appear, as promised. Only configure.ac is there from the start.
What should I do now?

Currently, es has been "tested" by running trip.es, but this is very insufficient. Contributors would be averse to making any serious feature-additions unless there was a good test suite to verify they didn't break anything important. I propose simply initializing such a test suite, written in es, that does something simple such as run one of the examples and assert on the standard-out and return code of the result (we can always add to it later once the structure is in place!)

In order to accomplish that, we also (ideally) need some sort of assert library written in es which captures those things from running any es command and allows us to do things like assert_equal, assert_not_equal, assert_success, assert_failure, assert_match and assert_no_match on any of stdout, stderr or return codes. Alternately, we could use an assert/test library written in another scripting language such as bash and run es code and assert on it from there. (I looked at a few- BATS is basically dead, and ShellSpec is full-featured but uses cucumber-esque syntax and since cucumber is gross (is it just me, or are API's which attempt to look like English, but are not English, basically the "uncanny valley" of API's?), I am EXTREMELY averse to using it). But it might make sense to bootstrap a suite with one of those (or another) with an eye towards replacing it with pure es in the future. Thoughts?

My main frustration with using es as an interactive shell is having to handle GNU style long arguments, as '=' is part of es syntax wherever it occurs in a line. i.e.

ls --color=always

will generate a syntax error. You can get around this with quotes or escapes:

ls --color\=always
ls --color'='always

But it's annoying, and means you can't copy/paste examples designed for other shells.

xs addressed this by replacing '=' with ':=', which is a good solution but drops backwards compatibilty and similarity with other shells.

I'm wondering if there's an easy way to make both uses of '=' work, since a normal usage of a long argument reliably results in a syntax error (as the LHS isn't valid).

This seems like syntax that should be valid, right? Minimal (useless) example:

; cat < <{echo *}
syntax error

Found while thinking about alternatives to #55.

Or is it just waiting to be implemented? I'm thinking about giving that a try honestly.

Fascinated by this shell (why has this project not gotten more attention??), but afraid to mess with anything due to the lack of a real test suite ("make trip" doesn't, or shouldn't, count!)

I actually forked es-shell, fixed compiler bugs on the current gnu toolchain and merged in the job control branch (and added some homebrew-friendly config stuff for macOS devs)... but I have no idea if I broke anything other than just running the examples and seeing a success. (Which is how I found this problem. I then came back here, built it without any of my changes, and adventure.es was still broken in an endless loop... unless my tooling somehow caused a bug!)

For starters, just a way to assert on stdout, stderr (and any other fd's, I suppose) and return code, given a statement or block or what have you, would be an outstanding step towards a fundamental test suite function (I found a very hacky way to do this in bash so I could test my bash scripts, an even hackier way to do this in zsh, but I'd have no idea how to build such an assert in es!)

By day I'm an Elixir/BEAM coder (hence the "functional shell" interest) and my level of C/C++ ability is basically "given an unlimited amount of time and all the googles, can probably build anything" 😅

Reproduce: $&. (include the full-stop)

Either es works, quits, or produces the message:
es panic: eval: bad closure node kind 3

Does es have an equivalent to Bash/POSIX's Ctrl-Z command to background a currently-running process?