xonsh / slug

A snail without its shell

xonsh

xonsh is a Python-powered shell. Full-featured and cross-platform. The language is a superset of Python 3.6+ with additional shell primitives. Xonsh word was made from conch (🐚, @) and indicates belonging to the command shells world.

Xonsh is the Shell	Xonsh is Python
cd $HOME id $(whoami) cat /etc/passwd | grep root > ~/root.txt $PROMPT = '@ '	2 + 2 var = "hello".upper() import json; json.loads('{"a":1}') [i for i in range(0,10)]
Xonsh is the Shell in Python	Xonsh is Python in the Shell
len($(curl -L https://xon.sh)) $PATH.append('/tmp') p'/etc/passwd'.read_text().find('root') id = $(@json docker ps --format json)['ID']	name = 'foo' + 'bar'.upper() echo @(name) > /tmp/@(name) ls @(input('file: ')) aliases['e'] = 'echo @(2+2)' aliases['a'] = lambda args: print(args)

If you like xonsh, ⭐ the repo, write a tweet and stay tuned by watching releases.

First steps

Install xonsh from pip:

python -m pip install 'xonsh[full]'

And visit https://xon.sh for more information:

• Installation - using packages, docker or AppImage.
• Tutorial - step by step introduction in xonsh.

Extensions

Xonsh has an extension/plugin system. We call these additions xontribs.

• Xontribs on Github
• Awesome xontribs
• Core xontribs
• Create a xontrib step by step from template

Projects that use xonsh or compatible

• conda and mamba: Modern package managers.
• Starship: Cross-shell prompt.
• zoxide: A smarter cd command.
• gitsome: Supercharged Git/shell autocompleter with GitHub integration.
• xxh: Using xonsh wherever you go through the SSH.
• any-nix-shell: xonsh support for the nix run and nix-shell environments of the Nix package manager.
• lix: A modern, delicious implementation of the Nix package manager.
• x-cmd: x-cmd is a vast and interesting collection of tools guided by the Unix philosophy.
• rever: Cross-platform software release tool.
• Regro autotick bot: Regro Conda-Forge autoticker.

Jupyter-based interactive notebooks via xontrib-jupyter:

• Jupyter and JupyterLab: Interactive notebook platform.
• euporie: Terminal based interactive computing environment.
• Jupytext: Clear and meaningful diffs when doing Jupyter notebooks version control.

The xonsh shell community

The xonsh shell is developed by a community of volunteers. There are a few ways to help out:

• Solve a popular issue or high priority issue or a good first issue. You can start with the Developer guide.
• Take an idea and create a new xontrib.
• Become xonsh core by deep diving into xonsh and improve the threading and subprocess logic.
• Become a sponsor to xonsh.
• Write a tweet, post or an article to spread the good word about xonsh in the world.
• Give a star to xonsh repository and to xontribs you like.

We welcome new contributors!

Credits

• Thanks to Zulip for supporting the xonsh community!

slug's Issues

Windows Support

Implement Windows support.

ProcessGroup.join() won't block on descendant processes

ProcessGroup.join() ignores descendant processes (not direct children).

• Base: Not sure if this is possible?
• POSIX: The wait*() family of functions support PGIDs
• Win32: I think WaitForSingleObject() will cover this? Language implies this only happens when job exceeds resource limits, not when it exits naturally.

Terminal Duplication

Keeping a PTY in sync with the terminal we're attached to.

https://github.com/asciinema/asciinema/blob/master/asciinema/pty_recorder.py

Test on Cygwin

Expand CI configurations to also test on cygwin.

POSIX Support

Implement against by-the-spec POSIX.

Process Return Code on premature death?

When a process aborts (ie, ends without setting a numeric return code), what should Process.return_code return?

Negative number matching the signal number?

String of the signal name?

Special object?

context issue

This is the code in current readme:

with ProcessGroup() as pg:
  pipe = mkpipe()
  spam = pg.process(['spam'], stdin=StdIn(), stdout=pipe.side_in, stderr=StdErr(), environ=...)
  eggs = pg.process(['eggs'], stdin=pipe.side_out, stdout=StdOut(), stderr=StdErr(), environ=...)
pg.start()
pg.join()

The last two line looks a bit weird. I mean it's valid but I'd like keep the pg only used inside the with block - make pg.start(), pg.join() implemented inside the context manager itself?

Implement Virtual Processes

Implement support for "processes" that are python classes.

State Change Notifications

Support some kind of callback system when process and process group states chage.

Cache pgid

Process (POSIX version) should somehow cache pgid so it's still available after the process exits.

PG doesn't initialize with staggered start

If you start the individual processes in a process group, the process group is not initialized correctly, causing the processes to not properly join the group nor the group created.

with ProcessGroup() as pg:
  pipe = Pipe()
  spam = Process(['spam'], stdout=pipe.side_in)
  pg.add(spam)
  eggs = Process(['eggs'], stdin=pipe.side_out)
  pg.add(spam)
spam.start()
eggs.start()
eggs.join()
pg.kill()
pg.join()

pipeline not work

Content of file test_slug.py:

from slug import Process, ProcessGroup, Pipe

with ProcessGroup() as pg:
  pipe = Pipe()
  spam = pg.add(Process(['ls'], stdout=pipe.side_in))
  eggs = pg.add(Process(['wc'], stdin=pipe.side_out))
pg.start()
pg.join()

Run py test_slug.py, on Linux, it will stuck. On Mac this issue shows:

$ py test_slug.py
Traceback (most recent call last):
  File "test_slug.py", line 8, in <module>
    pg.start()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/xonsh_slug-0.0.1-py3.6.egg/slug/posix.py", line 91, in start
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/xonsh_slug-0.0.1-py3.6.egg/slug/posix.py", line 63, in pgid
ProcessLookupError: [Errno 3] No such process
      71      72     750

Joining descendants of virtual processes to process groups

Currently, it's left up entirely to the calling application to track a VirtualProcess's current group and making sure any executions it makes get joined to the group.

Implementing this will also likely run afoul of #23.

Text-mode Pipes

Add Pipes that go between text and binary, or are just text on both sides.

[RFC] Dependency Injection?

We have a lot of singletons. Most of them get lumped on to builtins, which is ugly and hackish and they have to all be initialized carefully and at once.

Dependency injection would allow us to delay some initialization, clean up namespaces, and make sure that concerns are where they belong.

Test on WSL

Expand CI to cover Windows Subsystem for Linux (aka Bash on Windows, Ubuntu on Windows).

PyPy support

Now that PyPy3.5 is released, PyPy support should be added.

Note that I think there's a few places where we rely on specific CPython behaviors, so some work might be needed.

Add stop method to plumbing

Currently, Tee, Valve, and QuickConnect clean up when their source pipe hits EOF. Add a method to manually clean up.

Interruptible Reads on Windows

Both Valve and QuickConnect need extremely special windows implementations.

• Python's .read() will only break on an actual triggered interrupt, not just a faked exception through PyThreadState_SetAsyncExc().
• Window's select only works on sockets, not pipes or other IO types.

So I think we need to use win32's ReadFile() manually in conjunction with CancelSynchronousIo() in order to not read an extra block when switching.

Add topics to the repo

hi @AstraLuma ! Please add xonsh-dev topic to this repo to make it more visible in the list https://github.com/topics/xonsh-dev

Virtual file connectors

Add adaptors to go between purely virtual file-like objects and concrete types with file descriptors.