adam-mcdaniel / dune

A shell🐚 by the beach🏖️!

dune

A shell by the beach!

NOTE: Click the image above for a video demonstration.

About the Author

I'm a bored sophomore in college working on projects to fill the time. If you enjoy my work, consider supporting me by buying me a coffee!

Why write another shell?

I feel that bash is great in a lot of ways, but it doesn't exactly feel cozy: it's lacking a sort of personal touch, and it's also missing quick and easy customizability. With my last shell, Atom, I had accomplished some of the coziness that bash was missing, but I also introduced a lot of really fatal flaws in the syntax and the type system.

Dune, however, is designed completely differently from Atom (although you might notice the similarities in their widget systems). The interpreter itself is standalone, and it holds almost none of the functionality you see in the default distribution of Dune. If you wanted to, you could write a custom frontend and make a unique Dune based shell of your own!

This frontend implementation turns the coziness dial to 11. Just check out the shell's default startup script!

I put a lot of work into making Dune just fun to use. It's like a neat little operating system itself!

Dune also attempts to be a usable scripting language, and even offers a few niche metaprogramming features such as quoting (borrowed from Lisp), operator overloading, and macros!

Overall, I wrote Dune to have a complete shell of my own: one that's fast, useful, and pretty.

(Also, writing a shell is just kinda really fun)

Usage

Dune has a bunch of customizable components. Here's how you can change them and make your shell your own!

The Prelude

Before entering interactive mode, Dune executes the prelude. The prelude is just the startup file .dune-prelude stored in the home directory for your user. If you don't provide your own prelude file, Dune will execute its own default prelude with an introduction to the shell.

You can see my example personal prelude here.

The REPL

Dune's REPL is entirely customizable by overloading the following functions:

Name	Purpose	Default Implementation
prompt	This function is called to generate the text which prompts the user for input. It takes the current working directory, and returns a string.	let prompt = cwd -> fmt@bold ((fmt@dark@blue "(dune) ") + (fmt@bold (fmt@dark@green cwd)) + (fmt@bold (fmt@dark@blue "$ ")))
incomplete_prompt	This function is called to generate the text which prompts the user for input when they have entered an incomplete expression. It takes the current working directory, and returns a string.	let incomplete_prompt = cwd -> ((len cwd) + (len "(dune) ")) * " " + (fmt@bold (fmt@dark@yellow "> "));
report	This function is called to print a value to the console after evaluation.	The default implementation is a builtin function (implemented in Rust), but you can overload it with any callable value nonetheless.

I highly recommend using the fmt module when implementing your own customizations for your prompt!

Aliases

This distribution of Dune uses the Symbol type (the type of variable names and paths) to implement calling programs. Whenever an expression of type Symbol is evaluated as a command in interactive mode, it is invoked as a program.

Because of this, you can define aliases by assigning a variable to a program's name like so!

If you have defined a variable that overshadows your program's name (such as an alias), you can quote the program name to run it.

Macros

To write functions that modify your shell's environment and act like commands or programs themselves, use a macro!

Macros, when called with zero arguments, are passed the current working directory. When invoked, they assume the environment of the callee: if you execute a macro, it will execute as if you executed the contents of the macro itself with the parameter defined as the argument passed.

Piping and Redirection

Piping and redirection are done with the | and >> operators. Here's some example uses!

If a value is piped into a callable object, like a function or macro, it is performed as an application; otherwise, the expression is treated like a regular call to a program.

Standard Library

Dune offers an extensive standard library, and also provides a pretty interface to see all the functions available in each module!

Dune offers the following builtin libraries:

Name	Description
rand	A library for randomness
time	A library with date and time functions
math	A module for math and trig functionality
fs	A module for interacting with the file system
fn	A functional programming library
fmt	A library for text formatting on the console (color, styling, hyperlinks, text wrapping, etc.)
os	A small module with the host's OS info
widget	A module for creating text widgets
shell	A small module for information about the Dune shell
console	A library for manipulating the console

For more information about each, just run echo library-name.

Installation

To install, you must download Rust from here. If you already have Rust installed you will probably need to update. Dune uses a lot of recently stabilized features.

Development Build

# Install directly from git with cargo
cargo install --git https://github.com/adam-mcdaniel/dune

# Or, alternatively, the repo and install from source
git clone https://github.com/adam-mcdaniel/dune
cd dune
cargo install -f --path .

Releases

To get the current release build, install from crates.io.

# Also works for updating dune
cargo install -f dune

Currently, since Dune is in its early stages of development, I would recommend against using releases at the moment. There are a lot of bug fixes and new features added inbetween releases.

After Install

# Just run the dune executable!
dunesh

dune's Issues

builtin@exec seems to clash with exec subcommand

When using docker compose, every time I use the exec command e.g.

docker compose exec container_name

I get the following error:

unknown docker command: "compose builtin@exec"

It seems to me like the exec subcommand clashes with the builtin@exec 🤔

History file location

The history file is created as ~/history.txt by default. It would be better, and more conventional, if it was stored as ie. ~/.dunesh_history, ~/.cache/dune/history.txt or $XDG_CACHE_DIR/dune/history.txt.

Implement piping with builtin function

Because of the way builtin functions are implemented, it should be simple to implement piping with them.

Builtin functions act sort of like Lisp special forms (or macros): their arguments aren't evaluated by the interpreter before they're passed. Because of this, it's entirely possible to add some syntax like so:

"Hello world!" | cat

Such that it is expanded to:

pipe "Hello world!" cat

Similar to how Dune already implements addition.
In the pipe builtin, it would simply check the structure of the expressions it's called with using a match, and then link them together the appropriate piped stdin/stdouts.

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value

When starting Dune for the first time and typing ls, it works fine. However, after cd / I get this output when running ls:

(dune) /$ ls
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Io(Os { code: 13, kind: PermissionDenied, message: "Permission denied" })', src/bin.rs:440:48
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

There are no permission issues when running this under zsh, it calls ls and lists the files normally.

Proper parsing/syntax highlighting

Currently syntax highlighting is pretty rudimentary: It searches for tokens without context, so strings/comments can't get highlighted properly, and tokens are erroneously highlighted within strings and comments.

While investigating, I also noticed that the parser sometimes parses keywords within identifiers:

(dune) /home/ludwig$ let TrueStory = "The following is parsed as two tokens!"
(dune) /home/ludwig$ TrueStory
cannot apply `True` to the arguments [Story]

This shouldn't be too difficult to fix.

I'm not sure how to best implement syntax highlighting. I think the best solution is to use a tokenizer, i.e. a function that splits the input into individual tokens. This could also be used by the parser if desired.

"could not compile `dune`" on cargo install

Attempting to run $ cargo install -f dune (as per README instructions) fails with the following output:

   ...
   Compiling prettytable-rs v0.8.0
   Compiling dune v0.1.4
error[E0658]: binding by-move and by-ref in the same pattern is unstable
   --> /Users/djjonno/.cargo/registry/src/github.com-1ecc6299db9ec823/dune-0.1.4/src/expr.rs:670:41
    |
670 |             (Self::Map(m), Self::Symbol(name)) | (Self::Map(m), Self::String(name)) => {
    |                        -                ^^^^                -                ^^^^ by-move pattern here
    |                        |                |                   |
    |                        |                |                   by-ref pattern here
    |                        |                by-move pattern here
    |                        by-ref pattern here
    |
    = note: see issue #68354 <https://github.com/rust-lang/rust/issues/68354> for more information

error: aborting due to previous error

For more information about this error, try `rustc --explain E0658`.
error: failed to compile `dune v0.1.4`, intermediate artifacts can be found at `/var/folders/fj/4fhc5dsd2k328wyq912fp4t9t4jt1h/T/cargo-installLtfTmd`

Caused by:
  could not compile `dune`

To learn more, run the command again with --verbose.

System:

• os: macOS Catalina
• rustc: rustc 1.48.0 (7eac88abb 2020-11-16)
• cargo: cargo 1.48.0 (65cbdd2dc 2020-10-14)

Should be able to disable dune shrew on startup

He's really cute and all, but i don't want him getting in the way...

Why is `//` used for division?

There doesn't seem to be a compelling reason not to use /.

Builtin JSON and TOML parsers

I would really like to have builtin JSON and TOML parsers (into Dune Expressions) so that it's possible to implement something like a weather dashboard using curl, or an automatic Dune updater in my prelude. This would be really simple to add, since Dune's Expression type is basically a more expansive JSON object itself.

Package name

Hello,

Thanks for creating dune, I just tried it, and it's pretty fun.

If this package hypothetically were to be packaged for a Linux distro, what should the package name be?

dune is already taken by the build system for OCaml.

Would you prefer adam-mcdaniel-dune or something like dune-shell? Have you considered renaming the project to avoid name collisions?

Best regards,
Alexander F. Rødseth

Shell doesn't terminate on Ctrl-D or EOF

I tried exiting the shell after testing it using Ctrl-D and this didn't work. When I do something like echo "1 + 1" | dune I would expect to see "2" but I see the welcome screen of dune and it just hangs even without a prompt. Ctrl-C also doesnt work as expected then

README link for sample prelude broken

You can see my example personal prelude here.

"here" links to https://github.com/adam-mcdaniel/dune/blob/main/.dune-prelude , which 404s for me. (private repo?)

Error handling

When I'm not connected to the internet, curl fails to get weather data which causes parse@json to fail, and also terminates the execution of my prelude early. I think that we should try to make a convenient way to handle errors (although the parsing functions should be implemented such that they return None instead of throwing errors).

I'm thinking of implementing a try builtin function that might behave like the following:

try 2 // 0 { echo "divide by zero"; 0 }

try would attempt to evaluate the first argument If it does not throw an error, then try will return that result. If the first argument fails to evaluate, it would evaluate the second argument, and return its result.

Tests

There should be at least the following kinds of tests:

• Tokenization tests: Check if inputs are correctly tokenized
• Parser tests: Check if valid inputs are correctly parsed and invalid inputs show a usable error message
• UI tests: Verify that evaluating an expression does what we expect

Matching brackets break syntax highlighting

The MatchingBracketHighlighter inserts ANSI escape codes into the string which is then syntax highlighted, which breaks syntax highlighting.

For example, when typing let a = [foo, bar] and placing the caret at the right bracket:

Another issue is that the MatchingBracketHighlighter isn't aware of strings or comments:

The best solution would be to remove the MatchingBracketHighlighter. The same functionality could be implemented in the syntax_highlight function directly.

While the first problem could also be fixed by stripping the ANSI codes again in that function (and the second issue isn't as common), it's not an elegant solution and potentially error-prone. Also note that not all ANSI codes end with an m (although all ANSI codes currently used by the syntax highlighter do).

Config file

What a config file could contain:

• Color theme
• Keyboard shortcuts
• Paths to dune preludes

The config file should be read from ~/.config/dune/ or XDG_CONFIG_DIR or whatever the platform default is. This can be done with the dirs-next crate.

`fn@filter` doesn't work

this looks like a very cool project! I tried out some stuff and found that:

 [1, 2, 3, 4, 5] | fn@filter ( x -> x == 1 )

prints: cannot add [] and item

while:

fn@filter ( x -> x == 1 ) [1, 2, 3, 4, 5] 

works as expected and returns [1]

not sure whats the cause of the difference but I managed to fix it by changing fn@filters implementation

to:

f -> list -> { let result = [];
  for item in list { if (f item) { let result = (+ result [item]) } else (None) }; result
}

I've been wanting a more functional shell for a long time and I hope dunesh could turn into this for me, so I'd love to see this fixed, let me know if you need any help 😄.

Styled prompts on Windows behave incorrectly

Windows currently displays styled prompts correctly but places the cursor much too far to the right. This is because rustyline::Editor::readline does not expected an ANSI escaped string. ANSI escapes need to be added in the Highlighter::highlight_prompt implementation.

Stuck at piping with pv

I've installed pv to check the speed while pipe'ing, but it doesn't work.

$ seq 1 1000000 | pv -l | wc -l
12.8k 0:00:33 [0.00 /s] [ <=>                                     

and it's stuck.

This is Bash:

$ seq 1 10000000 | pv -l | wc -l
10.0M 0:00:00 [69.1M/s] [ <=>                                                                           ]
10000000

Change names of builtin operator functions

I don't like that the function for addition, add, conflicts with common subcommands to programs like git.

It should be changed to something like __add__, like python.

Show error when the file passed to `cd` doesn't exist

Current behaviour:

(dune) /home/ludwig$ cd thisdoesnotexist
(dune) /home/ludwig$

Expected behaviour:

(dune) /home/ludwig$ cd thisdoesnotexist
error: The directory thisdoesnotexist does not exist
(dune) /home/ludwig$

A suggestion in case of a typo would also be nice:

(dune) /home/ludwig$ cd deveolp
error: The directory deveolp does not exist
did you mean develop? (y)
(dune) /home/ludwig$ y
(dune) /home/ludwig/develop$

Widgets could be used to make it prettier:

But that's just a "nice to have".

By the way, scripting in dune is really pleasant! I wrote some code to print the widgets above, which was quite easy. Thank you for creating this ❤️

Optimize string allocations in parser

In the parser, as pointed out by @Aloso in #43, the parser spends most of its time allocating strings.

I believe this is probably from making calls to input.to_string(), which takes the entire remainder of the input program and allocates it to a string.

I think we may be able to optimize this by using Display instead of ToString, and then using a format call such as: format!("{:<width$}", input, width=MAX_INPUT_TO_SAVE). This would only grab the first MAX_INPUT_TO_SAVE number of characters from the input, as the user is not going to want to read the whole input to see where it failed to parse.

However, this specific fix would only work if format! only calls as much as it needs to given the format specifiers.

Another easier (and probably better) fix that would guarantee that format! only calls as much as it needs to is to limit it ourselves in the implementation of Display for Tokens. We might do something like the following:

const MAX_TOKENS_TO_DISPLAY: usize = 15;

impl fmt::Display for Tokens<'_> {
    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
        // Only display for first 15 tokens
        // from the start of the input.
        for t in self.0.iter().take(MAX_TOKENS_TO_DISPLAY) {
            write!(f, "{} ", t)?;
        }
        Ok(())
    }
}

I think this implementation would be a good solution.

Cannot redirect output to a file with `>`

Hello,

I'd like to redirect output to a file like:

curl https://loon.lib.utk.edu/scout-pdfs/MS.2271.pdf > MS_2271.pdf

Dune doesn't like this and ultimately I end up with something like this:

curl https://loon.lib.utk.edu/scout-pdfs/MS.2271.pdf > MS_2271.pdf
                            > 
syntax error:
 | on input `://loon.lib.utk.edu/scout-pdfs/MS.2271.pdf > MS_2271.pdf`
 | expected Eof

Create guide/examples for doing common things in dune

There are already a lot of features that are very well explained through the README. I feel like the project would profit from a step-by-step guide through some of the most common scenarios:

• completion
• aliases (e.g. dc for docker compose)
• examples for integrating common developer tools like nvm, volta, rbenv, frum, zoxide, etc...

I would also like to offer my input on these topics but feel unable to write it myself as I don't really know that much about the shell.

EDIT: Maybe there could be a section detailing the differences between a posix-like shell and dune or even a table to compare implementations of different features in each shell.

Bug: Any command kills dunesh with SIGSEGV

Valgrind output:

▲ ~ dunesh
(dune) /home/al1-ce$ ls
fish: Job 1, 'dunesh' terminated by signal SIGSEGV (Address boundary error)
△ ~ valgrind dunesh
==25291== Memcheck, a memory error detector
==25291== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
==25291== Using Valgrind-3.23.0 and LibVEX; rerun with -h for copyright info
==25291== Command: dunesh
==25291==
ls
(dune) /home/al1-ce$ ls
==25291== Invalid read of size 8
==25291==    at 0x28EA80: prettytable::TableSlice::get_all_column_width (accum.rs:53)
==25291==    by 0x28F1CE: __print<prettytable::utils::StringWriter, fn(&prettytable::row::Row, &mut prettytable::utils::StringWriter, &prettytable::format::TableFormat, &[usize]) -> core::result::Result<usize, std::io::error::Error>> (lib.rs:143)
==25291==    by 0x28F1CE: print<prettytable::utils::StringWriter> (lib.rs:169)
==25291==    by 0x28F1CE: <prettytable::TableSlice as core::fmt::Display>::fmt (lib.rs:410)
==25291==    by 0x2D8F8D: core::fmt::write (mod.rs:1232)
==25291==    by 0x2D9E52: core::fmt::Formatter::write_fmt (mod.rs:1686)
==25291==    by 0x26D6CD: <dune::expr::Expression as core::fmt::Display>::fmt (expr.rs:283)
==25291==    by 0x25B5BD: to_string<dune::expr::Expression> (string.rs:2532)
==25291==    by 0x25B5BD: {closure#1} (expr.rs:480)
==25291==    by 0x25B5BD: {closure#0}<(alloc::string::String, dune::expr::Expression), (alloc::string::String, alloc::string::String), (), core::ops::control_flow::ControlFlow<(alloc::string::String, alloc::string::String), ()>, dune::expr::{impl#14}::eval_mut::{closure_env#1}, core::iter::traits::iterator::Iterator::find::check::{closure_env#0}<(alloc::string::String, alloc::string::String), &mut dune::expr::{impl#14}::eval_mut::{closure_env#2}>> (map.rs:91)
==25291==    by 0x25B5BD: try_fold<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, (), core::iter::adapters::map::map_try_fold::{closure_env#0}<(alloc::string::String, dune::expr::Expression), (alloc::string::String, alloc::string::String), (), core::ops::control_flow::ControlFlow<(alloc::string::String, alloc::string::String), ()>, dune::expr::{impl#14}::eval_mut::{closure_env#1}, core::iter::traits::iterator::Iterator::find::check::{closure_env#0}<(alloc::string::String, alloc::string::String), &mut dune::expr::{impl#14}::eval_mut::{closure_env#2}>>, core::ops::control_flow::ControlFlow<(alloc::string::String, alloc::string::String), ()>> (iterator.rs:2299)
==25291==    by 0x25B5BD: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold (map.rs:117)
==25291==    by 0x268D44: find<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, &mut dune::expr::{impl#14}::eval_mut::{closure_env#2}> (iterator.rs:2768)
==25291==    by 0x268D44: next<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}> (filter.rs:56)
==25291==    by 0x268D44: extend_desugared<(alloc::string::String, alloc::string::String), alloc::alloc::Global, core::iter::adapters::filter::Filter<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}>> (mod.rs:2824)
==25291==    by 0x268D44: spec_extend<(alloc::string::String, alloc::string::String), core::iter::adapters::filter::Filter<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}>, alloc::alloc::Global> (spec_extend.rs:17)
==25291==    by 0x268D44: from_iter<(alloc::string::String, alloc::string::String), core::iter::adapters::filter::Filter<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}>> (spec_from_iter_nested.rs:43)
==25291==    by 0x268D44: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter (spec_from_iter.rs:33)
==25291==    by 0x27DD2B: from_iter<(alloc::string::String, alloc::string::String), core::iter::adapters::filter::Filter<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}>> (mod.rs:2724)
==25291==    by 0x27DD2B: collect<core::iter::adapters::filter::Filter<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}>, alloc::vec::Vec<(alloc::string::String, alloc::string::String), alloc::alloc::Global>> (iterator.rs:1891)
==25291==    by 0x27DD2B: <alloc::collections::btree::map::BTreeMap<K,V> as core::iter::traits::collect::FromIterator<(K,V)>>::from_iter (map.rs:2109)
==25291==    by 0x270A8E: dune::expr::Expression::eval_mut (iterator.rs:1891)
==25291==    by 0x26E7EB: dune::expr::Expression::eval (expr.rs:421)
==25291==    by 0x16616E: dunesh::repl (bin.rs:487)
==25291==    by 0x16A331: dunesh::main (bin.rs:655)
==25291==  Address 0x252400002524 is not stack'd, malloc'd or (recently) free'd
==25291==
==25291==
==25291== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==25291==  Access not within mapped region at address 0x2543FF000084
==25291==    at 0x28EA80: prettytable::TableSlice::get_all_column_width (accum.rs:53)
==25291==    by 0x28F1CE: __print<prettytable::utils::StringWriter, fn(&prettytable::row::Row, &mut prettytable::utils::StringWriter, &prettytable::format::TableFormat, &[usize]) -> core::result::Result<usize, std::io::error::Error>> (lib.rs:143)
==25291==    by 0x28F1CE: print<prettytable::utils::StringWriter> (lib.rs:169)
==25291==    by 0x28F1CE: <prettytable::TableSlice as core::fmt::Display>::fmt (lib.rs:410)
==25291==    by 0x2D8F8D: core::fmt::write (mod.rs:1232)
==25291==    by 0x2D9E52: core::fmt::Formatter::write_fmt (mod.rs:1686)
==25291==    by 0x26D6CD: <dune::expr::Expression as core::fmt::Display>::fmt (expr.rs:283)
==25291==    by 0x25B5BD: to_string<dune::expr::Expression> (string.rs:2532)
==25291==    by 0x25B5BD: {closure#1} (expr.rs:480)
==25291==    by 0x25B5BD: {closure#0}<(alloc::string::String, dune::expr::Expression), (alloc::string::String, alloc::string::String), (), core::ops::control_flow::ControlFlow<(alloc::string::String, alloc::string::String), ()>, dune::expr::{impl#14}::eval_mut::{closure_env#1}, core::iter::traits::iterator::Iterator::find::check::{closure_env#0}<(alloc::string::String, alloc::string::String), &mut dune::expr::{impl#14}::eval_mut::{closure_env#2}>> (map.rs:91)
==25291==    by 0x25B5BD: try_fold<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, (), core::iter::adapters::map::map_try_fold::{closure_env#0}<(alloc::string::String, dune::expr::Expression), (alloc::string::String, alloc::string::String), (), core::ops::control_flow::ControlFlow<(alloc::string::String, alloc::string::String), ()>, dune::expr::{impl#14}::eval_mut::{closure_env#1}, core::iter::traits::iterator::Iterator::find::check::{closure_env#0}<(alloc::string::String, alloc::string::String), &mut dune::expr::{impl#14}::eval_mut::{closure_env#2}>>, core::ops::control_flow::ControlFlow<(alloc::string::String, alloc::string::String), ()>> (iterator.rs:2299)
==25291==    by 0x25B5BD: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold (map.rs:117)
==25291==    by 0x268D44: find<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, &mut dune::expr::{impl#14}::eval_mut::{closure_env#2}> (iterator.rs:2768)
==25291==    by 0x268D44: next<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}> (filter.rs:56)
==25291==    by 0x268D44: extend_desugared<(alloc::string::String, alloc::string::String), alloc::alloc::Global, core::iter::adapters::filter::Filter<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}>> (mod.rs:2824)
==25291==    by 0x268D44: spec_extend<(alloc::string::String, alloc::string::String), core::iter::adapters::filter::Filter<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}>, alloc::alloc::Global> (spec_extend.rs:17)
==25291==    by 0x268D44: from_iter<(alloc::string::String, alloc::string::String), core::iter::adapters::filter::Filter<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}>> (spec_from_iter_nested.rs:43)
==25291==    by 0x268D44: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter (spec_from_iter.rs:33)
==25291==    by 0x27DD2B: from_iter<(alloc::string::String, alloc::string::String), core::iter::adapters::filter::Filter<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}>> (mod.rs:2724)
==25291==    by 0x27DD2B: collect<core::iter::adapters::filter::Filter<core::iter::adapters::map::Map<alloc::collections::btree::map::IntoIter<alloc::string::String, dune::expr::Expression, alloc::alloc::Global>, dune::expr::{impl#14}::eval_mut::{closure_env#1}>, dune::expr::{impl#14}::eval_mut::{closure_env#2}>, alloc::vec::Vec<(alloc::string::String, alloc::string::String), alloc::alloc::Global>> (iterator.rs:1891)
==25291==    by 0x27DD2B: <alloc::collections::btree::map::BTreeMap<K,V> as core::iter::traits::collect::FromIterator<(K,V)>>::from_iter (map.rs:2109)
==25291==    by 0x270A8E: dune::expr::Expression::eval_mut (iterator.rs:1891)
==25291==    by 0x26E7EB: dune::expr::Expression::eval (expr.rs:421)
==25291==    by 0x16616E: dunesh::repl (bin.rs:487)
==25291==    by 0x16A331: dunesh::main (bin.rs:655)
==25291==  If you believe this happened as a result of a stack
==25291==  overflow in your program's main thread (unlikely but
==25291==  possible), you can try to increase the size of the
==25291==  main thread stack using the --main-stacksize= flag.
==25291==  The main thread stack size used in this run was 8388608.
==25291==
==25291== HEAP SUMMARY:
==25291==     in use at exit: 203,713 bytes in 2,640 blocks
==25291==   total heap usage: 8,581 allocs, 5,941 frees, 862,385 bytes allocated
==25291==
==25291== LEAK SUMMARY:
==25291==    definitely lost: 0 bytes in 0 blocks
==25291==    indirectly lost: 0 bytes in 0 blocks
==25291==      possibly lost: 304 bytes in 1 blocks
==25291==    still reachable: 203,409 bytes in 2,639 blocks
==25291==         suppressed: 0 bytes in 0 blocks
==25291== Rerun with --leak-check=full to see details of leaked memory
==25291==
==25291== For lists of detected and suppressed errors, rerun with: -s
==25291== ERROR SUMMARY: 2 errors from 1 contexts (suppressed: 0 from 0)
fish: Job 1, 'valgrind dunesh' terminated by signal SIGSEGV (Address boundary error)

Expand ~

With cd ~ I would like to go to my home directory. I'd also like to be able to type vim ~/.dune-prelude to edit the prelude.

To accomplish this, dune should replace ~ at the beginning of a path with the home directory. A string should be considered a path if it

1. Is not in double quotes
2. It equals ~, or it starts with ~/

For example, when typing

$ foo bar ~ ~~ ~baz ~/hello ./~

Then the following arguments should be passed to foo (assuming the home directory is /home/aloso):

[ "bar", "/home/aloso", "~~", "~baz", "/home/aloso/hello", "./~" ]

(This is also what bash does.)

stdin redirection support

Currently, if I want to pipe something into stdin for a command, with most bash-based shells, I can run:

foo < bar.txt

and this gets the job done just right. With Dune(🚀), it seems I am stuck with:

cat bar.txt | foo

Better Windows support

Right now, it's pretty difficult to use Dune on Windows effectively because all of the filesystem commands such as mv, cp, ls, etc. are not implemented as programs (like on Linux), but as builtins to Powershell and cmd.exe. This means that it's basically impossible to manipulate files and directories in Dune on Windows right now.

touch doesn't work as expected

(dune) /Users/scaevola$ touch ~/.dune-prelude
touch: ~/.dune-prelude: No such file or directory

I want to create a file which shouldn't exist

Changelog

Just realized that dune doesn't have a changelog yet. I recommend using this template: https://keepachangelog.com/en/1.0.0/

Check if Unicode escape sequences are valid

for example, evaluating "test \u{D800}" panics. This should display a parser error.

Using backtick

Is it possible to use shell backtick to expand variables eg. vim `which COMMAND

Tab completion on paths with spaces

Tab completion seems to fail when there is a space in the current working directory, such as this example:

^ Tabbing here does nothing, but is expected to complete to hmm

I suspect that this is an issue with rustyline::completion::FilenameCompleter, but I've yet to investigate it further. It might be best to implement this functionality ourselves: it's just when finding the matching paths that we use FilenameCompleter, we do most of the heavy lifting to get the incomplete symbol relative to Dune's CWD variable.

Dune executable should accept `--help` and `--version` arguments

Typing dune --version should print version information. dune --help should print help information or open the man page.

Also, many shells accept a -c, --command argument to execute commands and print the result, e.g.

~ $ bash -c "echo hello world"
hello world
~ $

Add recursion limit

There needs to be a recursion limit without crashing. This can be accomplished by adding a depth: usize parameter to Expression::eval_mut.

how can i add dune as a default shell? (sorry, are questions allowed?)

hey, i've been meaning to do so for a while, but i haven't gotten around to it
how can i add dune as a default shell? online guides say whatever shells are installed should have an entry in /etc/shells but dune's installed to my /home directory!! (i think)
it's in /home/.cargo/bin and it works fine, would copying it as root to /bin and/or /usr/bin, then adding an entry for it to /etc/shells work?

Broken string escaping

The parser allows the \/ escape sequence, which is not supported by the used snailquote crate. It also parses a backslash followed by four hexadecimal characters, which is not valid. Using either of these causes a panic.

I propose to remove the snailquote dependency, because it has features we don't need, and implementing it manually isn't that hard.

I want to implement this.

Characters that should be allowed in a symbol

*

Common in regular expressions and globs, e.g. foo | grep a.*

Supporting this shouldn't be a problem.

=

Needed e.g. when running cargo --config KEY=VALUE

Supporting this is more difficult, because maps use the syntax { key1=value1, key2=value2 }. An alternative would be to require spaces: { key1 = value1, key2 = value2 } or accept = in symbols except when parsing the key of an object.

Storing output of commands doesn't work

$ let x = which which
/usr/bin/which
$ echo x
None
$ 

As you can see, the output of which is printed, but it is not stored in the variable x.