Coder Social home page Coder Social logo

reedline's Introduction

A feature-rich line editor - powering Nushell

GitHub Crates.io docs.rs GitHub Workflow Status codecov Discord

Introduction

Reedline is a project to create a line editor (like bash's readline or zsh's zle) that supports many of the modern conveniences of CLIs, including syntax highlighting, completions, multiline support, Unicode support, and more. It is currently primarily developed as the interactive editor for nushell (starting with v0.60) striving to provide a pleasant interactive experience.

Outline

Examples

For the full documentation visit https://docs.rs/reedline. The examples should highlight how you enable the most important features or which traits can be implemented for language-specific behavior.

Basic example

// Create a default reedline object to handle user input

use reedline::{DefaultPrompt, Reedline, Signal};

let mut line_editor = Reedline::create();
let prompt = DefaultPrompt::default();

loop {
    let sig = line_editor.read_line(&prompt);
    match sig {
        Ok(Signal::Success(buffer)) => {
            println!("We processed: {}", buffer);
        }
        Ok(Signal::CtrlD) | Ok(Signal::CtrlC) => {
            println!("\nAborted!");
            break;
        }
        x => {
            println!("Event: {:?}", x);
        }
    }
}

Integrate with custom keybindings

// Configure reedline with custom keybindings

//Cargo.toml
//    [dependencies]
//    crossterm = "*"

use {
  crossterm::event::{KeyCode, KeyModifiers},
  reedline::{default_emacs_keybindings, EditCommand, Reedline, Emacs, ReedlineEvent},
};

let mut keybindings = default_emacs_keybindings();
keybindings.add_binding(
    KeyModifiers::ALT,
    KeyCode::Char('m'),
    ReedlineEvent::Edit(vec![EditCommand::BackspaceWord]),
);
let edit_mode = Box::new(Emacs::new(keybindings));

let mut line_editor = Reedline::create().with_edit_mode(edit_mode);

Integrate with History

// Create a reedline object with history support, including history size limits

use reedline::{FileBackedHistory, Reedline};

let history = Box::new(
  FileBackedHistory::with_file(5, "history.txt".into())
    .expect("Error configuring history with file"),
);
let mut line_editor = Reedline::create()
  .with_history(history);

Integrate with custom syntax Highlighter

// Create a reedline object with highlighter support

use reedline::{ExampleHighlighter, Reedline};

let commands = vec![
  "test".into(),
  "hello world".into(),
  "hello world reedline".into(),
  "this is the reedline crate".into(),
];
let mut line_editor =
Reedline::create().with_highlighter(Box::new(ExampleHighlighter::new(commands)));

Integrate with custom tab completion

// Create a reedline object with tab completions support

use reedline::{default_emacs_keybindings, ColumnarMenu, DefaultCompleter, Emacs, KeyCode, KeyModifiers, Reedline, ReedlineEvent, ReedlineMenu};

let commands = vec![
  "test".into(),
  "hello world".into(),
  "hello world reedline".into(),
  "this is the reedline crate".into(),
];
let completer = Box::new(DefaultCompleter::new_with_wordlen(commands.clone(), 2));
// Use the interactive menu to select options from the completer
let completion_menu = Box::new(ColumnarMenu::default().with_name("completion_menu"));
// Set up the required keybindings
let mut keybindings = default_emacs_keybindings();
keybindings.add_binding(
    KeyModifiers::NONE,
    KeyCode::Tab,
    ReedlineEvent::UntilFound(vec![
        ReedlineEvent::Menu("completion_menu".to_string()),
        ReedlineEvent::MenuNext,
    ]),
);

let edit_mode = Box::new(Emacs::new(keybindings));

let mut line_editor = Reedline::create()
    .with_completer(completer)
    .with_menu(ReedlineMenu::EngineCompleter(completion_menu))
    .with_edit_mode(edit_mode);

Integrate with Hinter for fish-style history autosuggestions

// Create a reedline object with in-line hint support

//Cargo.toml
//  [dependencies]
//  nu-ansi-term = "*"

use {
  nu_ansi_term::{Color, Style},
  reedline::{DefaultHinter, Reedline},
};

let mut line_editor = Reedline::create().with_hinter(Box::new(
  DefaultHinter::default()
  .with_style(Style::new().italic().fg(Color::LightGray)),
));

Integrate with custom line completion Validator

// Create a reedline object with line completion validation support

use reedline::{DefaultValidator, Reedline};

let validator = Box::new(DefaultValidator);

let mut line_editor = Reedline::create().with_validator(validator);

Use custom EditMode

// Create a reedline object with custom edit mode
// This can define a keybinding setting or enable vi-emulation

use reedline::{
    default_vi_insert_keybindings, default_vi_normal_keybindings, EditMode, Reedline, Vi,
};

let mut line_editor = Reedline::create().with_edit_mode(Box::new(Vi::new(
    default_vi_insert_keybindings(),
    default_vi_normal_keybindings(),
)));

Crate features

  • clipboard: Enable support to use the SystemClipboard. Enabling this feature will return a SystemClipboard instead of a local clipboard when calling get_default_clipboard().
  • bashisms: Enable support for special text sequences that recall components from the history. e.g. !! and !$. For use in shells like bash or nushell.
  • sqlite: Provides the SqliteBackedHistory to store richer information in the history. Statically links the required sqlite version.
  • sqlite-dynlib: Alternative to the feature sqlite. Will not statically link. Requires sqlite >= 3.38 to link dynamically!
  • external_printer: Experimental: Thread-safe ExternalPrinter handle to print lines from concurrently running threads.

Are we prompt yet? (Development status)

Reedline has now all the basic features to become the primary line editor for nushell

  • General editing functionality, that should feel familiar coming from other shells (e.g. bash, fish, zsh).
  • Configurable keybindings (emacs-style bindings and basic vi-style).
  • Configurable prompt
  • Content-aware syntax highlighting.
  • Autocompletion (With graphical selection menu or simple cycling inline).
  • History with interactive search options (optionally persists to file, can support multiple sessions accessing the same file)
  • Fish-style history autosuggestion hints
  • Undo support.
  • Clipboard integration
  • Line completeness validation for seamless entry of multiline command sequences.

Areas for future improvements

  • Support for Unicode beyond simple left-to-right scripts
  • Easier keybinding configuration
  • Support for more advanced vi commands
  • Visual selection
  • Smooth experience if completion or prompt content takes long to compute
  • Support for a concurrent output stream from background tasks to be displayed, while the input prompt is active. ("Full duplex" mode)

For more ideas check out the feature discussion or hop on the #reedline channel of the nushell discord.

Development history

If you want to follow along with the history of how reedline got started, you can watch the recordings of JT's live-coding streams.

Playlist: Creating a line editor in Rust

Alternatives

For currently more mature Rust line editing check out:

reedline's People

Contributors

ahkrr avatar andreistan26 avatar aslynatilla avatar bnprks avatar clementnerma avatar cschierig avatar drbrain avatar e3uka avatar elferherrera avatar fdncred avatar hofer-julian avatar jasonrhansen avatar jmoore34 avatar kubouch avatar maxomatic458 avatar morzel85 avatar mzanrosso avatar nibon7 avatar nixypanda avatar nschoellhorn avatar perlindgren avatar rgwood avatar sholderbach avatar sophiajt avatar stormasm avatar tastaturtaste avatar uasi avatar valpackett avatar windsoilder avatar ysthakur avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

reedline's Issues

No support to overwrite keybindings.

Right now the way the keybindings are structured (i.e. in a list form) we can't really overwrite an existing keybinding. Is this intentional?

Cannot type curly brackets in engine-q

After building and running engine-q, I cannot type { or }.
I use an italian keyboard, where opening and closing curly brackets implies pressing: Shift + Alt Gr + Key.

Note that even just typing [ or ] seems to present a problem, and it implies pressing: Alt Gr + Key.
In this case, the square brackets are printed, but they can only be cancelled by pressing Backspace twice; it looks like "two characters were pushed" instead of one.

Feature Vision Discussion: History

This issue is part of the larger vision discussion managed in #63 to brainstorm and prioritize ideas around features that would make reedline as a line editor for you both viable as well as pleasant to use. Feel free to up-vote features you depend on in other tools/shells as well as suggest new ideas!

History Features

  • Ability to save commands in history
  • Ability to save multi-line commands in history
  • Ability to traverse backward and forward within history via keybindings
  • Ability to search history
  • Ability to search history and travese backward and forward within the search criteria
  • Ability to have rich data within the history, such as:
    • run time in ms
    • date/time command was ran
    • sessionid of who ran the command assuming multiple sessions
  • Ability to have multiple sessions successfull write to history file(s)
  • Ability for each session to have it's own history or the appearance of it's own history so within history searching in a single session you only see the commands you created in that session
  • Ability to send all commands from all sessions into a global history which search interfaces can use
  • Ability to have plugable technologies to make history searching and recall extensible

Feature Vision Discussion: Pop-ups/Annotations

This issue is part of the larger vision discussion managed in #63 to brainstorm and prioritize ideas around features that would make reedline as a line editor for you both viable as well as pleasant to use. Feel free to up-vote features you depend on in other tools/shells as well as suggest new ideas!

PopUps/PopUnders Features

  • Ability to have popups for things like progress bars
  • Ability to have popunders, think of this as row(s) under your prompt, to display things like statuses, documentation, or hints
  • Ability to have extensibility

CI failures

failures:

---- src/lib.rs - (line 120) stdout ----
Test executable failed (exit code 101).

stderr:
thread 'main' panicked at 'called Result::unwrap() on an Err value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', src/lib.rs:31:36
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

---- src/lib.rs - (line 34) stdout ----
Test executable failed (exit code 101).

stderr:
thread 'main' panicked at 'called Result::unwrap() on an Err value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', src/lib.rs:26:35
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

---- src/lib.rs - (line 98) stdout ----
Test executable failed (exit code 101).

stderr:
thread 'main' panicked at 'called Result::unwrap() on an Err value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', src/lib.rs:21:35
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

---- src/lib.rs - (line 79) stdout ----
Test executable failed (exit code 101).

stderr:
thread 'main' panicked at 'called Result::unwrap() on an Err value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', src/lib.rs:18:35
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

failures:
src/lib.rs - (line 120)
src/lib.rs - (line 34)
src/lib.rs - (line 79)
src/lib.rs - (line 98)

test result: FAILED. 15 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 6.84s

error: test failed, to rerun pass '--doc'
Error: The process '/usr/share/rust/.cargo/bin/cargo' failed with exit code 101

Resizing window causes lag/animation of resize

similar to #157

When the window is resized, all the resize events are used, causing the resize to lag and animate from one size to another.

We may want to check the event stream for other resize events and only use the latest one.

[ENH] Persist `History` in a file

Type: Good first issue

Add support to load/store history to a file similar to ~/.bash_history. Make the path reasonably configurable.

Bonus: Only write the new lines of the current session and avoid races on the file.

Brainstorming Features

1. Line Editor UI/UX Features

Brainstorming of ideas about what features would make a line editor and programs that use it best-in-class. I'm interested to see which of these people agree with. These aren't necessarily MVP type ideas and perhaps even greater than 1.0 ideas.

1.2. Prompt Features #64

  • Ability to have Colors/Styles/Themes
  • Ability to have Left Side Prompt
  • Ability to have Right Side Prompt
  • Ability to have Animated Prompt - Clock, Ansi Spinners like -|/
  • Ability to use external prompts like starship or oh-my-posh
  • Ability to make prompt extensible

1.3. Completions Features #65

  • Ability to works as expected or better
    • File and folder completion
      • zsh-style partial path expansion
    • Environment variable completion $nu.env.A<tab>
    • Table column completion (ls).n<tab>
    • Sub-Command completion str <tab>
    • Other
  • Ability to add external completions for things like git
  • Ability to make completions extensible

1.4. History Features #66

  • Ability to save commands in history
  • Ability to save multi-line commands in history
  • Ability to traverse backward and forward within history via keybindings
  • Ability to search history
  • Ability to search history and travese backward and forward within the search criteria
  • Ability to have rich data within the history, such as:
    • run time in ms
    • date/time command was ran
    • sessionid of who ran the command assuming multiple sessions
  • Ability to have multiple sessions successfull write to history file(s)
  • Ability for each session to have it's own history or the appearance of it's own history so within history searching in a single session you only see the commands you created in that session
  • Ability to send all commands from all sessions into a global history which search interfaces can use
  • Ability to have plugable technologies to make history searching and recall extensible
  • Ability to reset history

1.5. Hinting Features #65

  • Ability to change the color of the hints
  • Ability to create hints from history
  • Ability to have plugable hints specific to a vertical
    • Nushell
    • Azure
    • Dataframe/Polars/DataScience
    • Maths
    • Other
  • Ability to complete hints with keybinding
  • Ability to change location of hints
    • On prompt line
    • Under prompt line
    • On status bar at the top
    • On staus bar at the bottom

1.6. Syntax Highlighting Features #67

  • Ability to define word-types to highlight as you type and continue coloring even in failure modes
    • Keywords
    • Commands
    • External Commands
    • Bareword
    • Punctuation
    • Sub-Commands
    • Braces {}, Brackets [], Parentheses ()
    • Error modes
    • Operators
    • Paths
    • Pipe
    • Arguments/Flags
    • Variables
    • Comments
    • DataTypes (strings, int, datetime, duration, bool, units, etc)
    • Other
  • Ability to have add things you want syntax highlighted through extensibility

1.7. PopUps/PopUnders Features #68

  • Ability to have popups for things like progress bars
  • Ability to have popunders, think of this as row(s) under your prompt, to display things like statuses, documentation, or hints
  • Ability to have extensibility

1.8. Keybinding #69

  • All major functionality should be exposed via keybindings
  • Ability to have multiple keybindings
    • VI Mode
    • Emacs Mode
    • JT Mode (the ability to code like JT)
    • Other
  • Ability to load keybindings from file
  • Ability to execute previously defined code or externals via keybindings
  • Ability to have keybinding chords (multiple keyboard shortcuts in succession execute a particular internal or external command)

1.9. Miscellaneous #70

  • Validation to trigger multiline on incomplete lines
  • Tested behaviour under various environments
    • OSes
    • Terminal emulators: gnome-terminal, iTerm, alacritty
    • Multiplexers: screen, tmux
    • As ssh session
  • possible plugins to use file searches and terminal browsers: fzf, skim, broot, zoxide, ranger, etc
  • Responsive resizing

Unify functionality for history search

Currently there are two ways to search the History.

  1. There is the history_search::BasicSearch implementation used to provide the Ctrl+R backwards search.
  • It is integrated with a modal search prompt but lacks functionality to walk the search back and forward
  • seems to have accumulated some issues due to unrelated development (Using the arrow keys breaks the prompt)
  • Independent struct more amenable in an API to use more advanced search providers (e.g. fuzzy matching)
  1. History itself now provides a simple stateful prefix search.
  • Keeps track of cursor and searches based on that -> up and down very easy.
  • More readable and self contained search logic
  • Managing the search prefix is up to the user.

Feature Vision Discussion: Syntax Highlighting

This issue is part of the larger vision discussion managed in #63 to brainstorm and prioritize ideas around features that would make reedline as a line editor for you both viable as well as pleasant to use. Feel free to up-vote features you depend on in other tools/shells as well as suggest new ideas!

Syntax Highlighting Features

  • Ability to define word-types to highlight as you type and continue coloring even in failure modes
    • Keywords
    • Commands
    • External Commands
    • Bareword
    • Punctuation
    • Sub-Commands
    • Braces {}, Brackets [], Parentheses ()
    • Error modes
    • Operators
    • Paths
    • Pipe
    • Arguments/Flags
    • Variables
    • Comments
    • DataTypes (strings, int, datetime, duration, bool, units, etc)
    • Other
  • Ability to have add things you want syntax highlighted through extensibility

Mac: up-arrow history traversal

I can't get the up arrow to traverse history on my mac. I tried cargo run -- -k to see if crossterm recognized the keystroke but it doesn't see it either. I thought it used to work.

Some characters that use modifiers are not printed

Using a spanish keyboard layout, in order to access to the chars [ ] or { } one needs to use the modifier alt gr. Reedline does not print those characters when using this layout. If I change it to an english layout, the characters can be printed (they dont need the alt gr key).

This is the output from cargo run -- -k
image

It seems that the events are captured, but reedline doesnt print the characters

Make history multiline capable

As #108 introduced the support for multiline editing, we need to adapt the file based storage to account for entries spanning multiple lines. The current file format mimics the basic bash history files that are newline separated. Thus when starting a new session and loading the history from those files a multiline entry would be split in multiple entries.

By either escaping the newline characters or defining a regular record format this problem should be circumvented.

Steps:

  1. Write test cases for multiline history entries that should be recovered.
  2. Decide on a proper storage format
  3. Implement those changes for FileBackedHistory
  4. Validate the behavior in reedline
  5. Check if user input might be able to break this.

Move the clear screen/`Ctrl-L` logic inside the line editor engine

We currently handle the Ctrl-L screen clearing command by exiting the read_line() method with the Signal::CtrlL and keeping the line buffer state as is. This requires the read_line() method to respect the previous state of Reedline and collaboration from the side of the consumer.
This might be less than optimal. We might consider moving the Ctrl-L handling into the Reedline engine with a call to the painter and maybe provide the Option to disable this. A public method to clear from the side of the consumer is already existing.

Improving `read_line` output

From the twitch chat:

"boom_rang: In my mind the "exit" keyword is a concern of the REPL, read_line could return a enum where a string is one option and then Ctrl-C and Ctrl-D can have a special return each"

Move wrapping detection out of the single character insertion case

Currently wrapping detection is only performed when handling the insert of a single character...

reedline/src/engine.rs

Lines 813 to 828 in 75d9b4b

// Performing mutation here might incur a perf hit down this line when
// we would like to do multiple inserts.
// A simple solution that we can do is to queue up these and perform the wrapping
// check after the loop finishes. Will need to sort out the details.
EditCommand::InsertChar(c) => {
self.editor.insert_char(*c);
if self.require_wrapping() {
let position = cursor::position()?;
self.wrap(position)?;
}
self.editor.remember_undo_state(false);
self.repaint(prompt)?;
}

... but this is wrong as the line's content could also overflow on other operations (e.g. using the paste operations or undo) resulting in the requirement to move the cursor.

Thus we either do such a detection explicitly on inserting operations or move the check and cursor position update into the painter and do it there in the flow of the output.

#176 relates to other technical issues with the current implementation. #180 improved some of the details, but insertions in the middle of the line that might shift the cursor are not fully covered yet.

Properly output the list completions

Currently the list completions are not handled by the Painter thus are immediately overriden by a repaint.

Steps:

  1. Provide facilities in the painter to accept a completion list that can be painted if needed.
  2. Change the ListCompletionHandler to not use print!() directly and instead provide the information to be consumed by the painter
  3. Make sure to deal with potentially shifting offsets if printing new lines at the end off the screen

Correct cursor position if linewrapping occurs at the bottom of the screen

The cursor position has to be correctly calculated/updated if the line wraps and the screen coordinate changes as the extra line shifts everything at the bottom of the screen.

Constant polling of the crossterm::cursor::position has to be avoided as this causes a serve hit to performance.
When approximations are performed this needs to be flexible enough to take potential change to sizes of things into account:

  • unicode (there is a crate for that)
  • prompt
  • continuation prompt #112

Clear screen/Ctrl+L/Form Feed

To emulate the behavior in bash/zsh etc. :

Maybe we consider a separation of concerns between core line editor and typical interactive shell terminal to make screen clearing optional for non shell/REPL usecases e.g. single command entry like vi :

On the side of the line editor Engine:

  • Respond to Ctrl+L
  • Keep current line_buffer state (including cursor position/insertion_point)

Helper to provide the functionality (MVP: call this in main):

  • Print a screen worth of newlines (At least thats the behavior of
    FormFeed under gnome-terminal)
  • Scroll new prompt to first line and repaint including cursor position

Code Organisation/Design Discussion: Making Editing Mode do more work

Hey

Firstly, love the work you are doing on nushell and I hope one day this would be my primary shell. One of the things that are stopping me is the vi-mode. If I am correct you are experimenting with reedline to see if we can have something better for the end-user. I have a bit of interest in this particular project.

I was playing around with the idea - where we make the core part of reedline as a library that an edit mode plugin/module can make use of. Initially, I wanted to do break the Reedline struct in the following manner.

pub struct Reedline {
    painter: Painter,
    edit_mode: EditMode,
    prompt: Prompt,
    editing_engine: EditingEngine,
}

where the components would have had the following responsibilities
Painter: Handles all painting interactions.
EditMode: This should parse the KeyEvents that we get into a Vec<EditCommand>. This is more than an enum in this design
Prompt: -
EditingEngine: Handles the list of EditCommand.

The primary motivation was to have EditMode structured in a way where emcas and vi stuff can reside in a completely separate struct. Where they do not know about each other. The metric for code design that I was using was "how easy is it to get rid of this feature (vi editing mode or emacs editing mode)".

Eventually, I figured that the edit mode controls more stuff than just translation of KeyEvents to Vec<EditCommand>, like deciding on how to paint the prompt (VI: Normal, Visual, Insert), etc. So I thought of making a trait that will define the interface we will need from a line editor with an editing mode. ViReedline (or EmacsReedline).

Here is the trait in question. We can get a Painter object using a methond and all the methods can then have a default so we will not need to focus on stuff other than read_line function in it's implementors

trait LineEditor {
    fn print_line(&self);
    fn print_events(&self);
    fn print_crlf(&self);
    fn print_history(&self);
    fn clear_screen(&self);
    fn read_line(&self, prompt: Box<dyn Prompt>) -> Signal;
}

and the (VI) struct that will eventually implement it.

#[derive(Eq, PartialEq, Clone, Copy)]
enum Mode {
    Normal,
    Insert,
}

pub struct ViLineEditor {
    painter: Painter,
    // keybindings: Keybindings,
    mode: Mode,
    partial_command: Option<char>,
    edit_engine: EditEngine,
    need_full_repaint: bool,
}

The emacs counterpart would be less complex than this (as far as I can see)

pub struct EmacsLineEditor {
    painter: Painter,
    keybindings: Keybindings,
    edit_engine: EditEngine,
}

Let me know if this is a direction that makes sense to you, or if you would like to see how the codebase would look like if we chase this direction. I can then cook up a PR and share the same with you.

Incomplete coverage of operations by Undo/Redo functionality

At the moment not all available editing operations that affect the content of the line buffer are covered by the Undo/Redo logic.

e.g. using Alt+c to capitalize a character

The check for operations that require saving of the state is currently not exhaustive

reedline/src/engine.rs

Lines 846 to 865 in 82bc3ac

if [
EditCommand::MoveToEnd,
EditCommand::MoveToStart,
EditCommand::MoveLeft,
EditCommand::MoveRight,
EditCommand::MoveWordLeft,
EditCommand::MoveWordRight,
EditCommand::Backspace,
EditCommand::Delete,
EditCommand::BackspaceWord,
EditCommand::DeleteWord,
EditCommand::CutFromStart,
EditCommand::CutToEnd,
EditCommand::CutWordLeft,
EditCommand::CutWordRight,
]
.contains(command)
{
self.editor.remember_undo_state(true);
}

This should be replaced by a function using a fully specified match to catch any future additions of EditCommands.

Design decision: Coalesce manual delete or backspace as a single entry on the undo stack

The current undo stack is able to coalesce the set of characters typed to form a word to a single undoable entry for a word.

If we think this is reasonable, it would be great to also enable that for manual deletion of words by hitting either delete or backspace multiple times

Currently the word boundary is defined as whitespace:

if self.edits.len() > 1
&& self.edits.last()?.word_count() == self.line_buffer.word_count()
&& !is_after_action
{
self.edits.pop();
}

/// Counts the number of words in the buffer
pub fn word_count(&self) -> usize {
self.lines.trim().split_whitespace().count()
}

Ignore cursor movements in the undo stack

At the moment cursor moves are stored along with edits that affect the text in the undo stack.
This is probably not the desired behavior as you typically only want to undo edits (vi style replay of moves would probably a different advanced feature).

Current state

Moves are recorded, undos will follow the cursor (Current behavior on main)

undo_moving

(Moving backwards with left arrow, undo (Ctrl-z) a few times and redo (Ctrl-g) again)

Challenge

Simply ignoring the moves with the current architecture of the undo stack causes jumping of the cursor to unexpected positions.

Example discovered when excluding moves from tracking:

Excluding the moves as in 85dab62 has the weird side effect that after an undo the cursor jumps back to where it was after the last text changing operation
undo_test
(Deletion of is with Ctrl+w and then undo via Ctrl+z)

Potentially problematic part of the current implementation

Currently full line buffer states after edits are stored.

pub struct Editor {
line_buffer: LineBuffer,
cut_buffer: Box<dyn Clipboard>,
edits: Vec<LineBuffer>,
index_undo: usize,
}

pub fn reset_undo_stack(&mut self) {
self.edits = vec![LineBuffer::new()];
self.index_undo = 2;
}
fn get_index_undo(&self) -> usize {
if let Some(c) = self.edits.len().checked_sub(self.index_undo) {
c
} else {
0
}
}
pub fn undo(&mut self) {
// NOTE: Try-blocks should help us get rid of this indirection too
self.undo_internal();
}
pub fn redo(&mut self) {
// NOTE: Try-blocks should help us get rid of this indirection too
self.redo_internal();
}
fn redo_internal(&mut self) -> Option<()> {
if self.index_undo > 2 {
self.index_undo = self.index_undo.checked_sub(2)?;
self.undo_internal()
} else {
None
}
}
fn undo_internal(&mut self) -> Option<()> {
self.line_buffer = self.edits.get(self.get_index_undo())?.clone();
if self.index_undo <= self.edits.len() {
self.index_undo = self.index_undo.checked_add(1)?;
}
Some(())
}
pub fn remember_undo_state(&mut self, is_after_action: bool) -> Option<()> {
self.reset_index_undo();
if self.edits.len() > 1
&& self.edits.last()?.word_count() == self.line_buffer.word_count()
&& !is_after_action
{
self.edits.pop();
}
self.edits.push(self.line_buffer.clone());
Some(())
}
fn reset_index_undo(&mut self) {
self.index_undo = 2;
}

prior art as a reference

In this one image are several features that I'd love to see reedline strive to attain.

image

  • context aware syntax highlighting - note how the string 'Hello world' is colored differently from python keywords
  • configurable line continuation character(s) - note the ... on the second line
  • it's hard to miss the drop down combo box with scroll bar - no clue how to achieve this one, maybe tui? i guess this population would be language specific, but of course, i'm thinking about nushell. :) a use case is to use this with completions of sub commands in nushell and completions in general.
  • the status bar - would be nice to show hints, errors, etc in a status bar, opt-in configurable, of course

some of these we're well on our way to achieving.

This is taken from here and the github repo.

For reference, i found this collection of supported ansi escape sequences very informative.

To be clear, i'm not talking about baking in python or this python line editor. I'm just using it as a prior art reference.

Move prompt to the top/bottom of the terminal

Instead of repeating the prompt on every line, display the prompt only at the top/bottom of the terminal.

This is how it works now (the top/bottom lines denote terminal window boundary):

------------------------------------------------------------------------------------------------
~/git/extern/reedline                                                     12/05/2021 12:48:28 AM
〉foo
Our buffer: foo

~/git/extern/reedline                                                     12/05/2021 12:48:30 AM
〉bar
Our buffer: bar

~/git/extern/reedline                                                     12/05/2021 12:48:33 AM
〉baz
Our buffer: baz





------------------------------------------------------------------------------------------------

This is what I suggest:

------------------------------------------------------------------------------------------------
〉foo
Our buffer: foo

〉bar
Our buffer: bar

〉baz
Our buffer: baz







~/git/extern/reedline                                                     12/05/2021 12:48:33 AM
------------------------------------------------------------------------------------------------

Of course, this should be configurable.

Disadvantage of this is that you won't see the previous prompts so you cannot check e.g. what directory you were when running "foo". Whether this matters or not I think depends on personal preference.

Feature Vision Discussion: Prompt

This issue is part of the larger vision discussion managed in #63 to brainstorm and prioritize ideas around features that would make reedline as a line editor for you both viable as well as pleasant to use. Feel free to up-vote features you depend on in other tools/shells as well as suggest new ideas!

Prompt Features

  • Ability to have Colors/Styles/Themes
  • Ability to have Left Side Prompt
  • Ability to have Right Side Prompt
  • Ability to have Animated Prompt - Clock, Ansi Spinners like -|/
  • Ability to use external prompts like starship or oh-my-posh
  • Ability to make prompt extensible
  • Asynchronous prompt that can't be stalled by long running status extensions
  • Ability to have modules that can be used to construct part of the prompt like battery, ssh, git, memory, disk space, pwd, clock, exit status, etc

Feature Vision Discussion: Keybindings

This issue is part of the larger vision discussion managed in #63 to brainstorm and prioritize ideas around features that would make reedline as a line editor for you both viable as well as pleasant to use. Feel free to up-vote features you depend on in other tools/shells as well as suggest new ideas!

Keybinding Features

  • All major functionality should be exposed via keybindings
  • Ability to have multiple keybindings
    • VI Mode
    • Emacs Mode
    • JT Mode (the ability to code like JT)
    • Other
  • Ability to load keybindings from file
  • Ability to execute previously defined code or externals via keybindings
  • Ability to have keybinding chords (multiple keyboard shortcuts in succession execute a particular internal or external command)

On a mac running engine-q get Command k to work.

In nushell when I type Command-k it clears the screen and we are happy...

In engine-q it clears the screen and then
on the next keystroke
it takes me back to where I was in the terminal
when I typed Command-k

It is remembering where I last was and goes back there...

It would be great to have this working as I use it all the time...

History navigation requiring two enter presses

If you load up reedline, then hit up to find the history item you want, it seems that you have to currently hit enter twice to submit that as the input.

Would be great if you only had to hit enter once.

odd moving of text when on bottom row of terminal

this is hard to describe but it appears that in Windows using Windows Terminal, when you're on the bottom row of the terminal and you type something like 'now is the time for all good men' and hit enter, it briefly moves the text ~10 characters to the right, then accepts the text and prints it as normal. It's really odd and hard to explain.

I reproduce this by doing:

  1. clear
  2. cargo clean
  3. cargo run
  4. ls
  5. now is the time for all good men
  6. and watch it shift and then accept

i've tried to capture it with screentogif but i'm not able to.

ok, here's what it's doing. i changed screentogif to run at 60 fps and i caught the error. here are the 3 frames.

frame 1 - when i typed in some (any) text
image

frame2 - the flashing that i see, which i characterized above as ~10 chars to the right. it actually is moving the text to the right because it's prefixed by Our buffer:
image

frame3 - when it accepts the text
image

[Bug] Multiline repainting at the end of the screen

When entering a multiline new line (Alt-Enter) with the prompt positioned at the bottom of the terminal screen, repainting introduces erroneous new prompt lines on every repaint. This behavior stops when repositioning the prompt to the top via Ctrl-L.

Platform:
Linux/Ubuntu 20.04
Terminal Emulator:
gnome-terminal

image

Feature Vision Discussion: Miscellaneous

This issue is part of the larger vision discussion managed in #63 to brainstorm and prioritize ideas around features that would make reedline as a line editor for you both viable as well as pleasant to use. Feel free to up-vote features you depend on in other tools/shells as well as suggest new ideas!

Miscellaneous Features and Goals

  • Validation to trigger multiline on incomplete lines
  • Tested behaviour under various environments
    • OSes
    • Terminal emulators: gnome-terminal, iTerm, alacritty, windows terminal
    • Multiplexers: screen, tmux
    • As ssh session
  • possible plugins to use file searches and terminal browsers: fzf, skim, broot, zoxide, ranger, etc
  • Responsive resizing

Cursor flickering to wrong positions

During drawing of the content we have to explicitly turn of the cursor visibility by crossterm::cursor::Hide everytime we want to move away from the position were we actually want to display the cursor. After returning the cursor to it's correct position crossterm::cursor::Show is used.

Validate that we do this correctly for the following cases:

  • repainting only the buffer
  • repaint including the prompt
  • when drawing additional displays like completion lists

[ENH] Prompt segment API

Define a PromptSegment trait that specifies how we interact with a prompt segment which can display information about the environment (e.g. working directory, time, last return code, git branch...). A Vec of trait objects could be used to allow custom configs.

In the draw call we could print prompt segments in different possible locations: Separate line above/below, same line, left/right

Open questions:

Who is responsible for shrinking presented information if the window width is to low to print all segments?

Feature Vision Discussion: Completions/Hints

This issue is part of the larger vision discussion managed in #63 to brainstorm and prioritize ideas around features that would make reedline as a line editor for you both viable as well as pleasant to use. Feel free to up-vote features you depend on in other tools/shells as well as suggest new ideas!

Completions Features

  • Ability to works as expected or better
    • File and folder completion
      • zsh-style partial path expansion
    • Environment variable completion $nu.env.A<tab>
    • Table column completion (ls).n<tab>
    • Sub-Command completion str <tab>
    • Other
  • Ability to add external completions for things like git
  • Ability to make completions extensible

Hinting Features

  • Ability to change the color of the hints
  • Ability to create hints from history
  • Ability to have plugable hints specific to a vertical
    • Nushell
    • Azure
    • Dataframe/Polars/DataScience
    • Maths
    • Other
  • Ability to complete hints with keybinding
  • Ability to change location of hints
    • On prompt line
    • Under prompt line
    • On status bar at the top
    • On staus bar at the bottom

Problems with input requiring more lines than available in the terminal window screen

using engine-q, i was testing out pasting json and was presented with this panic.

this is the json i pasted.

{
  "name": "inc",
  "usage": "Increment a value or version. Optionally use the column of a table.",
  "extra_usage": "",
  "required_positional": [],
  "optional_positional": [],
  "rest_positional": {
    "name": "",
    "desc": "",
    "shape": "Any",
    "var_id": null
  },
  "named": [
    {
      "long": "help",
      "short": "h",
      "arg": null,
      "required": false,
      "desc": "Display this help message",
      "var_id": null
    },
    {
      "long": "major",
      "short": "M",
      "arg": null,
      "required": false,
      "desc": "increment the major version (eg 1.2.1 -> 2.0.0)",
      "var_id": null
    },
    {
      "long": "minor",
      "short": "m",
      "arg": null,
      "required": false,
      "desc": "increment the minor version (eg 1.2.1 -> 1.3.0)",
      "var_id": null
    },
    {
      "long": "patch",
      "short": "p",
      "arg": null,
      "required": false,
      "desc": "increment the patch version (eg 1.2.1 -> 1.2.2)",
      "var_id": null
    }
  ],
  "is_filter": false,
  "creates_scope": false,
  "category": "Default"
}

this is the backtrace

thread 'main' panicked at 'attempt to subtract with overflow', /Users/fdncred/.cargo/git/checkouts/reedline-e42026a78d91c510/c11aef2/src/engine.rs:928:60
stack backtrace:
   0: rust_begin_unwind
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/std/src/panicking.rs:517:5
   1: core::panicking::panic_fmt
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/panicking.rs:101:14
   2: core::panicking::panic
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/panicking.rs:50:5
   3: reedline::engine::Reedline::wrap
             at /Users/fdncred/.cargo/git/checkouts/reedline-e42026a78d91c510/c11aef2/src/engine.rs:928:60
   4: reedline::engine::Reedline::run_edit_commands
             at /Users/fdncred/.cargo/git/checkouts/reedline-e42026a78d91c510/c11aef2/src/engine.rs:822:25
   5: reedline::engine::Reedline::handle_editor_event
             at /Users/fdncred/.cargo/git/checkouts/reedline-e42026a78d91c510/c11aef2/src/engine.rs:600:21
   6: reedline::engine::Reedline::handle_event
             at /Users/fdncred/.cargo/git/checkouts/reedline-e42026a78d91c510/c11aef2/src/engine.rs:421:13
   7: reedline::engine::Reedline::read_line_helper
             at /Users/fdncred/.cargo/git/checkouts/reedline-e42026a78d91c510/c11aef2/src/engine.rs:406:39
   8: reedline::engine::Reedline::read_line
             at /Users/fdncred/.cargo/git/checkouts/reedline-e42026a78d91c510/c11aef2/src/engine.rs:315:22
   9: engine_q::main
             at ./src/main.rs:269:25
  10: core::ops::function::FnOnce::call_once
             at /rustc/59eed8a2aac0230a8b53e89d4e99d55912ba6b35/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.```

[BUG] Shrinking the window may cause prompt position repainting issues

Resizing the window vertically in a way that the current prompt position exceeds the new window height triggers the prompt repainting bug observed with #119 (in this case due to multiline wrapping)

Initial condition

image

keeping the entry buffer line within the resizing does not cause a problem
image

shrinking down one more line causes the problem
image

screen clearing not resetting some of the bookkeeping on a mac

On a mac if you type CMD K when you are at the bottom of the screen it clears the screen like it should but then redraws the cursor at the bottom of the screen. When you are in a regular shell / not reedline / it clears the screen like it should behave....

Ctrl L works fine as it should...

Make hints completable like `fish`

The inline hints in fish or zsh-autosuggestions are completable with the cursor at the end of the line by either hitting RIGHT_ARROW for the full line or ALT+RIGHT_ARROW to just complete the next token.
This should only be in effect in the standard edit mode (discussion for how to idiomaticallly handle that in vi mode necessary)

Attention:
Make sure there are no unwanted interactions/breakages with:

  • Arrow navigation
  • The full completion to be triggered by TAB

Autocomplete: recognize when user removed a completion, and suggest other

For example, I type:

git push it autocompletes origin; forming: git push origin.

Done! Pushed to origin.

But then I input and it is git push origin. Then I remove origin, but it still marks/suggests origin; suggestion:
In this case, display the next last used to autocomplete, in this case it autocompletes github, forming: git push github.

[Design][Pluggable Components] History and searching

Continuing with the theme of more pluggable components as discussed in #61. I wanted to get thoughts of people of the following History design. This is very close to what we already have and also takes into consideration #55.

Approach

Thinking about the ways we can search history. I could come up with the following ways -

  • Going forward/back in history normally
  • Going forward/back in history based on a prefix
  • Searching for a substring (optionally going forward and backwards)
  • Fuzzy search a given string (optionally going forward and backwards)

We can encapsulate this sort of a query into a struct like as follows -

pub enum HistoryNavigationQuery {
    Normal,
    PrefixSearch(String),
    SubstringSearch(String),
    // Suffix Search
    // Fuzzy Search
}

Now keeping this in mind we can have a couple of interfaces we can design for our History.

pub trait History {
    // append any given string (a command) into the history - store
    fn append(&mut self, entry: String) -> ();
    
    // This will set a new navigation setup and based on input query
    fn set_navigation(&mut self, navigation: HistoryNavigationQuery);
    
    // This moves the cursor backwards respecting the navigation query that is set
    // - Results in a no-op if the cursor is at the initial point
    fn back(&mut self);

     // This moves the cursor forwards respecting the navigation-query that is set
    // - Results in a no-op if the cursor is at the latest point
    fn forward(&mut self);
    
    // Returns the string (if present) at the cursor
    fn string_at_cursor(&self) -> Option<String>;
    
    // we will need an `init` method to I guess which Reedline can call to setup the more complicated ones.
}

Some features of this particular design

  • We can now have multiple implementors of this like FileBackedHistory, InMemoryHistory, DBBackedHistory, etc.
  • We can add any level of intelligence in the implementors as we like as long as they can adhere to this interface.
  • All the extensions would be to the type, on the other hand we would have to create more and more functions: like go_back, go_forward, go_back_with_prefix, go_forward_with_prefix, go_back_with_substring, go_back_with_fuzzy, etc
  • [Con] We can only have one kind of history navigation at a time in one Reedline process, as we maintain an internal state based on this info. As far as I can see this should be a reasonable assumption to make as a person will only do one kind of search. Well we can circumvent this by using a HashMap to store queries I guess, but I don't think that complication is necessary given that we can assume that there isn't more than one history navigation going on at a given time in one process.

Implementation

Based on whatever the discussion we have here I can start working on this. Though this will conflict heavily with #60. I am not super sure of what the state is there.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.