kurtlawrence / papyrus

(Rust) repl

Papyrus - A rust REPL and script running tool.

This project is no longer maintained.

For a similar REPL tool with a refreshing way to interact with data, checkout the ogma Project

See the rs docs and the guide. Look at progress and contribute on github.

papyrus=> 2+2
papyrus [out0]: 4

Overview

Papyrus creates a Rust REPL in your terminal. Code can be typed in, line by line with feedback on the evaluation, or code can be injected via stdin handles. Each code snippet is evaluated on an expression based system, so terminating with a semi-colon requires more input.

Example

[lib] papyrus=> 2+2
papyrus [out0]: 4
[lib] papyrus=> println!("Hello, world!");
[lib] papyrus.> out0 * out0
Hello, world!
papyrus [out1]: 16
[lib] papyrus=> :help
help -- prints the help messages
cancel | c -- returns to the root class
exit -- sends the exit signal to end the interactive loop
Classes:
  edit -- Edit previous input
  mod -- Handle modules
Actions:
  mut -- Begin a mutable block of code
[lib] papyrus=> :exit
[lib] papyrus=> Thanks for using papyrus!

Installation

Papyrus can be installed from crates.io or building from source on github. The default installation feature set requires a nightly toolchain, but stable can be used with fewer features enabled.

To install with all features:

rustup toolchain add nightly
cargo +nightly install papyrus

To install on stable without racer completion:

cargo +stable install papyrus --no-default-features --features="format,runnable"

Requirements

Features

Papyrus has features sets:

• format: format code snippets using rustfmt
• racer-completion: enable code completion using racer. Requires a nightly compiler
• runnable: papyrus can be run, without needing to manually handle repl states and output

All features are enabled by default.

Cargo

Papyrus leverages installed binaries of both cargo and rustc. This requirement may lift in the future but for now, any user wanting to use Papyrus will need an installation of Rust.

papyrus's Issues

Implement code coverage on Github Actions

Implment book testing and deployment on Github Actions

Update crossterm to 0.12

This creates compilation errors, so will need some time to work through them.

Linux: out# results hidden after call to `println!()`

eg

papyrus=> 5+6
papyrus [out0]: 11
papyrus=> println!("adf")
papyrus [out1]: ()
papyrus=> 8+9
papyrus [out2]: ()

might be related to #15, the out2 should be 17

Format Kserd using certain widths

Goal

Format the Kserd using an explicit formatter and have width options based on terminal width.

Aspects to Develop

• Set a Kserd formatter for outputting
• Create width option
  • Set width
  • No Width
  • Terminal Width
  • Min width
  • Max Width

Keep loaded libraries in memory

There is a very specific use case which can result in segfaults. The implementer must send through an object such as a channel that can be sent some actionable thing. Once the papyrus code finishes, if this code is called it will be called on invalid memory (as the library has been dropped at this point).

The solution may be to store loaded libraries, and give the ability to clear the cache at user defined points. Storing would not be the preferred option so would have to be opted into.

Todo:

• Rename and delete libs
• Add method to clear out cached libs
• Remove redirection on execution

Erratic typing performance in repl

REPL is performing erratically with typing input.

Input seems to be fine, just the terminal interface is weird.

Develop usage guide and deploy on gh-pages

A usage guide for papyrus that can be done in mdbook and deployed on gh-pages.

Racer completion with `.run()` rebuilds cache

Somehow keep cache outside to increase completion performance.

Linux stderr and stdout not printing

papyrus=> println!("stdout") on linux build, does not direct output

papyrus-pad: make text containers shrink to fit content

currently can't fix until fschutt/azul#41 is rectified.

use commander for repl

Add ability to add external library source code for completion.

Collect crash data in run()

On a crash the run interface writes the stack to the screen, which then gets wiped as part of the unwinding.

Implement a system to dump the stderr to a file on a crash when using the Repl.run() method.

Only accept &mut data

Problem

1. Three linking variants are frustrating to manage.
2. They provide little added value.
3. Mutating on eval stage will stack, same mutation is undertaken on every eval stage.

Scope

• Remove three linking types (NoRef, Brw, BrwMut)
• Always take &mut Data for eval function
• Always take Arc<Mutex<Data>> for eval_async function
• Always supply app_data: &T in for repl use
• Provide mechanism to update app_data mutably (for another issue).

Reasoning

Having the ability to take three different reference types was initially a good idea. It became apparent however that typing the reference method was unnecessary. The borrow lifetime would always be just the eval function, and if using eval_async then relevant reference counters are required anyway.

The stacking mutation were also buggy, it goes against what would be expected of the repl. Only providing a immutable reference will stop this. A mechanism will have to be developed that allows a mutation to occur, and occur only once...

`Repl.run()` uses racer with feature `racer-completion`

Remove the superfluous run_with_completion function.

Complete module documentation

• cmds
• code
• compile
• complete
• fmt
• input
• output
• linking
• repl
• run

extern crate crashes

log:

An unhandled error occurred in the operation of papyrus.
Please send this information to the required parties.

Panic Payload:
assertion failed: index < len

Location:
src/liballoc/vec.rs:990:9

Backtrace:
stack backtrace:
   0: papyrus::run::run::{{closure}}
             at src/run/mod.rs:48
   1: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:477
   2: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:380
   3: rust_begin_unwind
             at src/libstd/panicking.rs:307
   4: core::panicking::panic_fmt
             at src/libcore/panicking.rs:84
   5: core::panicking::panic
             at src/libcore/panicking.rs:49
   6: alloc::vec::Vec<T>::remove
             at <::core::macros::panic macros>:3
   7: papyrus::repl::eval::<impl papyrus::repl::ReplData<D>>::handle_program::{{closure}}
             at src/repl/eval.rs:297
   8: papyrus::repl::eval::<impl papyrus::repl::ReplData<D>>::handle_program
             at src/repl/eval.rs:323
   9: papyrus::repl::eval::map_variants
             at src/repl/eval.rs:131
  10: papyrus::repl::eval::<impl papyrus::repl::Repl<papyrus::repl::Evaluate,D>>::eval
             at src/repl/eval.rs:41
  11: papyrus::run::<impl papyrus::repl::Repl<papyrus::repl::Read,D>>::run::{{closure}}
             at src/run/mod.rs:25
  12: <alloc::boxed::Box<F> as core::ops::function::FnMut<A>>::call_mut
             at /rustc/032a53a06ce293571e51bbe621a5c480e8a28e95/src/liballoc/boxed.rs:917
  13: papyrus::run::do_eval
             at src/run/mod.rs:275
  14: papyrus::run::run
             at src/run/mod.rs:97
  15: papyrus::run::<impl papyrus::repl::Repl<papyrus::repl::Read,D>>::run
             at src/run/mod.rs:25
  16: papyrus::main
             at src/main.rs:25
  17: std::rt::lang_start::{{closure}}
             at /rustc/032a53a06ce293571e51bbe621a5c480e8a28e95/src/libstd/rt.rs:64
  18: std::rt::lang_start_internal::{{closure}}
             at src/libstd/rt.rs:49
      std::panicking::try::do_call
             at src/libstd/panicking.rs:292
  19: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  20: std::panicking::try
             at src/libstd/panicking.rs:271
      std::panic::catch_unwind
             at src/libstd/panic.rs:394
      std::rt::lang_start_internal
             at src/libstd/rt.rs:48
  21: std::rt::lang_start
             at /rustc/032a53a06ce293571e51bbe621a5c480e8a28e95/src/libstd/rt.rs:64
  22: main
  23: __libc_start_main
  24: _start

Derive `ToKserd` internally

It was originally intended to add #[derive(ToKserd)] to a defined struct or enum. Testing has shown this to increase compilation times as it has to expand the macro. Instead, it is proposed to derive the impl internally in code. This has the benefit of being compiled once inside papyrus but comes at the cost of maintainability.

• Research how quote and syn can be used procedurally to generate code.
• Implement inside papyrus.

Considerations:

• This can be developed alongside kserd_derive, expose the relevant functions.
• Initially develop to be run on each source code write, but could be memoized if noticeable performance improvements.

Move to Github Actions

Create mutating mechanism

Related to #22.

Scope

• Add mechanism to mutate app_data

Mechanism

• Add .mut command
  • Args afterwards can be sent straight to eval stage?
• Flag for mutating
  • Automatically close after eval?
• Change fn signature (app_data: &mut T)
• Once evaluated, further eval loops will not include the mutation code

The mutation has occurred with a single evaluation stage, such that it is unnecessary to repeat the same mutating code.
Movement of data into the app_data will also not capture the data as a repeated eval loop will reconstruct that data.
Assigned variables whilst in mutating state will not persist in further eval loops (the code is erased).

Injection of using statements

Goal

Add ability have a piece of code that gets injected on each compilation.

Discussion

The specific use case is to solve where an external library is relying on kserd to display data, but the papyrus version will be trying to solve using its version of kserd.

A fix can be applied by prefixing the source file with use libname::kserd as kserd and this acts as a drop in replacement for the usual crate kserd (which can be accessed using ::kserd). It requires library developers to expose kserd using pub extern crate kserd in their lib.rs file.

This would solve most use cases, however could fall over when the returning type is coming from a dependency in the REPL and implements ToKserd. This would cause the opposite problem (although have not tested) where the ::kserd is required for resolving.

A possible solution is to add a piece of code which can transform libname::kserd to ::kserd.

Aspects to Develop

• Add in edition = "2018" flag to memory code cargo toml
• Do test linking in local crate which also implements ToKserd and show that it breaks
• Test how to add a transformation from libname::kserd to ::kserd.
• Add an optional source prefix which gets added to every source module
• Add an optional source prefix which gets added to only lib source module.

add a `Repl.run_async()` method

Much like run but takes Arc<Mutex<D>>.

passing of structure to calling function

Add #![] acceptance

pass repl around in azul widget

once completed #9 look into keeping repl inside the azul app?

Refactor typing system to fix &mut types

Mutably borrow types will not work with the current type system. This is because an eval() call will return a Repl with a type parameter of Data. If this type parameter is &mut then the reference lifetime will follow through, such that once given some data Data, rust thinks the lifetime is still valid for the Read return value (as it is captures Data as well).

I will have to move back to typing the reference type, NoRef, Brw, and BrwMut.

Examples for repls

Document repl module

Load from file/string

The ability to load items/stmts/crates from a file (or other source).

Thinking that the source can be parsed using current technology and added into the current mod.

remove magic string for type info

Could have mismatch between data type and function signature in external library, try to match this through deriving type name and module path

restructure compile module

• given how data transfer works, won't need main() fn
• ReplData can be comprised of multiple mod files and a lib file
• a module can be used to handle the file and module handling

Remove three data variants

Instead of having NoData, BorrowData, and BorrowMutData, just use a single macro invocation and concrete typing to specify the modifier.

Should be able to reduce to two macros, one with an external crate and one without.

The default case would be of type ().

Failing to build on latest nightly

Nightly version 1.40 fails to compile.

Reason is crossbeam issue and is unretractable until a new version is pushed.

High CPU usage

Possibly related to the upgrade to crossterm 0.14, constant CPU usage is being seen.

writing stderr and stdout to Writer

When the external library is run, any stderr or stdout is not captured, needs to be passed to the terminal Writer

Poor formatting when More input is required

Seems to only occur on unix implementations, windows aligns nicely.

println!() does not carriage return on linux

Running the following. Does not carriage return with the println!(). Behaviour does not replicate on windows.

[lib] papyrus=> for i in 0..10 { println!("{}", i); }
0
 1
  2
   3
    4
     5
      6
       7
        8
         9
papyrus [out0]: ()

Switch to using `:` for command rather than `.`

Make it more like vim

Add mac right click menu

No code completion

Code completion is needed so that my expanded mind can be unleashed on papyrus

Remove papyrus-pad

Coerced app_data can hard crash

Take example below. Crashes, but using &String instead of &str works. Maybe something to do with coercion, and might be a little tricky solve =/.

let v = String::from("Hello, world!");

let mut data = repl_data!(&str);
let mut repl = papyrus::Repl::default_terminal(&mut data);

for ch in "app_data\n".chars() {
	repl = match repl.push_input(ch) {
		papyrus::repl::PushResult::Read(r) => r,
		papyrus::repl::PushResult::Eval(r) => r.eval(&v).unwrap().print(),
	}
}

Move to using dev branch

New run interface

New interface for Repl.run().

To build a more flexible base, mortal::Screen can be employed to build a whole screen repl.

• Initial implement
• Editing move left and right
• Backspace and delete
• Multi line support
• Initial completion (no prompt)
• Switch . to :

Colourise the papyrus-pad

Colourise and line item the "terminal"

Move pad widgets into wigets module

Add linux right click menu

REPL on windows not displaying anything

Since update to crossterm the REPL windows terminal not working on any windows terminals.

Add `read_str` and `read_input` for repl read stage -- just `push_input` now

Add ability to read a string straight into the input buffer, moving to the evaluate stage.