killercup / quicli

Quickly build cool CLI apps in Rust.

QuiCLI

Quickly build cool CLI apps in Rust.

Getting started

Read the Getting Started guide!

Thanks

This is only possible because of all the awesome libraries included here:

• Structopt and Clap for the nice CLI with awesome argument handling, great error messages, and nice composability!
• Serde for handling all things serializing and deserializing
• failure for ergonomic error handling.
• …and more to come!

License

Licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

quicli's Issues

Add serde_json?

We already have serde and serde-derive, it might make sense to be able to read/write JSON as well.

Ideas for guides

• subcommands (show the "MakeCookie"-way of structopt)
• log levels, RUST_LOG
• show when to stop using quicli built-ins and do your own thing

Add output helpers?

What is a typical thing in a CLI app where you need to output stuff? What can we add to quicli that would make outputting information/data to the terminal really convenient?

E.g. https://docs.rs/indicatif and https://docs.rs/prettytable-rs

fs::write_to_file doc test fails on Windows

/tmp/asdasidz81zasda isn't a valid path on Windows.

Use std::env::temp_dir for a cross-platform way to get the temp file directory.

Progress bars/indicators? Keyboard interactions?

How would you go about introducing progress visuals(common to see something like [=====---------]49%) or busy indicators(I've seen a project for this, maybe it was in zsh but should be portable to general CLI output from Rust).

I recall using a CLI app, Hashcat last year that would show several lines of what was going on followed by many lines of status of the computation that was updated and below that were some interactivity options along with shortcut keys that could pause/stop/save/etc the computation.

I guess these would both be a better story as async support develops further in the year, but a neat feature for quicli to be able to offer easily setting up? Item (key, description/label, function to call), view(list of items arranged horizontally I guess?)... or is that going too far into TUI land?

Let me know if you'd like me to split the two features into separate issues.

Add rayon?

Are there any CLI-typical tasks that can be done in parallel? Does exposing rayon's .par_iter make more users take advantage of this feature?

https://docs.rs/rayon

Publish fancy guides/docs

I want to add some more examples and show them in a nice way, but I'm not yet sure how best to do that.

One idea:

• Add docs/ folder that is deploy via Github Pages (with Jekyll)
• put "literate Rust" in there (Markdown with code blocks)
• parse the Markdown files in there with waltz and test that code as subcrate(s) on CI

Meta: Collect relevant use cases from the cookbook

The Rust Cookbook has a bunch of

simple examples that demonstrate good practices to accomplish common programming tasks, using the crates of the Rust ecosystem.

Many of these examples can work as CLI apps. We should go through them and see if there are some common patterns that we want to offer abstractions for.

Looser bounds on main! return type

Using failure::Error there actually requires all errors to be sync. std::error::Error should probably suffice

Switch to globwalk

Instead of glob, let's use https://github.com/Gilnaa/globwalk which is more powerful. This is also a good opportunity to review if this is still a good API to export.

move `main!` macro to its own crate

I mainly want this so I can steal it in and use it in stdcli!

vitiral/stdcli#2

Add user interaction functions?

E.g. using https://docs.rs/dialoguer

I usually only add "interaction" using CLI arguments. I'd be interested to hear what your use cases are.

Todo:

• Choose which features are commonly needed
• Design an API that exposes the commonly used features in a very simple manner
• Document this API with add examples/doc tests if possible
• Add a full-featured "cook book" example (cf. #16)

Verbosity flag does not work when package name != crate name

This was very weird to diagnose, since I couldn't figure out why it worked in playground but not in my real project (that has hyphens in its name) until I guessed that this was the problem.

When setting package.name = "crate-name", the actual crate name will be crate_name. The log filter should be registered for crate_name but is currently registered for crate-name. (It may be a failing of env_logger to not accept either, but we can do better.)

Main return type has not enough constraints?

See #41's CI logs:

Huh, some problems with compiling waltz? Ohhh, or maybe the quicli override in .cargo/context caught that? Which is super weird, and may point to the fact that #32 was not completely done? I'll need to investigate this.

Add regex?

What are typical regex use case in a CLI app?

Should use https://docs.rs/regex (and ideally wrap all regexs in https://docs.rs/lazy_static) but sadly cannot provide std::matches and std::replace as std already defines those.

Add hasmap!/btreemap!/etc. macros?

https://docs.rs/maplit

human-panic

Would it make sense for human-panic to be part of quicli? I feel it'd make a lot of sense for it to be part of almost every CLI application, but I'm not sure if it exceeds quicli's scope.

I think it might be an interesting addition to consider. Keen to hear thoughts on this!

Refs

• https://github.com/yoshuawuyts/human-panic

Configuration loading

In #19 (comment), @moosingin3space suggested adding configure.

Add `stream_in` and `stream_out` to fs.rs

I love the command line because of how easy it is to chain commands together (really the only other thing as easy are Rust iterators ;)).

For example:

head Sherlock_Complete_Works.txt | rg -w '^Sherlock | less

Adding the stream_in and steam_out fs.rs functions with read_file and write_file would reallly compliment this library for CLIs.

Readme example doesn't compile

Hi, I tried to compile the Readme example and I got two errors saying that StructOpt and Verbosity are not found in scope. Should these be added to the prelude?

Requirements Discussion: what is quicli?

This is a discussion on what the use case and requirements of these two libraries are.

stdcli's goal is basically to make creating a CLI in rust more like creating one in python from a user experience and ergonomics point of view. This means:

• batteries included
• almost everything you need is already imported
  • python, because of it's duct typing and general unsafetly, does not require you to import "something" to use methods on its types. Rust, on the other hand requires you to import a boatload of traits.
  • related: #18
• "flush out" macros that should be in the stdlib like hashmap!
• (not yet in stdcli but planned) provide feature flags to control which libraries you want so you can develop with "full batteries" and then pare everything down as your application dependencies become more clear. For instance, maybe ansi_term, pretty_tables, termstyle and tabwriter are all in the "full batteries" package, but then you can do features = ["style_termstyle"] to only have termstyle

Also related: require that all versions in quicli are of the form >=VERSION so that the user can mostly control which version is used.

What is quicli?

Qui CLI seems to have similar but not completely overlapping goals. I would prefer we merge forces rather than create competing libraries.

I think it should be possible through liberal use of feature flags to accomplish any usecase a user might want. I.e. features = "full_batteries" would depend-on-the-world, but features = "light" would give very minimal dependencies, and everything in between.

More conventions around main function?

Currently the main! macro only generates a very simple wrapper so you can use ? and get nice process exits. It might be cool to extend it to cover a bit more ground:

• parse args
• set up logging (cf. #7)
• maybe even catch panics and output them in a nice way as well?

I imagine something like

main!(|args: Cli| {
    if args.count > 4 {
        println!("yay");
    }
});

that expands to

fn main() {
    fn run() -> $crate::prelude::Result<()> {
        let args = Cli::from_args();
        loggerv::init_with_verbosity(args.occurrences_of("v"))?;

        if args.count > 4 {
            println!("yay");
        }

        Ok(())
    }

    match run() {
        Ok(_) => {}
        Err(e) => {
            eprintln!("{}", e);
            ::std::process::exit(1);
        }
    }
}

Add set_log_verbosity function

#51 adds a hidden set_log_verbosity function which is currently an implementation detail of the main! macro. However, I would like to break it out into a general enough function that could be used outside of main!.

This comment proposed the following API

fn set_log_verbosity(lowest: u64, levels: &[(&str, u64)])

• lowest is the lowest log level, i.e. set to 1 to enable all warnings.
• levels is a tuple containing the custom levels for specific crates.

This gives complete flexibility but is also very simple.

Discuss!

Update Rayon dependency to 1.0

• Bump rayon's version number
• Figure out if this is a breaking change to quicli
  • Is there code out there that will no longer compile?

StructOpt not found in scope

Hi,

I wrote a simple toy example today but it seems by StructOpt declaration might be invalid - I have a few errors:

error[E0425]: cannot find value `to` in this scope
  --> src\main.rs:24:17
   |
24 | #[derive(Debug, StructOpt)]
   |                 ^^^^^^^^^
   |                 |
   |                 `self` value is only available in methods with `self` parameter
   |                 help: try: `self.to`

error[E0425]: cannot find value `w` in this scope
  --> src\main.rs:24:17
   |
24 | #[derive(Debug, StructOpt)]
   |                 ^^^^^^^^^ not found in this scope

error[E0425]: cannot find value `from` in this scope
  --> src\main.rs:24:17
   |
24 | #[derive(Debug, StructOpt)]
   |                 ^^^^^^^^^
   |                 |
   |                 `self` value is only available in methods with `self` parameter
   |                 help: try: `self.from`

error[E0425]: cannot find value `r` in this scope
  --> src\main.rs:24:17
   |
24 | #[derive(Debug, StructOpt)]
   |                 ^^^^^^^^^ not found in this scope

error[E0425]: cannot find value `Fahrenheit` in this scope
  --> src\main.rs:24:17
   |
24 | #[derive(Debug, StructOpt)]
   |                 ^^^^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
   |
7  | use TemperatureUnit::Fahrenheit;

Here's the source code - https://gist.github.com/blakehawkins/5f848d3a4eff2f0a67282cc1cb730022

Any advice appreciated! May be a regression?

u64 args doesn't work

If I put this in the Cli struct:

    #[structopt(long = "timeout", short = "t", default_value = "3")]
    timeout: u64,

then args.timeout's value is 0 by default, and 1 if I pass a --timeout 5 argument.
If I replace u64 with u32 it works as expected.

Need to have structopt/serde as peer dependency

The structopt derive macro doesn't work unless the current project allows extern crate structopt;.

This means, user of quicli must cargo add structopt, which is not that cool.

This is part of this code which expands to something like:

const _IMPL_STRUCTOPT_FOR_Cli: () = {
    extern crate structopt as _structopt;
    use structopt::StructOpt;
    impl _structopt::StructOpt for Cli {

Maybe we can add a feature to either suppress or customize this upstream?

---

Same is also true for serde! (Thanks to this code)

No context in error output?

It would appear that context strings don't appear in the error output when using failure::Error. Noticed this long after I replaced some custom code with quicli. I'm not 100% sure this is why the context error isn't showing, but I'm sure enough that I don't want this to get lost.

Update failure to 1.0.0 on March 15

cf. https://boats.gitlab.io/blog/post/2018-02-22-failure-1.0/

• Try out the git version/release candidates
• Figure out all the breakage
• Figure out all the improved ergonomics we should point out
• Maybe land #66/parts of #67 together with this upgrade

Set log verbosity of libraries

When setting verbosity via StructOpt, logging from the binary is set to the listed verbosity, but all dependencies are always Warn only.

quicli/src/main_macro.rs

Lines 46 to 47 in 3a2f20e

	.filter(Some(env!("CARGO_PKG_NAME")), log_level)
	.filter(None, $crate::prelude::LogLevel::Warn.to_level_filter())

In my case I have a binary which is a different crate from the library it's a front to, because I'm already in a large workspace and I want to separate out the binary's dependencies from the library's, which is used elsewhere as well.

I would like to be able to allow the verbosity setting to effect certain dependencies as well. Side note, when at verbosity=0, the dependencies are louder than the first-party code 😄

Quicli in the wild

Who's using quicli?

• waltz
• wasm-pack

Add sub process helpers?

What is a typical thing in a CLI app where you need to spawn process? What can we add to quicli that would make dealing with subprocess really convenient that goes beyond just re-exporting https://docs.rs/duct?

export clap macros

clap exports very useful macros such as crate_version etc. These should be exported if possible.

Also, I think it is okay to export the other clap types, since sometimes you want to access App directly even if you are using structopt.

Add itertools?

Do typical Rust CLI apps use iterators a lot? Do they benefit from the convenience functions that itertools provides?

Will new Rust users discover, understand, and use these methods if we re-export the Itertools trait?

Add logging?

https://docs.rs/log for log macros and https://docs.rs/loggerv for output.

Testing, examples, docs, CI

While we have compile-tests for the guides and examples, we currently don't really have a good testing story. We have doc tests for the functions we provide, but nothing in regard of integration tests. We should change that.

My idea is this to add full crates to examples/ instead of in-crate binaries.

Each example:

• has a Cargo.toml, src/main.rs
• depends on the current quicli version as well as other necessary crates
• has a tests/integration.rs that uses a crate like assert_cli to actually test that the correct things happen

I propose we do it like this:

• Convert existing example(s) to this structure (#64)
• Write integration tests for these examples (#65)
• Create a Cargo workspace and add examples to prevent compiling libs over and over again (#64)
• Convert waltz call in doc/_test.sh to write to examples/<guide-name> (#64)
• Link to full code in guides
• Add link checker to CI (e.g. using this)
• Fail CI when running waltz creates a diff (#64)
• Write integration tests for code in the guides

Integrate with the ergo ecosystem

Hey @killercup, you've probably been waiting for this feature request 😄

I am nearing completion of the initial release of ergo. There is a LOT of overlap between these two crates but there is also several areas that there is not overlap.

My suggestion is to have a minimal feature set which only

• exports the main! macro
• exports log and envlog crates and types
• exports structopt macro and types
• exports failure macros and types

This would allow quicli to fit cleanly within the ergo crate ecosystem, as I have no desire for ergo to perform these operations. Theoretically I could create an ergo-cli crate which is pretty much a fork of quicli with these features -- but I don't want to do that!

Another advantage of this is that it could be part of the "story" of developing an application:

• when you start out, use just quicli
• when you need a larger stdlib, set features=minimal and import ergo
• keep rusting!

Unix pipes support out of the box

It will be great if I could just set some flag and callback to get an ability to read from stdin and write to stdout.

Move Result type out of the prelude

The Result type exposed by quicli::prelude conflicts with the Result type that's in the default Rust namespace, since it has a different type signature. This makes it inconvenient to drop quicli into existing projects. It'd be great to move this out of the prelude and make it an optional import instead.

For example, here's a PR where I'm switching from bare clap to quicli. I ended up tossing a use std::result::Result right after the quicli prelude import, which is the kind of thing which makes me think this should work the other way around.

Can't use `debug!()` in mod

Hi,

I've created an executable with 2 files, used #[macro_use] extern crate quicli; and use quicli::prelude::*; in the first one but now I can't use debug!() in the second one:

error: cannot find macro `debug!` in this scope
  --> src/endmi.rs:28:9
   |
28 |         debug!("start decoding");
   |         ^^^^^

error: aborting due to previous error

I've tried adding use quicli::prelude::*; in the second file, but now some types are polluted (e.g. Result takes only 1 arg now, I guess io::Result has shadowed result::Result).

Is that intended behavior ? Or do I have to do something to have log macros available short of reimporting the log crate ?

Make sure backtraces from `error?` in main! are cool

• Make sure backtraces for errors can show up
  • We may need to use libbacktrace ourselves or go back to using failure in main! which has backtraces
• Document how to make backtraces show up
• Make backtraces pretty and have useful line numbers (ideally main.rs:42 as the first thing)

Add random?

Do typical CLI apps use randomness?

https://docs.rs/random

rexport std_prelude in prelude

https://github.com/vitiral/std_prelude

I personally feel like this makes writing quick things in rust much more ergonomic.

I'm openeing a separate issue for a discussion on the overal design goals of this lib

"pub extern crate" for all dependencies

Summary

This is to discuss whether all dependencies sould use pub extern crate. This is the current strategy for ergo and regardless of the decision for quicli I would like to understand the arguments from the various angles.

Argument FOR

quicli is a "wrapper/conglomeration" crate. It's goal is to pull parts of the ecosystem together into a unified API and make it simple to use them. It's secondary goal is to "get out of the way" when users want more complexity, and even provide paths for not using quicli at all.

A usecase might be needing to use clap directly. The current (suggested) path is that you have to add clap to your Cargo.toml. However, clap is directly exported (as part of the public api) of structopt, so all that would be really necessary is use quicli::structopt::clap.

Edit: Another advantage is that pub extern crate makes the external crate docs more accessible in the autogenerated docs

Argument AGAINST

I'll just copy @killercup's comment from here

Not sure this is a good idea. If you are honestly using clap in your crate for anything, you should probably add it to your Cargo.toml and use the same version as quicli. That's more obvious to any contributors including your future self. I'd be interested in what other think about this.

I'll just add that publically exporting the crate means that API breakages in external crates could theoretically mean that your API changed.

Unresolved Questions

I disagree about the API change from external crates -- I think if the user doesn't want the wrapped crate to be upgraded they should have to pin it in some way. However, I currently don't know how to do this so documenting how to pin dependencies is an important unknown.

#[derive(Fail) fails

I receive an error about missing crate "failure" - looks like it's probably the same situation as serde. Perhaps failure can be reexported inside a namespace with the same name to preserve the macro functionality? Or it should be added to serde and structopt in the list of things you should add to Cargo.toml to use everything.

docs and crates to support re-parsing the command line (readline style)

Using the below crates it's possible to re-parse StructOpt/clap::App objects:

• rustyline
• shlex

Rough example (copy/pasted from my novault application):

    let mut rl = rustyline::Editor::<()>::new();
    loop {
        eprintln!("\nType \"help\" for help and \"exit\" to exit.");
        let readline = match rl.readline(">> ") {
            Ok(l) => l,
            Err(e) => {
                eprintln!("Got {}", e);
                exit(0);
            }
        };
        let line = readline.trim();
        if line.to_ascii_uppercase() == "EXIT" {
            exit(0);
        }
        let mut args = match shlex::split(&readline) {
            Some(a) => a,
            None => {
                eprintln!("Invalid shell syntax");
                continue;                                                                                                                                                                                         
            }                                                                                                                                                                                                     
        };                                                                                                                                                                                                        
        args.insert(0, "novault".into());                                                                                                                                                                         
                                                                                                                                                                                                                  
        let matches = match super::LoopOpt::clap().get_matches_from_safe(args) {                                                                                                                                  
            Ok(m) => m,                                                                                                                                                                                           
            Err(err) => {                                                                                                                                                                                         
                eprintln!("{}", err);                                                                                                                                                                             
                continue;                                                                                                                                                                                         
            }                                                                                                                                                                                                     
        };

        if let Err(err) = super::run_cmd_single(global, LoopOpt::from_clap(matches).cmd) {                                                                                                                        
            eprintln!("{}", err);                                                                                                                                                                                 
        }

Add glob?

Do typical Rust CLI apps use glob-style queries to access files and directories?

http://docs.rs/glob

add and export path_abs crate

I am the creator and maintainer of the new path_abs crate and I think it solves a lot of ergonomic problems in rust (problems I see you re-solving with the fs module!).

Questions:

• should we export path_abs types?
• should fs module be deprecated?

Thanks!

upgrade to structopt v0.2

This release could not have come at a better time.

Seriously, that was the most needed feature of all time for that library. Ship it!

"methods are not allowed for flattened entry"

I get this message from the #[derive(StructOpt)] with structopt v0.2.10 and quicli v0.3.0 due to #[structopt(flatten)] on the Verbosity member. Thanks.