mkohlmyr / scrum

GNU Screen Session Management

scrum

What is GNU Screen?

Screen is a essentially a terminal multiplexer. You can run multiple processes and access them all through one terminal. You can switch between processes easily and even detach entirely until you actually need to interact with the process. GNU Screen is a hard dependency of scrum and can (must) be installed from the standard repositories of most linux distributions.

What is scrum?

Scrum is a CLI proxy for GNU Screen which enables the use of JSON files (a "scrumfile") to configure groups of services or commands to run together in a single GNU Screen session.

When does scrum come in handy?

• You are developing a microservices architecture and need to run many different services together at the same time.
• You are developing a web application and want to run your web server, task runners and a database shell in the background but with easy access.

Scrumfile

Scrumfiles can be provided in three ways and will be used in the following order of preference by the CLI.

1. A .json file provided with the flag --scrumfile ../../myscrumfile.json.
2. A file named scrumfile.json under the current directory in which scrum is being invoked.
3. A file named scrumfile.json under the ~/ directory. Scrumfiles adhere to a very simple structure, making it easy to add new sessions and windows when needed. The following is an example of a scrumfile that defines a single session with two windows (processes).

{
    "session_1": {
        "Window A": "cd somedir; node somefile.js -n {{opt.n}}",
        "Window B": "cd otherdir; python somefile.python -w {{opt.w}} --user {{env.user}}"
    }
}

Available commands

• scrum start <session>
• scrum open <session>
• scrum stop <session>
• scrum version
• scrum version latest
• scrum sessions

Dynamic flags

When a group is started by scrum, all the commands to be executed for it will be compiled and rendered as handlebars templates with a context that contains two variables opt and env.

If you run scrum start my-services -f true {{opt.f}} will be replaced by true and can be used in the commands that will execute. Under env you have access to the username (env.user), home directory (env.home), working directory (env.pwd), terminal name (env.term), shell name (env.shell), platform (env.platform) and process id (env.pid) as provided by the Node.js process API. To correctly start the session we defined in the above example scrumfile, we would run scrum start session_1 -n somevalue -w othervalue.

Links

• NPM scrum
• GNU Screen Manual
• Gnu Screen Quick Reference

scrum's Issues

restartable windows

if scrumfile -> config file and windows are labeled, they could be individually startable by name, useful when e.g. personServer dies.

error handling & messages

• handle errors (e.g. non-existing session name, no scrumfile) gracefully and output useful error messages

scrum version

• easily check package.json version with scrum version
• check npm for latest version available with scrum version latest
• easy npm update with scrum version upgrade

remove babel dependency, target specific node version

By not using imports/exports we can get 99% of ES6 without babel by explicitly targeting a recent node engine in package.json.

Will modularise better and clean up project structure as part of this effort.

scrum restart

If all windows are ran with ; exec bash (note, may not be sufficient) (preferably in the directory where the session was started from via scrum) appended to the end of execution command, we could have a command scrum restart which would restart the command that previously ran in that window.

support multiple shells

bash -c appears to be matched by csh, zsh and dash (alias sh on ubuntu).

The desired result could be reached by allowing scrumfiles to define a shell name (as they did previously) and making use of that once again for start.coffee L19 and L31. This would allow setting the used shell per scrumfile, where one scrumfile can define multiple sessions. It might however also be nice to be able to set this per execution (argument --shell) as well as using the global default as already picked up in env.shell by args.coffee

scrum sessions & scrum status

• get a list of running sessions with scrum sessions e.g. 29733.node-services
• get comprehensive list of sessions and related windows with scrum status

Markdown Scrum & Screen Manuals

It would be handy to accompany the readme with two actual manual files.

• scrum.md
• screen.md

Showing basic GNU Screen usage and all the possible Scrum syntaxes with a good canonical usage example.

Next Steps

Feature list

Current

• scrum start session
• scrum start session window
• scrum open session
• scrum kill session

Desired

• A scrum daemon. Keep track of all scrum sessions that have been started. Keep track of their windows. Protocol? UNIX socket?
• scrum open session window
• scrum kill session window
• scrum status
• scrum status session
• scrum status session window

Misc todo

• Do not open extra bash window (or rather close it once other windows are open)
• Rename kill to end or stop? (note to self, kill should alias stop, don't break the API)
• Shorthand window names? Mapped to full name, e.g. scrum start node-services am. This allows the full name to be the title, but for us to know it by another name when executing commands. Essentially turning the window title into more of a descriptive header.
• Building on shorthand: include pid in title.
• Record and keep track of window number for each started window so that it can be used for commands in place of window name? This works hand in hand with shorthand.

Moonshot

• Would it be possible to do individual ssh's in different windows (similar to cwd) and thus end up with a serssion showing terminals from not just separate processes but separate servers? That would be very neat.
• A way to chroot sessions and / or windows
• Some kind of integration (or instructions for working) with a deployment system / strategy

dynamic window parameters

this problem can be solved both for scrumfile/scrumrc and scrumfile.json approach by setting and using $SCRUM_ARGUMENT_NAME / $SCRUM_ARGS_ARGUMENT_NAME environment variable in shell.

e.g.

• node executes scrum start services -w asdqwe
• sets $SCRUM_W = asdqwe
• screen executes -c in given shell with "-w $SCRUM_W" in -c
• service starts with -w asdqwe

this is an established pattern given use of $USER in -c string already.

readme & examples

• scrumfile examples in a subdirectory
• README.md in said subdirectory describing and linking to each scrumfile
• simple usage example in (root) README.md

prune use cases

Let's eliminate anything not part of core functionality for now until the rewrite is done.

What is core functionality?

• Start, open and stop scrum sessions.
• See active scrum sessions.
• Keep scrum up to date.

scrum init

• create a scaffolding scrumfile.json in current directory when scrum init is called

ES6 Process

Human-friendly cli arguments parsing proposal. The Args.get method should probably be re-written to return something more manageable. Either a POJSO with path and opts or simply turn it into an instantiable class and just use the constructor instead of a static get. Most likely the latter.

"use strict";
const minimist = require("minimist");
//const compiled = minimist(process.argv.slice(2));
const compiled = {_: ["start", "activity-monitor", "in", "node-services"], scrumfile: "/home/mikaelk/scrumfile.json"};

class Route {
    constructor(template, actionfn) {
        this.template = template;
        this.actionfn = actionfn;
        this.cache = undefined;
    }

    accepts(path) {
        this.cache = path.match(this.template);
        return this.cache ? true : false;
    }

    follow() {
        return this.actionfn.apply(undefined, Array.from(this.cache).slice(1));
    }
}

class Map {
    constructor() {
        this.store = [];
        this.cache = undefined;
    }

    route(template, actionfn) {
        this.store.push(new Route(template, actionfn));
    }

    accepts(path) {
        let self = this;
        this.cache = undefined;
        return this.store.some(function (route) {
            return route.accepts(path) && (self.cache = route);
        });
    }
}

class Args {
    static get() {
        let opts = Object.assign(compiled);
        let path = compiled._.join(" ");
        return delete opts._ && [path, opts];
    }
}

let map = new Map();

map.route(/^hello$/, function () {
    console.log('Hello!');
});

map.route(/^start ([a-z-]+) in ([a-z-]+)$/, function (window, session) {
    console.log(`Starting ${window} in ${session}`);
});

map.route(/^goodbye$/, function () {
    console.log('Goodbye!');
});

let args = Args.get();
if (map.accepts(args[0])) {
    map.cache.follow(args[1]);
}

Scrum Daemon Design Document

• The scrum daemon is a net server listening on the /tmp/scrumd.sock unix socket.
• The scrum daemon maintains a map s.t.

{
    [ppid]: {
        session: [session slug],
        children: {
            [pid]: {
                window: [window slug]
                command: [ps-aux command column]
            }
        }
    }
}

• When the scrum command is ran we attempt to connect to the unix socket, and if it fails we spawn the daemon in the background.
• When the scrum daemon exits or hits an exception, it should remove the unix socket file.
• The scrum daemon is queried by scrum status, scrum status session and scrum status session window and sent data by scrum start. Each of these will trigger the daemon to look up process information for all processes under the available ppids, and update the process map.
• scrum status and related commands should all receive the full map back from the daemon (brings us down to a minimal daemon). They can then scale that data down to what was actually asked for.
• Data returned from daemon to scrum status and related should print as what is essentially a process tree, not unlike ps -ejH. The scrum command will extract and print what was asked for.

scrum help

• print a non-markdown readme file to screen when scrum help is called

scrum do for non-screen execution

• add scrumfile property actions which defines bash commands that do not run in screen (inherit io) e.g.

{
  "sessions": {},
  "actions": {
    "deploy": "cd ...; ./deploy.sh"
}

refactor / fix scrumfile handling

• scrumfile should not be required for all commands, check for it when needed

e.g. scrum stop node-services should not need to be called from scrumfile directory

scrum heal

• scrum heal {{session slug}} looks at services that should run for a given session and re-starts any that appear to have stopped in the target session

command for stopping all or multiple sessions

Run screen -ls, execute execute screen-S {{session slug}} -X quit for each session matched.

An option would be scrum stop all stops all sessions whereas scrum stop all node-services stops all node-services sessions if multiple have been started (by accident or on purpose).

Currently scrum stop node-services when there are multiple will have no effect. At the very least we should forward the screen error message for this case.

• scrum stop all
• scrum stop all {{session slug}}
• scrum stop {{session slug]} must print an error message when it fails due to multiple sessions

error message for unmatched command

Change required in cli.js in CommandLineInterface.main:

    static main(commands) {
        const cli = new CommandLineInterface(commands);

        if (cli.accepts(cli.argv.cmd)) {
            cli.follow();
        } return cli;
    }

Need to add an else clause because entering a non-matched command name will simply exit without any message.

else {
    # print error message and some helpful information about available commands
}