xaf / omni Goto Github PK

the omnipotent tool

Rust 92.75% JavaScript 0.64% TypeScript 0.42% CSS 0.34% Shell 5.84%

dev developer-tools git rust rust-lang tools

omni's Introduction

  :?:.!YG#&@@@&#GJ~.
 7#@&#@@@&#BGB#&@@@#Y^
 ^&@@@@&?.     :~Y#@@@Y
.G@@&#@@&5^       .J@@@G.   OMNI
P@@@! 7B@@@P~       7@@@5
@@@P    !B@@@G~      G@@&     THE OMNIPOTENT
@@@P    !B@@@B!      G@@&        DEV TOOL
P@@@~ 7B@@@P~       !@@@5
.G@@&B@@&P~       .?@@@G.
 ^#@@@@&?.     .~J#@@@Y.
 7&@&#@@@&BGGGB&@@@#5^
  :?^.!YG#@@@@@#GY!.

omni - omnipotent tool

Get started

sh -c "$(curl -fsLS get.omnicli.dev)" -- help

Documentation available on omnicli.dev.

omni's People

Contributors

Stargazers

Watchers

Forkers

idletea tminusplus krousey

omni's Issues

Feature: add `--search-path` option or equivalent to `omni tidy`

This allows to automatically include other paths to be reorganized into the structure managed by omni.

Right now this is doable hackingly with OMNI_ORG=git@fake=/old/path omni tidy but would be neat to be directly supported as a (repeatable) parameter of the command.

Add command to switch between package and worktree modes

With the new packages features, new clone suggestions will be handled as packages by default.
This is useful to have a stable version of the repository to be used for providing commands, while the development version stays untouched.

However, sometimes you may want to work on a command or something in the repository that requires switching from a package to the live development version of the repository. We should provide a command allowing to do so. This should work as the following:

# From anywhere, running this will switch repository to <switchto>
# being one of package or worktree
omni config path switch --<switchto> <repository>

# From anywhere, running this will toggle repository to package if
# currently worktree, or worktree if currently package
omni config path switch <repository>

# From a repository, affects that specific repository
omni config path switch [--<switchto>]

This will:

If the repository does not exist as the switch-to target, clone that repository in that target location (should we also run omni up ?)
update the configuration(s) mentioning the repository in the path to use the new location
print a message indicating that <repo> was switched to <switchto> mode or something like that

This will make using packages an actual easy thing!

Feature: `omni tidy --up-all` or equivalent to run the `omni up` command for every repository

Thoughts:

Filter the repositories ahead, instead of blindly calling omni up in all the repositories (would work, but lots of clutter)
probably want to support the extra --update-repository and --update-user-config flags too... how?

`omni status` should not show everything and provide options

Right now omni status is a bit cramped as showing the whole configuration alongside other information.
Default return should just be the general information, and options (such as --config) should be provided to see details of other areas.

Documentation does not specify that `$0` will refer to the first encountered symlink

When using symlinks for path commands, omni calls out directly the first encountered file and stores others as aliases to the command. This means that $0 (in bash, and equivalent in other languages) will always refer to the first encountered file and not to the command being called, even if it's an alias.

We could:

Make it work so that when calling a command through a symlink-alias, the symlink file is called (making $0 accurate)
Document that $0 is going to be stable when using symlink, and recommend using the OMNI_SUBCOMMAND environment variable

Idea: support for dynamic environment handling `.xxx-version` files

This would also happen on omni up.
This could be using a special up command which:

scrapes the git directory for .xxx-version files (e.g. .ruby-version, .python-version, .tool-versions, etc.)
trigger install the relevant versions, bound to specific subdirectories

This would require to add support for directory-specific environments in the cache.

Feature:`omni cd --locate` to only print a path

This should be returning the string with the path vs. Switching directory directly, so omni cd can be used easily for scripts and automations.

Bug: fish integration seems to limit the number of layers of autocompletion

Example:

Create the following directory structure:
```
cmd/
   a/
      b/
         c/
            d/
               e/
                  f.sh
```
i.e. mkdir -p cmd/a/b/c/d/e && touch cmd/a/b/c/d/e/f.sh
Run chmod +x cmd/a/b/c/d/e/f.sh
Add the directory to your omnipath (ideally do this in a git repository): set -Ux OMNIPATH $PWD/cmd
Try omni autocompletion: omni a <tab>

This does autocomplete the b but does not seem to pursue after.
This seems similar with any other commands.

Omni does return the right autocompletion value (COMP_CWORD=2 omni --complete a returns b, COMP_CWORD=3 omni --complete a b returns c, etc.).

This works as expected on bash and zsh (i.e. autocomplete until omni a b c d e f), fish is the only one having a weird behavior.

Feature: add `terraform` support for `up`

Now that we have support for many-versions environments, we should add support for terraform to be installed with omni up. This will be able to fetch all needed versions, install them, and switch to the right version in the right place when moving around directories.

Move to use `musl` builds for the linux binaries

The main reason for building the new deployment pipeline was to allow using musl to build the linux binaries. We will want to double-check one of the musl-built binaries before flipping the switch though.

up tool version: handle versions using semver parsing

Semver parsing, just as NodeJS uses has a number of values possible: https://stackoverflow.com/a/25861938
It also supports || to allow different ranges.
omni should support specifying versions that way, which makes it more flexible. This should also support versions specified that way in files. The prefix approach, as used today, should be an alternative to the matching.

Feature: `omni hook` subcommands should have proper help with parameters

At the moment the commands are hidden, but with omni hook init [--alias xx] [--command-alias xx xx] added, it would be better to be able to show help for that. This might require handling #20 first.

Suggestions could support conditionals or templating

Allowing to support an if for suggest_clone and suggest_config would allow to build-in better logic.
Alternatively and maybe even better: could the suggest_config and suggest_clone simply reference a template file to be loaded, which supports environment variables?

omni should check trust of repositories before running commands

At the moment, we just run commands as they're being called. For security reasons, omni should check trust and, if needed, ask the user before running commands.

Bug: `omni up --update-repository` does not reload the up configuration

The whole interest of --update-repository is to make sure the repository is up to date before running omni up, but currently this still uses the old up configuration after --update-repository as it is not reloaded. This needs fixing!

bug: omni self updater should replace running process after update

At the moment, omni will update itself but still run the current comment with the older, currently running version.
Omni should call exec to replace the current process once the self update has happened.

Support `.metadata` files next to binaries in omni path

This will allow to avoid having to create a wrapper to call the binary; if a metadata file exists, it should be used to read the file metadata.

Bug: Running a command with `-` in the name presents an error message that it was not found

If you define a command with - in it, like ko-build, it will let you run the command as omni ko-build but presents an error that the command was not found before doing so.

I was expecting that since the command is defined as ko-build it would alias over to ko build.

Omni config: https://github.com/tminusplus/caddy-tailscale/blob/ts/.omni.yaml
Example:

~/git/tminusplus/caddy-tailscale ts
❯ omni help
omni - omnipotent tool (v0.0.12)

Usage: omni <command> [options] ARG...

General
  help               Show help for omni commands
  hook               Call one of omni's hooks for the shell.
  status             Show the status of omni

Git commands
  cd                 Change directory to the git directory of the specified repository
  clone              Clone the specified repository
  scope              Runs an omni command in the context of the specified repository
  tidy               Organize your git repositories using the configured format
  up, down           Sets up or tear down a repository depending on its up configuration

Build < Configuration < .omni.yaml
  ko build           Build a Caddy container image with the tailscale module.

~/git/tminusplus/caddy-tailscale ts
❯ omni ko build
Error: error creating publisher: KO_DOCKER_REPO environment variable is unset

~/git/tminusplus/caddy-tailscale ts
❯ omni ko-build
omni: command not found: ko-build
omni: found: ko build
Error: error creating publisher: KO_DOCKER_REPO environment variable is unset

Feature: For Golang, use the version specified in go.mod

For Golang, support using the version specified in go.mod. Similar to https://github.com/actions/setup-go#getting-go-version-from-the-gomod-file

Auto-updates fail to `git pull` when a password needs to be input

When omni auto-updates repositories, the git pull fails if the user has to input a password.
This should instead hang and ask the user for the password.

`suggest_clone` should clone all other repos first, then do all the `omni up` steps

There are two main uses for suggest_clone:

cloning another repository that can be useful to provide other commands the current repository depend on
cloning another repository that we depend on to run some operations, maybe including during omni up

For 1 we don't really care about when the clone is happening vs. the omni up.
For 2, we do want the clone to happen before omni up.

We also already handle the cloning of all before calling all omni up for all the suggested repositories... so might be more fitting to handle all in the same way (user just cloned current repo, other repos are suggested to clone then cloned, then we run omni up for all cloned repos, suggested or current, maybe even handling a dependency tree for that?)

Add the `clone/auto_up` configuration option

This option was available in the ruby version and is not available in the rust version.
This needs to be ported.

Fish shell integration uses an absolute path for $HOME

Noticed that the Fish shell integration uses an absolute path, instead of a variable, to refer to the home directory:

❯ cat ~/.config/fish/conf.d/omni.fish
test -f "/Users/tszucs/git/XaF/omni/shell_integration/omni.fish"; and source "/Users/tszucs/git/XaF/omni/shell_integration/omni.fish"

This means that syncing my fish config to another computer (ex. Mac -> Linux) will break the fish config, if the home directory or user is different. I did not check the other shell integrations.

Arguments and options shouldn't be split, and just ordered as provided

At the moment, we have a list of arguments and a list of options for commands.
This should be a single list with a parameter (required or not) that would allow to consider the order in which those parameters have been provided by the command.

Bug: if duplicate `tool:version` after `omni up`, we should dedup

Currently:

The cache stores the duplicate
The dynamic environment duplicates the variables

Feature: `suggest_config` and `suggest_clone` fingerprints so that next `omni up` will tell if any change since bootstrap

omni up --bootstrap offers to handle suggest_config and suggest_clone.
Having a fingerprint of their contents would allow to easily identify when something is added that the user didn't see yet, and suggest the user to run omni up --bootstrap when calling omni up if the fingerprint was updated.

Feature: `omni tidy` should clean the cache

If a repository is simply deleted without calling omni down, its cached information are kept.
If omni up is ran in a random repository, leading to the use of .omni/id, its cached information are kept after deletion too.

We want to be able to clean the cache, and of course remove all the installed tools that are not necessary anymore.
Since calling omni tidy already lists all the repositories, we would need to:

add support for handling .omni/id directories when using omni tidy
store a last up information for all workdir ids
make the inventory of all repository ids seen when using omni tidy
compare that inventory with the cached repository ids
automatically remove from the cache all ids for which the last up information is older than N (could default to something between 30 and 90 days, and provide a configuration parameter)

This would be a safe approach since not too invasive thanks to the grace period before deletion.

Feature: binary update when not installed through a package manager

Either:

provide a command to run to update the binary
do it as omni ruby, where we also check for omni version when doing updates, and update it too
check for new versions during the omnipath updates, but still requires the user to call a command (we're replacing a binary after all)

Optimisation: use lazy static instead of Env struct

We don't need all the environment information every time, so building the whole environment structure is not necessary. We can take advantage of the lazy features to only build the environment variables we need, when we need them.

Already did that for OMNI_GIT and HOME, can pursue doing it for all other environment variables that require some logic.

The following values are currently provided through the Env struct and should be moved on their own:

Add test of builds for FreeBSD and OpenBSD

Github Actions can support FreeBSD and OpenBSD testing using VMs:

Add the `apt` operation for `omni up`

This was available in the ruby version and is not available in the rust version.
This needs to be ported.

Docs: Add a documentation website

Given omni's intricacies, we need a documentation that covers:

Omni itself (installation, configuration, baseline/builtin commands)
How to write commands for omni

Enhancement: split the cache into multiple files

Omni's cache is currently a single file targetted by the cache/path key in the configuration (which defaults to the XDG cache directory + /omni). This means that as we add contents to be cached, this can have an impact to all cache operations, whether speed-dependent or not. For instance, the dynamic environment requires read-only access to the up environment cache in order to load the workdir environment, and this is a time-sensitive operation.

We should thus:

split the cache per main key (up environment would be its own cache, same as asdf operation and homebrew operation, etc.)
support separate cache handling for reading and writing
automatically detect if still using the single file, and convert it to a directory with the split files

Panic in command loader: thread 'main' panicked at 'range end index 2 out of range for slice of length 1', src/internal/commands/loader.rs:338:25

Repro:

Temporal-swgillespie :: ~/temporal » omni devcell
omni: command not found: devcell
✔ omni: Did you mean? · devcell create
thread 'main' panicked at 'range end index 2 out of range for slice of length 1', src/internal/commands/loader.rs:338:25
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

feature: "stable clone" of repositories for the omnipath

If working regularly on a repository, the omnipath updates will often fail to update the repository as it's not checked out on the expected branch. Most of the time this is desirable, as it allows people to dynamically work on building a command. But in other cases, this forces to manually update the repository to have the latest commands from it.

Wondering what could be a good interface:

special flag at clone time to clone a repo "in path mode", in a separate directory, which would make that repo not a "work repo" but a "path repo" that the user does not touch ?
command to create a "path mode" repository from the current repository ?
command to create a "path mode" worktree from the current repository ? (would be an issue when updating repositories by tag though)

Add support for `justfiles` similarly to `Makefile` support

See: https://github.com/casey/just

Since this is also a rust crate, should that be directly imported as a lib and ran from omni, or should we depend on just being available in the system similarly to the make expectation? The main difference being that most system will have make available right away, while just is not shipped with most systems.

Feature: support shell aliases and autocompletion with aliases

For instance, would be nice to provide a way to set ocd as an alias to omni cd.
Maybe that could be added as part of the init hook: omni hook init --alias "ocd" "cd" for instance would alias ocd to omni cd and handle the autocompletion while considering the full command.

Path commands maybe shouldn't strip extensions

Specifically talking about this

It's possible that 2 or more commands will end up with identical names due to having different extensions (or "extensions" which are really just incidentally a dot being in the name). The case that made me scratch my head and go looking for what happened was this

$ omni
omni - omnipotent tool (v0.0.13-24-g60a2ab6-dirty)
...
Uncategorized
  python, python3, python3

In this case the real files picked up as commands were python, python3, and python3.11 which are all in a standard python virtual environment.

Possible ways to address this:

Don't worry about it, it's kind of a niche thing
Don't truncate the extension, people can rename the file and drop the extension themselves if they like
Only truncate some common shell/utility extensions (sh, bash, zsh, py, js, etc)
If any duplicate path commands are found remove both from the list and emit a warning

Add `--help` / `-h` support for builtin commands

omni cd --help should act similar to omni help cd so that people's habits to get command help would not be met with errors.

Bug: 'python latest' tries to install `stackless-3.7.5` on the unreleased version

[1/5] ✖ python (latest): error: execution error: ASDF_DATA_DIR="/Users/xaf/.local/share/omni/asdf" ASDF_DIR="/Users/xaf/.local/share/omni/asdf" "/Users/xaf/.local/share/omni/asdf/bin/asdf" "install" "python" "stackless-3.7.5"
omni: up command failed: issue while setting repo up: execution error: ASDF_DATA_DIR="/Users/xaf/.local/share/omni/asdf" ASDF_DIR="/Users/xaf/.local/share/omni/asdf" "/Users/xaf/.local/share/omni/asdf/bin/asdf" "install" "python" "stackless-3.7.5"

Seems that some of the new logic for handling asdf tools versions is picking up stackless-3.7.5 as a valid version number for latest. This is normal as the new logic validates all values when using latest, whereas we would want to at least check that the value only contains numbers and dots, starting with a number.

Bug: parser for `commands` panics when some parameters are missing

When run is missing for a command, the parser will panic. This shouldn't happen, the command should simply be ignored instead. We should be able to verify that it works for other parameters too, or identify and fix all the panics when loading the configuration.

Feature: cache some command answers to avoid having to repeat them

Commands such as asdf update, or listing of a specific plugin available versions should be commands that should be cacheable. It would allow omni up to go faster when doing it on multiple repositories using common dependencies for instance.

Something like bkt's retrieve_streaming could allow us to keep the current UI while having bkt handling the caching for us. We could put the stored information in the temp directory of the OS, to be emptied upon reboot.

Feature: fold commands in `omni help`

omni help can become very long if many commands are available.
When commands have a common prefix, we are able to fold them.

We would want:

for any common prefix (full word), if >1 command, fold into single command and add ↘ after in the help to indicate there are more commands under this one
in the help of that specific prefix, show all the underlying commands, folding if necessary
Add configurability to fold:
- Disable entirely folding
- Minimum number of commands with common prefix before folding
- Minimum number of commands in the help before folding
Add a --long option to omni help to show the full help message without folding upon request

Add support for configuration for the init hook

The init hook now takes --alias and --command-alias parameters.
It would be useful to provide support to set that configuration through a configuration file too. Something like init.yaml, shell.yaml or even aliases.yaml could work. Maybe it could simply be part of the default configuration file?

The file could for now contain:

shell_aliases:
  - alias: o              # Provides an alias to `omni`
  - alias: ocd            # Provides an alias to `omni cd`
    target: cd
  - alias: whatever       # Provides an alias to `omni whatever`
    target: whatever

Feature: support for `suggest_clone` in repositories' omni configuration

A repository can specify configuration, but could also suggest repositories to clone, which would trigger a prompt asking the user which repositories they want to clone.

Something like:

suggest_clone:
  - [email protected]:XaF/omni
  - [email protected]:omnicli/omni-example

We could also support a if parameter to conditionnally suggest repositories:

suggest_clone:
  - [email protected]:XaF/omni1
  - if: <bash -c code>
    then:
      - [email protected]:XaF/omni2
    else:
      - [email protected]:omnicli/omni-example

Feature: `https://get.omnicli.dev` should provide an install script

In the image of what other tools such as chezmoi are doing with get.chezmoi.io, we should provide a script on get.omnicli.dev that will allow to:

install omni with the working method for the current user's setup
run any omni command at first

This would allow organizations to simply suggest their users to run:

$ sh -c "$(curl -fsLS get.omnicli.dev)" -- clone https://github.com/<orgname>/<mainrepo>

idea: self update should show/offer to show the changelog from the version

This could be done by adding into https://github.com/XaF/homebrew-omni/blob/main/Formula/resources/omni.json a changelog field with the list of things that changed.

Maybe moving to Conventional Commits would also allow for easier/cleaner changelog generation, since if we want to show the changelog, we probably want to hide most of the chores.

Failure to `git pull` or any other command handled in a spinner should allow to print the log

At the moment, a number of commands are doing work in the background and only showing progress.
This is good for the UI but not great for the UX when an error happens, forcing to run the following to see details:

omni ... | tee /dev/null

This shouldn't be necessary and errors should be either printed when they happen, or stored into a file that the user can then read or provide for debugging.

Allow for background updates of the omnipath

The whole point of doing foreground updates is being able to see if anything does not work as we would expect.
However, it would be possible to:

Trigger a background update
In case of error, store the error and log file
On next shell prompt, show a message saying there was an error during the last background update, and offer a command to see the error log

It would even be better if when calling omni, if an update is needed and the directory of the command being called is part of what needs updating, this is done sync first so the command can resume, and the rest is triggered as background update.

Caveats:

How does that work when the repository refresh needs a password ?
When it needs other kind of authentication ?

Feature: ask new users for basic configuration

This is useful for new user onboarding.

Basic configuration such as the default worktree they want to use and the format for cloned repositories in their worktree should be asked to new users. We could use the fact that the configuration file is nowhere to be found to suggest setting those up instead of using the default.

In some cases, organizations might want to push a default configuration file for their users. In which case, we need an alternate way of triggering the request as the configuration files might already exist.

Open question: should this also suggest setting up the shell integration if detecting it's not installed and able to identify the use of bash, zsh or fish?