financial-times / athloi

💪 Athloi is a tool to assist with the management of multi-package repositories (a.k.a. monorepos)

Athloi

Athloi is a tool to assist with the management of multi-package repositories (a.k.a. monorepos) with git and npm. It provides an interface to execute commands and scripts within the scope of each package.

Features

• Provides a number of helpers to improve your monorepo workflow such as: installing all dependencies, running npm or custom scripts, publishing public packages, updating version numbers, etc.
• Capable of running tasks serially or in parallel whilst preserving topological sort order between cross-dependent packages.
• Options to filter the packages to target either by name or package.json field
• Works well with Yarn workspaces.

Getting Started

Start by installing Athloi within your project using npm.

npm install --save-dev @financial-times/athloi

Next you must configure where Athloi should look for the directories containing your packages. To do this you must add an extra "workspaces" property to your root package.json file if you do not have one already.

This property is an array of globs matching your package directories. This designed to be compatible with Yarn.

{
	"workspaces": ["components/*", "tools/*"]
}

Commands

Please note: Before executing a command Athloi will sort the packages [topologically] based on their cross-dependencies and run tasks in this order.

exec

Runs an arbitrary command within the scope of each package.

athloi exec npm install

A double-dash (--) is necessary to pass any dashed arguments to the command being executed.

athloi exec -- npm i -D lodash

run

Runs an npm script in each package that defines that script.

athloi run build

script

Runs the given Node script in the scope of each package.

athloi script path/to/task.js

version

Updates the release number for public packages and their cross-dependencies and writes the data back to package.json. The given tag must parseable as a valid semver number.

athloi version v1.0.0

Please note that when using the version command with the filter option any cross-dependent packages which have been excluded will fall back to their latest published version on npm.

publish

Runs npm publish in the scope of each public package.

athloi publish

Dashed arguments may be passed using a double dash (--)

athloi publish -- --access=public

Options

concurrency

A global option which will execute up to the given number of tasks concurrently. By default one task will be run at a time.

# run a lint script in up to 3 packages at a time
athloi run lint --concurrency 3

preserve-order

A global flag which will ensure tasks maintain topological sort order. When used with a concurrency value higher than 1 this option will force queued tasks to wait for any still running tasks in cross-dependent packages to finish first.

# run a concurrent build script but ensure dependencies are built first
athloi run build --concurrency 5 --preserve-order

filter

A global option which can be used for all tasks. It filters packages based on the value of a field within their package manifest or the package name.

# Run a build script in only the packages marked as private
athloi run build --filter private:true

The value of the field will be coerced using JSON.parse() so boolean and number values can be used and string values must use double-quotes.

Property values inside arrays and objects can also be matched:

# Run the script for packages with a keyword of "demo"
athloi run build --filter 'keywords:"demo"'

# Run the script for packages with a dependency on the "lodash" package
athloi run build --filter 'dependencies:"lodash"'

The field name preceeding the colon (:) is optional and if omitted Athloi will default to checking package names, ignoring the npm organisation name.

# Run a build script for only the package named "@financial-times/x-interaction"
athloi run build --filter x-interaction

When filtering by package name you can also use a glob pattern to match them using wildcard characters:

# Run a build script for all packages with names beginning "@financial-times/x-"
athloi run build --filter 'x-*'

What's with the name?

One of the twelve labours of Hercules (hoi hērakleous athloi) was to slay the Lernean Hydra.

licence

isc

athloi's Issues

Add global filter option

It would be very useful to add a global filter option which may be applied to all tasks provided by this tool. The use case for this is to restrict tasks to a subset of packages within the repository being managed.

The filter option could be a pattern match based on the package name, e.g.

athloi publish --filter my-package

Alternatively, and more flexibly, it could be based on a field of the package manifest, e.g.

athloi run build --filter "private:true"

The most common use case may be for running NPM tasks in specific packages without the need to change directory, or for releasing a subset of packages.

Update Dependencies (Renovate Bot)

This repository is up-to-date and has no outstanding updates open or pending.

Problem installing package as dev dependency

I've been getting horrible install problems all day. I think I've narrowed down the cause.

Say packageA in the monorepo has a dependency on packageB in the monorepo . Then adding packageA as a dependency of the root package.json and packageB as a devDependency seems to cause all sorts of problems.

Nicer protip argv0

It's not always run as athloi, can we work out what the command was?

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Cannot find preset's package (github>financial-times/renovate-config-next)

Update Dependencies (Renovate Bot)

This master issue contains a list of Renovate updates and their statuses.

Pending Approval

These branches will be created by Renovate only once you click their checkbox below.

• Update dependency jest to v26

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to ignore the schedule.

• Pin ft-snyk-orb orb to 0.0.14
• Update dependency commander to v5

---

Advanced

• Check this box to trigger a request for Renovate to run again on this repository

lerna add replacement

When a monorepo.json#packages is incorrect, the error is unhelpful

given this monorepo.json:

{
	"packages": "examples/*"
}

(note that "packages" is a string), i get the error patterns.join is not a function

could this be caught earlier?

if (!Array.isArray(packages)) {
  throw new Error("The packages config must be an array. Check your athloi config in 'monorepo.json' or 'workspaces' in 'package.json'")

or something

Nicer parallel output

If running in a TTY show the output of all the tasks in separate Blessed (or whatever) panes

Fix docs regarding filtering

Filtering in the following ways don't work:

• a) athloi run build --filter scope:anvil
• b) athloi run build --filter scope:"anvil"

But filtering like this works:

• c) athloi run build --filter scope:'"anvil"'

This is however not accurately documented in the README.
As an aside, it seems that the intention is for mechanism b but this is not the case.

See: https://github.com/Financial-Times/athloi/blob/master/src/filter-packages.js#L6

readme does not explain "cross-dependencies" or "topological order"

• these terms are quite jargony and confusing to a newcomer
• this also means the readme doesn't adequately explain how to set up your monorepo to work with athloi (i.e. using file: version specifiers in the package.json)

Dependency versions don't update when using version task with filter

Running the version task with a filter will update the target package/s but not their dependencies. This is because there is an explicit list of dependencies to update. This also requires the addition of an option to specify a second version tag. This is as expected but allows packages to be published with invalid dependencies.

Steps to recreate:

npx athloi version v2.0.0 -F package-a

Output:

{
  "name": "package-a",
  "version": "2.0.0",
  "dependencies": {
    "package-b": "file:../package-b"
  },
  "devDependencies": {
    "package-c": "file:../package-c"
  }
}

Expected:

{
  "name": "package-a",
  "version": "2.0.0",
  "dependencies": {
    "package-b": "???"
  },
  "devDependencies": {
    "package-c": "???"
  }
}

Custom tasks

Let a monorepo create its own tasks (and document the task format)

You didn't actually write the create command before shipping

🙊

Problem installing version 2

This branch of biz-ops-api https://github.com/Financial-Times/biz-ops-api/tree/rhys/absorb-treecreeper installs with atthloi 1 but not athloi 2

@apaleslimghost any idea why?

don't exit on package missing a package.json

ninety nine times out of a hundred, this happens because somebody upstream has deleted a package, you've pulled their branch, but git has kept the folder around because it contains gitignored files. it's not your fault the package.json isn't there (in fact, the problem isn't that the package.json isn't there, it's that the folder is) and the correct fix (delete the folder) isn't obvious. the other one time, a warning would be enough