giantswarm / nomi

Benchmarking tool for fleet

Nomi

Nomi is a benchmarking tool that tests a fleet cluster. With Nomi, you can deploy benchmark units that employ Docker, rkt containers, or just raw systemd units. Nomi is able to collect some metrics and generate some plots from those. To make use of Nomi, you just need to define your own benchmark using a YAML file. Nomi parses this file and runs the benchmark according to the instructions defined in it. Additionally, Nomi provides the possibility to define instructions in one line using the parameter raw-instructions.

Requirements

Nomi requires to be installed on a fleet cluster-node to run properly.

Dependencies:

• fleet and systemd running on the host machine.
• In case you want to run Docker or rkt containers, the respective tool needs to be running on the host machines, too.
• Optional: To generate gnu plots, support for gnuplot is required on the host machine. Alternatively, you can run Nomi as a Docker container, which comes with gnuplot installed, as shown below.

Getting Nomi

Download the latest tarball from here: https://downloads.giantswarm.io/nomi/latest/nomi.tar.gz

Clone the latest git repository from here: https://github.com/giantswarm/nomi.git

Get the latest docker image from here: https://hub.docker.com/r/giantswarm/nomi/

Running Nomi:

nomi help

Run Nomi from source:

make
./nomi help

More information on how to run Nomi and its required parameters in: docs

Further Steps

Check more detailed documentation: docs

Check code documentation: godoc

Future Development

• Future directions/vision

Contact

• Mailing list: giantswarm
• IRC: #giantswarm on freenode.org
• Bugs: issues

Contributing & Reporting Bugs

See CONTRIBUTING for details on submitting patches, the contribution workflow as well as reporting bugs.

License

Nomi is under the Apache 2.0 license. See the LICENSE file for details.

Origin of the Name

nomi (のみ[蚤]) is Japanese for flea.

nomi's Issues

enh: Add option to simulate failure in a HA setting

In Chaos-Monkey style, introduce random failure options to benchmark fleet/etcd behavior.

Use-case: I need to benchmark fleet performance in split-brain scenarios (ex. losing an availability zone) or during raft elections of a new leader upon a node failure, or at times of unreliable networking.

Add units metrics for the y2 axes of one of the GNUplots

Units metrics are missing in one of the plots which has y2 axes, more in particular 'units_start_multi'.

Add support to run units that deploy RKT containers

We want to deploy units with Fleemmer that create containers using rkt. Therefore we could test the behavior of Fleemmer when playing with rkt.

Allow submission of custom fleet systemd units and timers

Benchmark complex fleet units and timers

• Add support to use custom a fleet unit service
• Add support to use a group of fleet units and timers

Minimize the impact in the system when collecting system resource metrics

Our current implementation deploys a docker container to collect the metrics from systemd, fleet, etcd (coming soon). We need a wide amount of metrics and a collector system that minimizes its noise over the global picture.

Enable to use custom docker and rkt containers as benchmark units

We want to enable users to define which containers to use when running a benchmark that makes use of rkt or docker with Nomi. A generic mechanism of notification + keep alive should be needed to support such a feature.

Add fine-grain measurement for more operations (create, load, start)

We want to extract the timing of fine-grain operations when dealing with fleet. At this moment, we handle the start operation which internally measure (create/load/start) as a single operation. Therefore we can get times that don't identify the times spent when processing different operations.

ping @giantswarm/team-core

Add support to run units that deploy Docker containers

We want to deploy units with Fleemmer that create containers using Docker. Therefore we could test the behavior of Fleemmer when playing with Docker.