This is the router (or relay, or reverse-proxy) for Graphite. It routes incoming records according to the specified rules. The Nanotube is designed for high-load systems. It is used at Booking.com to route up to a million incoming records/sec on a single box with a typical production config.
- Get this repository with
go get -u github.com/bookingcom/nanotube
- Navigate to it and run
make
to build - Run it with
./nanotube -config config/config.toml
Command line options:
-config string
Path to config file.
-validate
Validate configuration files.
-version
Print version info.
The only supported Go version is 1.18
.
are Linux and Darwin.
The main supported protocol is line. The records have the structure:
path.path.path value datetime
The support of Open Telemtry gRPC is experimental.
Please refer to the sample config for examples and documentation.
Is defined in the rules config that is in turn referred to in the main config. This is how it works:
- Routing rules are applied to each incoming record in order;
- If regex or prefix in a rule matches, the record is sent to the clusters in the
clusters
list;- If
continue
istrue
continue matching next rules, stop otherwise.false
is the default;
- If
- If no regex or prefix matches, continue down the list of rules;
- If regex or prefix in a rule matches, the record is sent to the clusters in the
- Multiple rules can be matched to each record;
- Each record is sent to a single cluster only once. If two rules send it to same cluster, only one instance will be sent;
- Cluster names must be from the set in the clusters config.
Optionally, it is possible to apply the rewrite rules. This is how they work:
- rewrites are applied before the routing;
- all rules are applied to each record one-by-one in order. The record may be modified along the way;
- rule matches if
from
matches;- then metric path is rewriten to
to
in place; - if
copy
istrue
the original metric is copied and sent directly to be routed skipping the following re-writes.copy
isfalse
be default.
- then metric path is rewriten to
Nanotube validates every record it receives. Validation checks performed are:
- does the record consist of three fields
- is value in a record a valid real number
- is time in the record a valid epoch
It is possible to turn on record normalization with the NormalizeRecords
option in general configuration. It is on by default. Normalizations performed are mainly for backward compatibility with carbon-c-relay
, namely:
- Double or more subsequent dots in path are converted to a single dot: a.b..c -> a.b.c
- Prefix and trailing dots in path are removed: .a.b.c. -> a.b.c
- Spaces in record are normalized to a single space character: a.b.c␣␣␣1.23\t1234567 -> a.b.c␣1.23␣1234567
- Characters not in the set
[0-9a-zA-Z-_:#]
are replaced with_
. Tags have their own allowed chars but are not supported for now.
Nanotube supports zero-downtime reload that can be triggered with USR2
signal. It will update the binary and try to load updated config. If the config is invalid, the old instance will keep running.
Tags are not supported for now.
Design details are in the design doc.
We support OpenTelemetry by using v0.7.0 of protobuf protocol.
This program was originally developed for Booking.com. With approval from Booking.com, the code was generalized and published as Open Source on GitHub, for which the authors would like to express their gratitude.