A monitoring solution for Osmosis full nodes with Prometheus, Grafana, cAdvisor, NodeExporter and alerting with AlertManager.
- Install Docker Engine
>= 1.13
sudo apt-get remove docker docker-engine docker.io
sudo apt-get update
sudo apt install docker.io -y
- Install Docker Compose
>= 1.11
sudo apt install docker-compose -y
-
Run an Osmosis full node. See https://get.osmosis.zone or https://docs.osmosis.zone/developing/cli/install.html to install the Osmosis binary
-
Enable the Prometheus metrics
- Set
prometheus = true
under instrumentation in the config.toml - Set
enable = true
andprometheus-retention-time = 1
under telemetry in the app.toml
- Ensure the following ports are not in use
3000
9100
9092
8001
- Export your public IP to an environment variable
export HOST_IP=$(dig +short txt ch whoami.cloudflare @1.0.0.1)
- Clone this repository on your host
git clone https://github.com/osmosis-labs/osmosis-monitor.git
cd osmosis-monitor
-
(optional) Define additional alerting rules. Edit the
prometheus/alert.rules
to configure additional rules. See Prometheus documentation for more details. -
(optional) Configure a
receiver
for the alerting notifications. Edit thealertmanager/config.yaml
to configure additional rules. See Alertmanager documentation for more details. -
Run the containers via
docker-compose
:
docker-compose up -d
The command will create the following containers:
Grafana
(visualize metrics)http://<host-ip>:3000
Prometheus
(metrics database)http://<host-ip>:9092
Prometheus-Pushgateway
(push acceptor for ephemeral and batch jobs)http://<host-ip>:9091
NodeExporter
(host metrics collector)cAdvisor
(containers metrics collector)
Once you have run docker-compose
(and Osmosis has caught up to the head of the chain):
-
Go to
http://<host-ip>:3000
-
Login with
admin
andadmin
as the username and password and set your new password -
Go to the dashboards tab (the icon that looks like four squares). Select browse and then select the
Osmosis Dashboard
.
The dashboard can also be reached directly at http://<host-ip>:3000/d/UJyurCTWz/osmosis-dashboard
To shut down all of the above docker containers but retain the data
docker-compose down
To shut down and delete all metrics collected
docker-compose down --volumes
The Osmosis Dashboard shows key metrics for monitoring the chain state as well as machine resource usage: