Dashboard for Tarantool application and database server monitoring, based on grafonnet library.
Our pages on Grafana Official & community built dashboards: InfluxDB version, Prometheus version.
Refer to dashboard documentation page for prerequirements and installation guide.
-
Open Grafana import menu.
-
To import a specific dashboard, choose one of the following options:
- paste the dashboard id (
12567
for InfluxDB dashboard,13054
for Prometheus dashboard), or - paste a link to the dashboard (https://grafana.com/grafana/dashboards/12567 for InfluxDB dashboard, https://grafana.com/grafana/dashboards/13054 for Prometheus dashboard), or
- paste the dashboard JSON file contents, or
- upload the dashboard JSON file.
- paste the dashboard id (
-
Set dashboard name, folder, uid, choose corresponding datasource from drop-down list and set datasource-related query parameters.
You need to set the following variables for InfluxDB datasource:
Measurement
,Policy
(default valie isdefault
).
You need to set the following variables for Prometheus datasource:
Job
,Rate time range
(default valie is2m
).
Datasource variables can be obtained from your datasource configuration. Variables for example monitoring cluster are described in Monitoring cluster section.
For guide on setting up your monitoring stack refer to documentation page.
This repository provides preconfigured monitoring cluster with example Tarantool app and load generatior for local dashboard development and tests.
docker-compose up -d
will start 6 containers: Tarantool App, Tarantool Load Generator, Telegraf, InfluxDB, Prometheus and Grafana, which build cluster with two fully operational metrics datasources (InfluxDB and Prometheus), extracting metrics from Tarantool App example project. We recommend using the exact versions we use in experimental cluster (e.g. Grafana v8.1.5). After start, Grafana UI will be available at localhost:3000. You can also interact with Prometheus at localhost:9090 and InfluxDB at localhost:8086.
To set up an InfluxDB dashboard for monitoring example app, use the following variables:
Measurement
:tarantool_app_http
;Policy
:default
.
To set up an Prometheus dashboard for monitoring example app, use the following variables:
Job
:tarantool_app
;Rate time range
:2m
.
If you want to monitor Tarantool cluster deployed on your local host, you can use monitoring cluster similar to example app one.
Configure Telegraf/Prometheus to monitor your own app in example_cluster/telegraf/telegraf.localapp.conf
and example_cluster/prometheus/prometheus.localapp.yml
.
Use host.docker.internal
as your machine host in configuration and set cluster instances ports as targets and correct metrics HTTP path.
See more setup tips in documentation.
Start cluster with
docker-compose -f docker-compose.localapp.yml -p localapp-monitoring up -d
After start, Grafana UI will be available at localhost:3000. You can also interact with Prometheus at localhost:9090 and InfluxDB at localhost:8086.
go
v.1.14 or greater is required to install build and test dependencies.
Run
make build-deps
to install dependencies that are required to build dashboards.
Run
make test-deps
to install build dependencies and dependencies that are required to run tests locally.
You can compile Prometheus dashboard template with
jsonnet -J ./vendor/ -e "local dashboard = import 'dashboard/prometheus_dashboard.libsonnet'; dashboard.build()"
and InfluxDB dashboard template with
jsonnet -J ./vendor/ -e "local dashboard = import 'dashboard/influxdb_dashboard.libsonnet'; dashboard.build()"
To save output into output.json
file, use
jsonnet -J ./vendor/ -e "local dashboard = import 'dashboard/prometheus_dashboard.libsonnet'; dashboard.build()" -o ./output.json
and to save output into clipboard, use
jsonnet -J ./vendor/ -e "local dashboard = import 'dashboard/prometheus_dashboard.libsonnet'; dashboard.build()" | xclip -selection clipboard
You can run tests with
make run-tests
Compiled dashboard test files can be updated with
make update-tests
It also formats all source files with jsonnetfmt
.
You can add your own custom panels to the bottom of the template dashboard.
-
Add tarantool/grafana-dashboard as a dependency in your project with jsonnet-bundler. Run
jb init
to initialize jsonnet-bundler and add this repo to
jsonnetfile.json
as a dependency:{ "version": 1, "dependencies": [ { "source": { "git": { "remote": "https://github.com/tarantool/grafana-dashboard" } }, "version": "master" } ], "legacyImports": true }
Run
jb install
to install dependencies.
grafonnet
library will also be installed as a transitive dependency. -
There are two main templates:
grafana-dashboard/dashboard/prometheus_dashboard.libsonnet
andgrafana-dashboard/dashboard/influxdb_dashboard.libsonnet
. Import one of them in your jsonnet script to build your own custom dashboard.# my_dashboard.jsonnet local prometheus_dashboard = import 'grafana-dashboard/dashboard/prometheus_dashboard.libsonnet'; local influxdb_dashboard = import 'grafana-dashboard/dashboard/influxdb_dashboard.libsonnet';
-
To add your custom panels to a dashboard template, you must create panel objects.
A row panel can be created by using the following script:
# my_dashboard.jsonnet local common_panels = import 'grafana-dashboard/dashboard/panels/common.libsonnet'; local my_row = common_panels.row('My custom metrics')
Panel with metrics data consists of a visualisation base (graph, table, stat etc.) and one or several datasource queries called "targets". To build a simple visualization graph, you may use
common_panels.default_graph
util.# vendor/grafana-dashboard/dashboard/panels/common.libsonnet default_graph( # graph panel shortcut title, # The title of the graph panel description, # (optional) The description of the panel datasource, # Targets datasource. Use dashboard/variable.libsonnet to fill this value format, # (default 'none') Unit of the Y axes min, # (optional) Min of the Y axes max, # (optional) Max of the Y axes labelY1, # (optional) Label of the left Y axis decimals, # (default 3) Override automatic decimal precision for legend and tooltip decimalsY1, # (default 0) Override automatic decimal precision for the left Y axis legend_avg, # (default true) Show average in legend legend_max, # (default true) Show max in legend panel_height, # (default 8) Panel heigth in grid units panel_width, # (default 8) Panel width in grid units, max is 24 )
Panel size is set with grid units. Grafana uses square-type grid where dashboard width is 24 units. For example, row size is 24 x 1 units and Grafana new panel size is 12 x 9 units.
If you want to build non-graph panel or a graph panel with more complicated configuration, use
grafonnet
templates. You must set a size of each panel before adding it to our dashboard template. For eachgrafonnet
panel, add{ gridPos: { w: width, h: height } }
to it. For example,local grafana = import 'grafonnet/grafana.libsonnet'; local my_graph = grafana.graphPanel.new( title='My custom panel', points=true, ) { gridPos: { w: 6, h: 4 } };
To build a target, you may also use
common_panels
utils.# vendor/grafana-dashboard/dashboard/panels/common.libsonnet default_metric_target( # plain "select metric" shortcut datasource, # Target datasource. Use grafana-dashboard/dashboard/variable.libsonnet to fill this value metric_name, # Target metric name to select job, # (Prometheus only) Prometheus metrics job. Use grafana-dashboard/dashboard/variable.libsonnet to fill this value policy, # (InfluxDB only) InfluxDB metrics policy. Use grafana-dashboard/dashboard/variable.libsonnet to fill this value measurement, # (InfluxDB only) InfluxDB metrics measurement. Use grafana-dashboard/dashboard/variable.libsonnet to fill this value converter, # (InfluxDB only, default 'mean') InfluxDB metrics converter (aggregation, selector, etc.) ), default_rps_target( # counter metric transformed to rps shortcut datasource, # Target datasource. Use grafana-dashboard/dashboard/variable.libsonnet to fill this value metric_name, # Target metric name to select job, # (Prometheus only) Prometheus metrics job. Use grafana-dashboard/dashboard/variable.libsonnet to fill this value rate_time_range, # (Prometheus only) Prometheus rps computation rate time range. Use vendor/grafana-dashboard/dashboard/variable.libsonnet to fill this value policy, # (InfluxDB only) InfluxDB metrics policy. Use grafana-dashboard/dashboard/variable.libsonnet to fill this value measurement, # (InfluxDB only) InfluxDB metrics measurement. Use grafana-dashboard/dashboard/variable.libsonnet to fill this value )
To build more compound targets, use
grafonnet
libraryprometheus
andinfluxdb
templates.To use dashboard-wide input and template variables in your queries you must use
grafana-dashboard/dashboard/variable.libsonnet
. It imports json object with variable values you neet to set in your queries.If you want to build a Prometheus dashboard, use
datasource=variable.datasource.prometheus, job=variable.prometheus.job, rate_time_range=variable.prometheus.rate_time_range
in your targets.
If you want to build an InfluxDB dashboard, use
datasource=variable.datasource.influxdb, policy=variable.influxdb.policy, measurement=variable.influxdb.measurement
in your targets.
To add a target to a panel, call
addTarget(target)
.To summarise, you can build a simple 'select metric' prometheus panel with
local common_panels = import 'grafana-dashboard/dashboard/panels/common.libsonnet'; local variable = import 'grafana-dashboard/dashboard/variable.libsonnet'; local my_custom_component_memory_graph = common_panels.default_graph( title='My custom component memory', description=||| My custom component used memory. Shows mean value. |||, datasource=variable.datasource.prometheus, format='bytes', panel_width=12, panel_height=6, ).addTarget(common.default_metric_target( datasource=variable.datasource.prometheus, metric_name='my_component_memory', job=variable.prometheus.job, ))
and a simple rps panel with
local common_panels = import 'grafana-dashboard/dashboard/panels/common.libsonnet'; local variable = import 'grafana-dashboard/dashboard/variable.libsonnet'; local my_custom_component_rps_graph = common.default_graph( title='My custom component load', description=||| My custom component processes requests and collects info on process to summary collector 'my_component_load_metric'. |||, datasource=variable.datasource.prometheus, labelY1='requests per second', panel_width=18, panel_height=6, ).addTarget(common.default_rps_target( datasource=variable.datasource.prometheus, metric_name='my_component_load_metric_count', job=variable.prometheus.job, rate_time_range=variable.prometheus.rate_time_range, ))
Corresponding InfluxDB panels could be built with
local common_panels = import 'grafana-dashboard/dashboard/panels/common.libsonnet'; local variable = import 'grafana-dashboard/dashboard/variable.libsonnet'; local my_custom_component_memory_graph = common_panels.default_graph( title='My custom component memory', description=||| My custom component used memory. Shows mean value. |||, datasource=variable.datasource.influxdb, format='bytes', panel_width=12, panel_height=6, ).addTarget(common.default_metric_target( datasource=variable.datasource.influxdb, metric_name='my_component_memory', policy=variable.influxdb.policy, measurement=variable.influxdb.measurement, )), local my_custom_component_rps_graph = common.default_graph( title='My custom component load', description=||| My custom component processes requests and collects info on process to summary collector 'my_component_load_metric'. |||, datasource=variable.datasource.influxdb, labelY1='requests per second', panel_width=18, panel_height=6, ).addTarget(common.default_rps_target( datasource=variable.datasource.influxdb, metric_name='my_component_load_metric_count', policy=variable.influxdb.policy, measurement=variable.influxdb.measurement, ))
For more panel tips and examples, please examine this template dashboard source code and test cases.
To add your custom panels, call
addPanel(panel)
oraddPanels(panel_array)
in dashboard template:# my_dashboard.jsonnet local prometheus_dashboard = import 'grafana-dashboard/dashboard/prometheus_dashboard.libsonnet'; ... local my_dashboard_template = prometheus_dashboard.addPanels([ my_row, my_custom_component_memory_graph, my_custom_component_rps_graph ]);
Finally, call
build()
to compute panels positions and build a resulting dashboard:# my_dashboard.jsonnet ... my_dashboard_template.build()
Do not use
;
in the end of your script so resulting dashboard will be returned as output. -
To save resulting dashboard into
output.json
file, usejsonnet -J ./vendor/ my_dashboard.jsonnet -o ./output.json
and to save output into clipboard, use
jsonnet -J ./vendor/ my_dashboard.jsonnet -o ./output.json | xclip -selection clipboard
If you have questions, please ask it on StackOverflow or contact us in Telegram: