Gather data from Zabbix back-end and load to InfluxDB in near real-time for enhanced performance and easier usage with Grafana.
As InfluxDB provides an excellent compression rate (in our case: 7x), this project could be used also to archive Zabbix data.
-
InfluxDB:
-
Grafana:
-
influxdb-zabbix:
- Install GO
- Setup you GOPATH
- Run
go get github.com/zensqlmonitor/influxdb-zabbix
- Edit influxdb-zabbix.conf to match your needs
-
PostgreSQL:
Create user:
CREATE USER influxdb_zabbix WITH PASSWORD '***'; GRANT USAGE ON SCHEMA public TO influxdb_zabbix;
Grants at the database level:
GRANT SELECT ON public.history, public.history_uint TO influxdb_zabbix; GRANT SELECT ON public.trends, public.trends_uint TO influxdb_zabbix;
Create indexes:
CREATE UNIQUE INDEX idx_history_clock_ns_itemid ON public.history USING btree (clock) TABLESPACE zabbixindex; CREATE UNIQUE INDEX idx_history_uint_clock_ns_itemid ON public.history_uint USING btree (clock) TABLESPACE zabbixindex; CREATE INDEX idx_trends_clock_itemid ON public.trends USING btree (clock) TABLESPACE zabbixindex; CREATE INDEX idx_trends_uint_clock_itemid ON public.trends_uint USING btree (clock) TABLESPACE zabbixindex;
-
MariaDB / MySQL:
Create user:
CREATE USER 'influxdb_zabbix'@'localhost' IDENTIFIED BY '***';
Grants at the database level:
GRANT SELECT ON zabbix.trends TO influxdb_zabbix@localhost; GRANT SELECT ON zabbix.trends_uint TO influxdb_zabbix@localhost; GRANT SELECT ON zabbix.history TO influxdb_zabbix@localhost; GRANT SELECT ON zabbix.history_uint TO influxdb_zabbix@localhost; flush privileges;
Create indexes:
CREATE UNIQUE INDEX idx_history_clock_ns_itemid ON history (clock) USING btree; CREATE UNIQUE INDEX idx_history_uint_clock_ns_itemid ON history_uint (clock) USING btree; CREATE INDEX idx_trends_clock_itemid ON trends (clock) USING btree; CREATE INDEX idx_trends_uint_clock_itemid ON trends_uint (clock) USING btree;
NB: For trends_* tables we can use pt-online-schema-change for online index create without lock, but for history_* tables we can only use create index, because primary key for these tables does not exist.
- Run in background:
go run influxdb-zabbix.go &
- Build in the current directory:
go build influxdb-zabbix.go
- Install in $GOPATH/bin:
go install influxdb-zabbix.go
- Go 1.7+
- TOML parser (https://github.com/BurntSushi/toml)
- Pure Go Postgres driver for database/sql (https://github.com/lib/pq/)
- Pure Go MySQL driver for database/sql (https://github.com/go-sql-driver/mysql/)
-
PostgreSQL and MariaDB/MySQL supported.
-
Tables that can be replicated are:
- history
- history_uint
- trends
- trends_uint
-
Tables like history_log, _text and _str are not replicated.
-
Configurable at table-level:
- interval: polling interval, minimum of 15 sec
- hours per batch : number of hours/batch to extract from zabbix backend
- output rows per batch : allow the destination load to be splitted in multiple batches
MIT-LICENSE. See LICENSE file provided in the repository for details