adefossez-zenika / gravitee-prometheus-exporter

My attempt to code an exporter for gravitee.io

Gravitee.IO Prometheus exporter

An exporter for Prometheus based on the logs backend (ElasticSearch) of Gravitee.io.

Environment variables

• GPE_GIO_URL : Gravitee.IO management Url (default : http://localhost:8005/management )
• GPE_GIO_USR Gravitee basic auth Username (default : admin)
• GPE_GIO_PWD Gravitee basic auth Password (default : admin)
• GPE_ES_URL Elasticsearch Url (default : http://localhost:9200))
• GPE_ES_USER ElasticSearch basic auth user (default : 'None')
• GPE_ES_PWD Elasticsearch basic auth password (default : 'None')
• GPE_ES_INDEX Index Pattern for Gravitee index. Must contains %Y, %m, %d. (default : gravitee-%Y.%m.%d)
• GPE_PORT Port to listen on. (default : 8888)
• LOG_LEVEL (ERROR, WARNING, INFO, DEBUG)

Metrics returned

# HELP api_count Number of APIS in Gravitee.io
# TYPE api_count gauge
api_count 2.0
# HELP gio_http_call Api calls
# TYPE gio_http_call gauge
gio_http_call{api_id="...",api_name="V1Customer",description="About customer ressources...",owner="admin",responseCode="200",uri="/v1/customers"} 166.0
gio_http_call{api_id="...",api_name="V1Customer",description="About customer ressources...",owner="admin",responseCode="201",uri="/v1/customers"} 10.0
gio_http_call{api_id="...",api_name="V1Customer",description="About customer ressources...",owner="admin",responseCode="404",uri="/v1/customers"} 3.0
# TYPE gio_http_call_familly gauge
gio_http_call_familly{api_id="...",api_name="V1Customer",description="About customer ressources...",owner="admin",responseCodeFamily="1XX",uri="/v1/customers"} 0.0
gio_http_call_familly{api_id="...",api_name="V1Customer",description="About customer ressources...",owner="admin",responseCodeFamily="2XX",uri="/v1/customers"} 176.0
gio_http_call_familly{api_id="...",api_name="V1Customer",description="About customer ressources...",owner="admin",responseCodeFamily="3XX",uri="/v1/customers"} 0.0
gio_http_call_familly{api_id="...",api_name="V1Customer",description="About customer ressources...",owner="admin",responseCodeFamily="4XX",uri="/v1/customers"} 3.0
gio_http_call_familly{api_id="...",api_name="V1Customer",description="About customer ressources...",owner="admin",responseCodeFamily="5XX",uri="/v1/customers"} 0.0

Internals

This application scrape Elasticsearch to retrieve gravitee logs. It consolidate datas by calling Gravitee Management API.

Cache

To avoid too much trafic load on elasticsearch & gravitee, a little cache layer is added

• Api informations is available for 300 seconds (api name, owner name, description etc)
• Logs Count is available for 10 seconds
• Api count is available for 300 seconds

gravitee-prometheus-exporter's Issues

Logs format

Logs format should be chosen (plaintext, json)

restarts on K8S

Got some random restarts on my cluster, but no info why it happens.
More logs, please!

Logs output

Logs are displayed on stdout, I want to be able to write logs in a file

api_count non-histogram and non-summary metrics should not have "_count" suffix

promtool return an error.

CHange api_count mettric name.

Loggin

Add logLevel in log messages 👍

2019-10-17 05:50:17 Something went wrong when fetching https://lys-d-elastic-hmbu-p-lys-loyalty.aivencloud.com:24115/gravitee-request-2019.10.17/_search, got status code 404

Exit on Gravitee Call

gravitee-prometheus-exporter/app/application.py

Line 45 in 3e69f50

logging.error("Something went wrong when fetching {}, got status code {}" .format(

Should only Log when Gravitee Api is unavailable