yunlzheng / prometheus-pusher

collection jobs/exportes data and push to pushgateway

prometheus-pusher

collection prometheus data and push to pushgateway

Architecture

Feature

• Support Prometheus Config File
• Support Prometheus Service Discovery
• Support Custom Label

How To Use

Prepare prometheus config file /ect/prom-conf/prometheus.yml

global:                                                                         
  scrape_interval:     15s
  evaluation_interval: 15s
  external_labels:                                                              
      monitor: 'exporter-metrics'                                                
                                                                                
scrape_configs:

- job_name: 'HostsMetrics'
  dns_sd_configs:
  - names:
    - node-exporter
    refresh_interval: 15s
    type: A
    port: 9100

- job_name: 'ContainerMetrics'
  static_configs:
    - targets:
      - 'rancher-server:9108'
- job_name: 'RancherServerMetrics'
  dns_sd_configs:
  - names:
    - cadvisor
    refresh_interval: 15s
    type: A
    port: 8080

- job_name: 'RancherApi'
  dns_sd_configs:
  - names:
    - 'prometheus-rancher-exporter'
    refresh_interval: 15s
    type: A
    port: 9173

- job_name: 'Prometheus'
  static_configs:
    - targets:
      - '127.0.0.1:9090'

In command line:

export PUSH_GATEWAY=http://pushgateway.example.org:9091
./prometheus_pusher -config.file=prometheus.yml 

In docker-compose

Note: you should set the environment variable of PushGateway address

version: '2'
services:
  pusher:
    image: wisecity/prometheus-pusher
    environment:
      PUSH_GATEWAY: http://pushgateway.example.org:9091
    volumes:
    - /ect/prom-conf:/etc/prom-conf
    entrypoint:
    - /bin/prometheus_pusher
    - -config.file
    - /etc/prom-conf/prometheus.yml

Add custom metrics labels

In some case, if you want add external metrics key for the origin metrics data. You can use customLabels.

In our case, we collection container data from mutil rancher environment with cadvistor. We want the prometheus query express can precise positioning the container from different environment. So we add rancher environment uuid as the custom label.

-config.customLabels=environmentUUID -config.customLabelValues=$(curl http://rancher-metadata/latest/self/host/environment_uuid)

Note. customLabel will overwrite the origin metrics label

./prometheus_pusher -config.file=prometheus.yml -config.customLabels=label1,label2 -config.customLabelValues=value1,value2

prometheus-pusher's Issues

main.go运行状况

allow using more metrics

i noticed when trying this out that this line https://github.com/yunlzheng/prometheus-pusher/blob/master/scrape/exporterscrape.go#L147 is filtering labels. could an option to added to specify prefixes via environment var, and maybe even allow a * to pass all metrics through?

i dont mind submitting a PR if necessary.

pushgateway看不到prometheus-pusher推送的metrics

能看懂中文么？我就不写英语了。

我使用docker-compose启动的prometheus-pusher。而且prometheus和pushgateway都搭建成功，但不知道为什么pushgateway看不到prometheus-pusher推送的任何metrics，是pusher没有UI界面吗？怎么才能知道pusher有没有搭建成功？找了好久，也找不到原因。

prometheus-pusher无法将sql-exporter收集的数据推送到pushgateway上面

图片如下：

部署文件如下：

[root@prometheus pushgateway-pusher]# cat docker-compose.yml
version: '2'
services:
  prometheus-pusher:
    image: wisecity/prometheus-pusher:latest
    environment:
      PUSH_GATEWAY: http://192.168.10.30:9091
      TZ: Asia/Shanghai
    volumes:
    - ./prom-conf:/etc/prom-conf/
    ports:
    - "8080:8080"
    entrypoint:
    - /bin/prometheus_pusher
    - -config.file
    - /etc/prom-conf/prometheus.yml

目前我这边也看不到pushgateway上面的值。日志如下：

send data to pushgateway :http://192.168.10.30:9091/metrics/job/SqlMetrics/instance/192-168-10-20-9237-metrics

prometheus-pusher配置如下：

cat prom-conf/prometheus.yml
global:
  scrape_interval:     15s
  evaluation_interval: 15s
  external_labels:
      monitor: 'exporter-metrics'

scrape_configs:
  - job_name: 'SqlMetrics'
    static_configs:
    - targets:
      - '192.168.10.20:9237'

sql-exporter采集数据如下：

[root@postgres-master mysql-exporter]# curl -s 192.168.10.20:9237/metrics |grep "ids"
sql_sql_queries{col="ids",database="mydb",driver="mysql",host="192.168.10.20:3306",sql_job="SqlMetrics",user="root"} 5

但是pushgateway上面没有将上述的指标推送到pushgateway
@yunlzheng 麻烦帮忙看看

prometheus-pusher 配置consul

prometheus server 和 consul server 都在一个机房， 通过部署pushgateway+ pusher采集另外机房的数据 但是怎么配置pusher所在机房取到异地机房consul 中注册服务的信息呢？