CPU stats not working ? about dockprom

[rtmcdo]

I have the same issue as TZK. I've taken the {image!="", container_label_org_label_schema_group!="monitoring"} from the query string, and it starts working with following results:

[stefanprodan]

Maybe is something wrong with cAdvisor, you could expose cAdvisor port on the host and check the dashboard for CPU stats.

In order to this you need to add the port 8080:

  cadvisor:
    image: google/cadvisor:v0.26.1
    container_name: cadvisor
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:rw
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
    restart: unless-stopped
    expose:
      - 8080
    ports:
      - 8080:8080

[TZK-]

I already tried this and the container CPU stats are good that's why I don't understand dashboard isn't grabbing the right data:

[stefanprodan]

Does the Monitor Services dashboard show any CPU activity?

It should look like this

[TZK-]

It does but it seems to be not working such as the Container Dashboard. I also tried to stress the grafana container but data stay like so:

[stefanprodan]

I've installed stress into grafana container and this is what I get

What's your Prometheus version? I'm running on v1.7.1

[TZK-]

I'm also running on Prometheus 1.7.1...

Some informations about my system:

Distributor ID: Ubuntu
Description:    Ubuntu 16.04.2 LTS
Release:        16.04
Codename:       xenial

Client:
 Version:      17.06.0-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:23:31 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.06.0-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:19:04 2017
 OS/Arch:      linux/amd64
 Experimental: false

docker-compose version 1.8.0, build unknown
docker-py version: 1.9.0
CPython version: 2.7.12
OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016

[stefanprodan]

The only difference is that I'm on 17.05.0-ce, going to upgrade a test server and get back to you.

[TZK-]

I got it work on a different server (Debian 9) with the same Docker/docker-compose versions...

[mihalikv]

Hi, i am facing issue that CPU load for containers is all zero. Related issue in cadvisor is (google/cadvisor#2051). Compose file is following:

version: "2.1"
services:
  node_exporter:
    image: prom/node-exporter:v0.17.0
    user: root
    privileged: true
    volumes:
    - /proc:/host/proc:ro
    - /sys:/host/sys:ro
    - /:/rootfs:ro
    command:
    - '--path.procfs=/host/proc'
    - '--path.sysfs=/host/sys'
    - '--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|host|etc)($$|/)'
    restart: unless-stopped
    networks:
    - app_net

#other exporters

  cadvisor:
    image: google/cadvisor:latest
    command:
    - '-enable_load_reader=true'
    volumes:
    - /var/run/docker.sock:/var/run/docker.sock:ro
    - /dev/kmsg:/dev/kmsg:ro
    - /:/rootfs:ro
    - /var/run:/var/run:rw
    - /sys:/sys:ro
    - /var/lib/docker/:/var/lib/docker:ro
    depends_on:
    - webserver
    - webapp
    - redis
    - rabbitmq
    - db
    restart: unless-stopped
    networks:
    - app_net
  grafana:
    image: grafana/grafana
    volumes:
    - ./services/grafana/config:/etc/grafana
    - ./services/grafana/dashboards_json:/var/lib/grafana/dashboards
    depends_on:
    - prometheus
    restart: unless-stopped
    networks:
      app_net:
        ipv4_address: 172.16.238.250
  prometheus:
    build: ./services/prometheus
    ports:
    - "9090:9090"
    volumes:
    - ./data/prometheus:/prometheus
    - ./services/prometheus/config:/etc/prometheus
    command:
    - --config.file=/etc/prometheus/prometheus.yml
    depends_on:
    - cadvisor
    - nginx_exporter
    - elasticsearch_exporter
    - rabbitmq_exporter
    - redis_exporter
    - postgres_exporter
    - node_exporter
    restart: unless-stopped
    networks:
    - app_net
  #webapp related containers

networks:
  app_net:
    driver: bridge
    ipam:
      driver: default
      config:
      - subnet: 172.16.238.0/24
        gateway: 172.16.238.1

I added flag enable_load_reader and also tried to run it in host mode but it didn't help.
I writing here because this repo is more related to Docker -> cadvisor things and maybe you have some trick for this :)