ingensi / dockbeat Goto Github PK

View Code? Open in Web Editor NEW

287.0 287.0 70.0 25.7 MB

Dockbeat - the elastic Beat for docker daemon monitoring

License: Other

Makefile 1.79% Go 96.75% Python 0.82% Shell 0.64%

dockbeat's People

Contributors

Stargazers

Watchers

Forkers

erwanncloarec marminthibaut mrkschan hmalphettes pcallewaert shibaonthemoon jonnybgod kangzhenkang gogo106 douaejeouit rchicoli ozbillwang albamc rbitia daguero zhyu ghoranyi xuxinran mjrsnyder jlecren macbash rquinnlv gmanweb erkules homevive kira8565 syukur91 scotdalton acrees guayeguaye luckydonald dispatchbot bgagnon durmo11 up1 zdtsw phagunbaya ericomsoftwareltd ozlevka-work deivit78 ttrumm comtech-et abhilash-ravindran gdm teazj maron78 msellamitn infomoss lebronjames pmcpsantana robertjustjones edujgurjc rochyjiang alejgh vbuciuc frankz vickybansal patrykk2252 lukehuang char-n mixeur phoenixsn13 jeroenkops akosbalogh005 cnb0 adriellee iq-scm

dockbeat's Issues

Setup template field generation

Put the python template generator program as developped in the packetbeat and topbeat repos:

scripts/generate_field_docs.py
scripts/generate_template.py

Python scripts used to generate the etc/dockerbeat.template.json file.

Per CPU stat is just an array

It's currently impossible to create a chart to visualize each cpu consumption, because the metric 'perCpuUsage' is only an array.

It can be possible to construct a map with the cpu name and its value.

First network stats does not push well named fields

The first network stats after dockerbeat startup is not well formatted:

event = common.MapStr{
            "timestamp":      common.Time(stats.Read),
            "type":           "net",
            "containerID":    container.ID,
            "containerNames": container.Names,
            "net": common.MapStr{
                "rxBytes":   0,
                "rxDropped": 0,
                "rxErrors":  0,
                "rxPackets": 0,
                "txBytes":   0,
                "txDropped": 0,
                "txErrors":  0,
                "txPackets": 0,
            },
        }

Fields should be suffixed by "_ps" to match with the template.

cpu,blkio,network value type conflicts with 1.0.0-beta2

Using Docker image ingensi/dockerbeat:1.0.0-beta2.

Seems the input data has long type, while the index mapping requires double.

logs are:

elasticsearch_1 | [2016-01-28 02:33:45,085][DEBUG][action.admin.indices.mapping.put] [Cloud] failed to put mappings on indices [[dockerbeat-2016.01.28]], typ
e [cpu]
elasticsearch_1 | MergeMappingException[Merge failed with failures {[mapper [cpu.percpuUsage.cpu3] of different type, current_type [double], merged_type [lon
g], mapper [cpu.totalUsage] of different type, current_type [double], merged_type [long]]}]
elasticsearch_1 | at org.elasticsearch.cluster.metadata.MetaDataMappingService$2.execute(MetaDataMappingService.java:392)
elasticsearch_1 | at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:388)
elasticsearch_1 | at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsT
hreadPoolExecutor.java:231)
elasticsearch_1 | at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoo
lExecutor.java:194)
elasticsearch_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
elasticsearch_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
elasticsearch_1 | at java.lang.Thread.run(Thread.java:745)
elasticsearch_1 | [2016-01-28 02:33:45,088][DEBUG][action.bulk ] [Cloud] [dockerbeat-2016.01.28][2] failed to execute bulk item (index) index {[
dockerbeat-2016.01.28][cpu][AVKGFE_njViCxbrU49UJ], source[{"@timestamp":"2016-01-28T02:33:38.179Z","beat":{"hostname":"0802849db28f","name":"0802849db28f"},"
containerID":"21d8b2f4b250012a731ebb68f8458713f492c9247fb58b24da2092302aa0600f","containerName":"dockerbeatek_kibana_1","count":1,"cpu":{"percpuUsage":{"cpu0
":0,"cpu1":0,"cpu2":0,"cpu3":0},"totalUsage":0,"usageInKernelmode":0,"usageInUsermode":0},"type":"cpu"}]}
elasticsearch_1 | MergeMappingException[Merge failed with failures {[mapper [cpu.percpuUsage.cpu3] of different type, current_type [double], merged_type [lon
g], mapper [cpu.totalUsage] of different type, current_type [double], merged_type [long]]}]
elasticsearch_1 | at org.elasticsearch.cluster.metadata.MetaDataMappingService$2.execute(MetaDataMappingService.java:392)
elasticsearch_1 | at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:388)
elasticsearch_1 | at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsT
hreadPoolExecutor.java:231)
elasticsearch_1 | at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoo
lExecutor.java:194)
elasticsearch_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
elasticsearch_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
elasticsearch_1 | at java.lang.Thread.run(Thread.java:745)
elasticsearch_1 | [2016-01-28 02:33:45,098][DEBUG][action.admin.indices.mapping.put] [Cloud] failed to put mappings on indices [[dockerbeat-2016.01.28]], typ
e [blkio]
elasticsearch_1 | MergeMappingException[Merge failed with failures {[mapper [blkio.total_ps] of different type, current_type [long], merged_type [double], ma
pper [blkio.write_ps] of different type, current_type [long], merged_type [double]]}]
elasticsearch_1 | at org.elasticsearch.cluster.metadata.MetaDataMappingService$2.execute(MetaDataMappingService.java:392)
elasticsearch_1 | at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:388)
elasticsearch_1 | at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsT
hreadPoolExecutor.java:231)
elasticsearch_1 | at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoo
lExecutor.java:194)
elasticsearch_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
elasticsearch_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
elasticsearch_1 | at java.lang.Thread.run(Thread.java:745)

Mapping should set fields as "not analyzed"

To avoid splitting field problem :)

see topbeat project

Logstash output?

Didn't find this one in README, are we able to output to logstash?

Manage errors due to a too old version of Docker

Dockerbeat uses docker stats api, which has been introduced from 1.5 version.

Dockerbeat should check docker version on startup and display compatibility error message if docker daemon is not up to date.

Minimal docker version should be updated to 1.9.x

Reference for #71

Dockerbeat should display an error message when docker is too old. Required version is 1.9.x (API version > 1.21)

Container type event should parse status field

Container events have a status field, formatted as string ("Up 36 minutes").

Dockerbeat could parse this string to extract two nested fields:

state ("up", "stopped", etc.)
time (in seconds: "2160")

Fields names are not definitive.

Memory metrics should show a percentage

It could be usefull to have percent of memory used by container, instead of raw values.

CONTRIBUTING.md is missing

Repo should contains a contributing markdown file.

Code refactoring

The app should be tested...

container.ports (container type) is malformed when container does not have any published port

When a monitored container does not publish any port, then the container.ports field take the string value "-". It should be an empty array [].

Change to new Libbeat Github repo

All the beats have merged into one github repos. The Libbeat deps should be changed to that repos.

See https://www.elastic.co/blog/the-beats-are-moving-together-into-single-git-repository for annoucement.
See https://github.com/elastic/beats for new repos

Should Dockerbeat be able to monitor many docker daemons?

The question is: if we have many docker daemons to monitor,

should we have to launch as many dockerbeat as docker daemon
or should we configure one dockerbeat to monitor many docker daemons (socket configuration property could be modified to sockets and be an array of sockets)

This case could append when there is many docker daemons deployed on several machines and listening on TCP. It could be usefull to centralize stats collect on one dedicated host with only one dockerbeat instance, pulling many TCP sockets.

Related to #58 (to identify the docker daemon) and #58 (to allow remote connection to docker daemon).

Add new document type for docker daemon status

Dockerbeat could ship an event type for docker daemon status by pulling the /info endpoint : https://docs.docker.com/engine/reference/api/docker_remote_api_v1.21/#2-3-misc

When API is unreachable, we could ship an event with special status. (need discuss)

Next release?

Hi @marminthibaut

Do you know ETA for next release?

Network type output.

Can't find information about network type in a container (like bridge or host). Is there any support for this function?

Display proper error message when docker daemon is unreachable

When dockerbeat successfully runs and docker daemon does not respond, dockerbeat should properly log error message with logp.

Some message should be displayed:

when dockerbeat cannot access to docker socket
when dockerbeat can access to docker socker, but an error occurred on listing containers
when dockerbeat can access to docker socker, but an error occurred on getting container stats

Errors due to an unreachable elasticsearch node should be managed by libbeat.

Clean code :)

Clean the code:

remove useless log debug
...

What about Dockerbeat and Swarm?

See #59 @gpolaert comment:

I'm wondering if there is a place for swarm and dockerbeat. Swarm can provide some information about nodes, ip, cpu, and so on, plus containers
2 thoughts:

Can we handles beats via swarm/dockerbeat? Is it interesting (not sure)?
Do we have to provide a json swarm event if the socket is a instance of swarm node?

Network stats could be enhanced

Network stats should include a bandwidth instead of Rx/Tx values.

Set some common parameters via environment

Instead of creating custom a config file just to set a non-default elasticsearch port for example, it would be more convenient to be able to set it via the environment variables.
Like this:

docker run -d ... -e HOST=192.168.0.1 -e PORT=9999

See the example of this approach here: https://github.com/athieriot/docker-elastic-topbeat

I suggest to support this list of variables: PERIOD, HOST, PORT, SHIPPER_NAME, SHIPPER_TAGS.

Setup doc field generation

Generate asciidoc from scripts/generate_fields_doc.py script.

Issue starting dockerbeat with ES_HOSTS

Hello,

We are getting an error when starting dockerbeat with the ES_HOSTS parameter in our docker run (our elasticsearch cluster is not running in docker and on other machines, so we use ES_HOSTS):
docker run --net bridge -m 0b --detach=true -e ES_HOSTS=xxx.xxxxxx.xxx.domain.tld -v /var/run/docker.sock:/var/run/docker.sock --name dockerbeat-app00 ingensi/dockerbeat:develop

Start error: loading config file error: YAML config parsing failed on dockerbeat.yml: yaml: unmarshal errors:
line 29: cannot unmarshal !!str xxx-xx... into []string. Exiting.

panic: close of nil channel

goroutine 1 [running]:
github.com/ingensi/dockerbeat/beat.(_Dockerbeat).Stop(0xc82014ae10)
/go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:122 +0x24
github.com/elastic/beats/libbeat/beat.(_Beat).Stop(0xc82014aea0)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:262 +0x77
github.com/elastic/beats/libbeat/beat.Run(0x95e790, 0xa, 0x955710, 0xb, 0x7fe7940a51b8, 0xc82014ae10, 0x0, 0x0)
/go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:127 +0xd0
main.main()
/go/src/github.com/ingensi/dockerbeat/main.go:14 +0xa4

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1696 +0x1

goroutine 5 [syscall]:
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
/usr/local/go/src/os/signal/signal_unix.go:28 +0x37

Kr,
Jarko

Fatal error: concurrent map writes

I've just launched your last development version and in a few minutes it threw following exception:

$ sudo ./dockerbeat
fatal error: concurrent map writes

goroutine 2529 [running]:
runtime.throw(0xa32d20, 0x15)
    /usr/local/go/src/runtime/panic.go:530 +0x90 fp=0xc8203c3b00 sp=0xc8203c3ae8
runtime.mapassign1(0x8459a0, 0xc82018e720, 0xc82008d290, 0xc8203c3d90)
    /usr/local/go/src/runtime/hashmap.go:445 +0xb1 fp=0xc8203c3ba8 sp=0xc8203c3b00
github.com/ingensi/dockerbeat/beat.(*EventGenerator).getBlkioEvent(0xc82008c0d8, 0xc82008d290, 0xc820211b80, 0xc82032ff20)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/EventGenerator.go:225 +0x93f fp=0xc8203c3e28 sp=0xc8203c3ba8
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func2(0xc82034efc0, 0xc82034f080, 0xc82008c090, 0xc82008d290)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:184 +0x1c5 fp=0xc8203c3f90 sp=0xc8203c3e28
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc8203c3f98 sp=0xc8203c3f90
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:197 +0x210

goroutine 1 [chan receive, 22 minutes]:
github.com/elastic/beats/libbeat/beat.Run(0x9e07e0, 0xa, 0x9d6d30, 0xb, 0x7fe11aaa0bd0, 0xc82008c090, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:126 +0xc2
main.main()
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/main.go:14 +0xa5

goroutine 17 [syscall, 22 minutes, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 5 [syscall, 22 minutes]:
os/signal.signal_recv(0x0)
    /usr/local/go/src/runtime/sigqueue.go:116 +0x132
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 34 [select]:
github.com/elastic/beats/libbeat/publisher.(*messageWorker).run(0xc82016e300)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:61 +0x44a
created by github.com/elastic/beats/libbeat/publisher.(*messageWorker).init
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:55 +0x11e

goroutine 7 [select]:
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).Run(0xc82008c090, 0xc82008c120, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:101 +0x4e8
github.com/elastic/beats/libbeat/beat.(*Beat).Run(0xc82008c120, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:250 +0x335
github.com/elastic/beats/libbeat/beat.(*Beat).Start(0xc82008c120, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:162 +0x10f
github.com/elastic/beats/libbeat/beat.Run.func1(0xc82008c120)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:110 +0x42
created by github.com/elastic/beats/libbeat/beat.Run
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:122 +0x8d

goroutine 35 [select]:
github.com/elastic/beats/libbeat/publisher.(*bulkWorker).run(0xc8201d2000)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/bulk.go:57 +0x3cb
created by github.com/elastic/beats/libbeat/publisher.newBulkWorker
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/bulk.go:41 +0x20a

goroutine 36 [select, 22 minutes, locked to thread]:
runtime.gopark(0xacf2b8, 0xc8201d8f28, 0x9d2b78, 0x6, 0x18, 0x2)
    /usr/local/go/src/runtime/proc.go:262 +0x163
runtime.selectgoImpl(0xc8201d8f28, 0x0, 0x18)
    /usr/local/go/src/runtime/select.go:392 +0xa67
runtime.selectgo(0xc8201d8f28)
    /usr/local/go/src/runtime/select.go:215 +0x12
runtime.ensureSigM.func1()
    /usr/local/go/src/runtime/signal1_unix.go:279 +0x358
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 37 [chan receive, 22 minutes]:
github.com/elastic/beats/libbeat/service.HandleSignals.func1(0xc8201a30e0, 0xc82018cb40, 0xc82018cb30)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/service/service.go:29 +0x3a
created by github.com/elastic/beats/libbeat/service.HandleSignals
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/service/service.go:32 +0x21e

goroutine 133 [select]:
net/http.(*persistConn).writeLoop(0xc8202ac1a0)
    /usr/local/go/src/net/http/transport.go:1273 +0x472
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:854 +0x10cb

goroutine 132 [IO wait]:
net.runtime_pollWait(0x7fe11aaa1478, 0x72, 0xc820229000)
    /usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc820054300, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820054300, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8200542a0, 0xc820229000, 0x1000, 0x1000, 0x0, 0x7fe11aa9c028, 0xc82000a120)
    /usr/local/go/src/net/fd_unix.go:250 +0x23a
net.(*conn).Read(0xc820028020, 0xc820229000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7fe11aa202c0, 0xc820028020, 0xc8202ac208, 0xc820229000, 0x1000, 0x1000, 0x404df3, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1683 +0x67
net/http.(*noteEOFReader).Read(0xc820168ac0, 0xc820229000, 0x1000, 0x1000, 0xc8203c7d1d, 0x0, 0x0)
    <autogenerated>:284 +0xd0
bufio.(*Reader).fill(0xc820184420)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc820184420, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8202ac1a0)
    /usr/local/go/src/net/http/transport.go:1069 +0x177
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:853 +0x10a6

goroutine 1875 [select]:
net/http.(*persistConn).writeLoop(0xc820246270)
    /usr/local/go/src/net/http/transport.go:1273 +0x472
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:854 +0x10cb

goroutine 1874 [IO wait]:
net.runtime_pollWait(0x7fe11aaa1538, 0x72, 0xc820295000)
    /usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc820121fe0, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820121fe0, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820121f80, 0xc820295000, 0x1000, 0x1000, 0x0, 0x7fe11aa9c028, 0xc82000a120)
    /usr/local/go/src/net/fd_unix.go:250 +0x23a
net.(*conn).Read(0xc82025eb40, 0xc820295000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7fe11aa5c000, 0xc82025eb40, 0xc8202462d8, 0xc820295000, 0x1000, 0x1000, 0x404df3, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1683 +0x67
net/http.(*noteEOFReader).Read(0xc820446960, 0xc820295000, 0x1000, 0x1000, 0xc820335d1d, 0x0, 0x0)
    <autogenerated>:284 +0xd0
bufio.(*Reader).fill(0xc820263d40)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc820263d40, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc820246270)
    /usr/local/go/src/net/http/transport.go:1069 +0x177
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:853 +0x10a6
[cloud-user@mi-worker-001 dockerbeat]$ sudo ./dockerbeat
fatal error: concurrent map writes

goroutine 924 [running]:
runtime.throw(0xa32d20, 0x15)
    /usr/local/go/src/runtime/panic.go:530 +0x90 fp=0xc820247b00 sp=0xc820247ae8
runtime.mapassign1(0x8459a0, 0xc820061f50, 0xc8202cefc0, 0xc820247d90)
    /usr/local/go/src/runtime/hashmap.go:445 +0xb1 fp=0xc820247ba8 sp=0xc820247b00
github.com/ingensi/dockerbeat/beat.(*EventGenerator).getBlkioEvent(0xc820176048, 0xc8202cefc0, 0xc82030eb00, 0xc8202ca390)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/EventGenerator.go:225 +0x93f fp=0xc820247e28 sp=0xc820247ba8
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func2(0xc8202686c0, 0xc820268780, 0xc820176000, 0xc8202cefc0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:184 +0x1c5 fp=0xc820247f90 sp=0xc820247e28
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc820247f98 sp=0xc820247f90
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:197 +0x210

goroutine 1 [chan receive, 7 minutes]:
github.com/elastic/beats/libbeat/beat.Run(0x9e07e0, 0xa, 0x9d6d30, 0xb, 0x7f7ac2b291e0, 0xc820176000, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:126 +0xc2
main.main()
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/main.go:14 +0xa5

goroutine 17 [syscall, 7 minutes, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 5 [syscall, 7 minutes]:
os/signal.signal_recv(0x0)
    /usr/local/go/src/runtime/sigqueue.go:116 +0x132
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 18 [select, 1 minutes]:
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).Run(0xc820176000, 0xc820176090, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:101 +0x4e8
github.com/elastic/beats/libbeat/beat.(*Beat).Run(0xc820176090, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:250 +0x335
github.com/elastic/beats/libbeat/beat.(*Beat).Start(0xc820176090, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:162 +0x10f
github.com/elastic/beats/libbeat/beat.Run.func1(0xc820176090)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:110 +0x42
created by github.com/elastic/beats/libbeat/beat.Run
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:122 +0x8d

goroutine 22 [IO wait, 1 minutes]:
net.runtime_pollWait(0x7f7ac2b29cc8, 0x72, 0xc8201c4000)
    /usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc820147f70, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820147f70, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820147f10, 0xc8201c4000, 0x1000, 0x1000, 0x0, 0x7f7ac2ba9028, 0xc82000a120)
    /usr/local/go/src/net/fd_unix.go:250 +0x23a
net.(*conn).Read(0xc82013c0e8, 0xc8201c4000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f7ac2b2a608, 0xc82013c0e8, 0xc820145cd8, 0xc8201c4000, 0x1000, 0x1000, 0x404df3, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1683 +0x67
net/http.(*noteEOFReader).Read(0xc8201c20a0, 0xc8201c4000, 0x1000, 0x1000, 0xc820034d1d, 0x0, 0x0)
    <autogenerated>:284 +0xd0
bufio.(*Reader).fill(0xc820143740)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc820143740, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc820145c70)
    /usr/local/go/src/net/http/transport.go:1069 +0x177
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:853 +0x10a6

goroutine 23 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc820145c70)
    /usr/local/go/src/net/http/transport.go:1273 +0x472
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:854 +0x10cb

goroutine 7 [select, 1 minutes]:
github.com/elastic/beats/libbeat/publisher.(*messageWorker).run(0xc820001b00)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:61 +0x44a
created by github.com/elastic/beats/libbeat/publisher.(*messageWorker).init
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:55 +0x11e

goroutine 8 [runnable]:
github.com/elastic/beats/libbeat/publisher.(*bulkWorker).onEvents(0xc820054d20, 0x0, 0x0, 0xc8202be4a0, 0x4, 0x4)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/bulk.go:83
github.com/elastic/beats/libbeat/publisher.(*bulkWorker).run(0xc820054d20)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/bulk.go:63 +0x352
created by github.com/elastic/beats/libbeat/publisher.newBulkWorker
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/bulk.go:41 +0x20a

goroutine 9 [select, 7 minutes, locked to thread]:
runtime.gopark(0xacf2b8, 0xc820027f28, 0x9d2b78, 0x6, 0x18, 0x2)
    /usr/local/go/src/runtime/proc.go:262 +0x163
runtime.selectgoImpl(0xc820027f28, 0x0, 0x18)
    /usr/local/go/src/runtime/select.go:392 +0xa67
runtime.selectgo(0xc820027f28)
    /usr/local/go/src/runtime/select.go:215 +0x12
runtime.ensureSigM.func1()
    /usr/local/go/src/runtime/signal1_unix.go:279 +0x358
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 10 [chan receive, 7 minutes]:
github.com/elastic/beats/libbeat/service.HandleSignals.func1(0xc82005f3e0, 0xc82021a250, 0xc82021a240)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/service/service.go:29 +0x3a
created by github.com/elastic/beats/libbeat/service.HandleSignals
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/service/service.go:32 +0x21e

goroutine 79 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc8202540d0)
    /usr/local/go/src/net/http/transport.go:1273 +0x472
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:854 +0x10cb

goroutine 78 [IO wait, 1 minutes]:
net.runtime_pollWait(0x7f7ac2b29c08, 0x72, 0xc8204ba000)
    /usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc8204a2300, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc8204a2300, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8204a22a0, 0xc8204ba000, 0x1000, 0x1000, 0x0, 0x7f7ac2ba9028, 0xc82000a120)
    /usr/local/go/src/net/fd_unix.go:250 +0x23a
net.(*conn).Read(0xc820028658, 0xc8204ba000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7f7ac2bb1c20, 0xc820028658, 0xc820254138, 0xc8204ba000, 0x1000, 0x1000, 0x404df3, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1683 +0x67
net/http.(*noteEOFReader).Read(0xc8200c08c0, 0xc8204ba000, 0x1000, 0x1000, 0xc820256d1d, 0x0, 0x0)
    <autogenerated>:284 +0xd0
bufio.(*Reader).fill(0xc82043f860)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc82043f860, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8202540d0)
    /usr/local/go/src/net/http/transport.go:1069 +0x177
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:853 +0x10a6
[cloud-user@mi-worker-001 dockerbeat]$ sudo ./dockerbeat
fatal error: concurrent map writes

goroutine 1323 [running]:
runtime.throw(0xa32d20, 0x15)
    /usr/local/go/src/runtime/panic.go:530 +0x90 fp=0xc820269b00 sp=0xc820269ae8
runtime.mapassign1(0x8459a0, 0xc8201c0720, 0xc82017e750, 0xc820269d90)
    /usr/local/go/src/runtime/hashmap.go:445 +0xb1 fp=0xc820269ba8 sp=0xc820269b00
github.com/ingensi/dockerbeat/beat.(*EventGenerator).getBlkioEvent(0xc82016c048, 0xc82017e750, 0xc8201a18c0, 0xc82012e9f0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/EventGenerator.go:225 +0x93f fp=0xc820269e28 sp=0xc820269ba8
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats.func2(0xc820523f80, 0xc8200da060, 0xc82016c000, 0xc82017e750)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:184 +0x1c5 fp=0xc820269f90 sp=0xc820269e28
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc820269f98 sp=0xc820269f90
created by github.com/ingensi/dockerbeat/beat.(*Dockerbeat).exportContainerStats
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:197 +0x210

goroutine 1 [chan receive, 11 minutes]:
github.com/elastic/beats/libbeat/beat.Run(0x9e07e0, 0xa, 0x9d6d30, 0xb, 0x7fe48f96f150, 0xc82016c000, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:126 +0xc2
main.main()
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/main.go:14 +0xa5

goroutine 17 [syscall, 11 minutes, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 5 [syscall, 11 minutes]:
os/signal.signal_recv(0x0)
    /usr/local/go/src/runtime/sigqueue.go:116 +0x132
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
    /usr/local/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 34 [select]:
github.com/elastic/beats/libbeat/publisher.(*messageWorker).run(0xc820060300)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:61 +0x44a
created by github.com/elastic/beats/libbeat/publisher.(*messageWorker).init
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/worker.go:55 +0x11e

goroutine 18 [select]:
github.com/ingensi/dockerbeat/beat.(*Dockerbeat).Run(0xc82016c000, 0xc82016c090, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/beat/Dockerbeat.go:101 +0x4e8
github.com/elastic/beats/libbeat/beat.(*Beat).Run(0xc82016c090, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:250 +0x335
github.com/elastic/beats/libbeat/beat.(*Beat).Start(0xc82016c090, 0x0, 0x0)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:162 +0x10f
github.com/elastic/beats/libbeat/beat.Run.func1(0xc82016c090)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:110 +0x42
created by github.com/elastic/beats/libbeat/beat.Run
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/beat/beat.go:122 +0x8d

goroutine 35 [runnable]:
github.com/elastic/beats/libbeat/publisher.(*bulkWorker).run(0xc820204000)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/bulk.go:57 +0x134
created by github.com/elastic/beats/libbeat/publisher.newBulkWorker
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/publisher/bulk.go:41 +0x20a

goroutine 36 [select, 11 minutes, locked to thread]:
runtime.gopark(0xacf2b8, 0xc82020af28, 0x9d2b78, 0x6, 0x18, 0x2)
    /usr/local/go/src/runtime/proc.go:262 +0x163
runtime.selectgoImpl(0xc82020af28, 0x0, 0x18)
    /usr/local/go/src/runtime/select.go:392 +0xa67
runtime.selectgo(0xc82020af28)
    /usr/local/go/src/runtime/select.go:215 +0x12
runtime.ensureSigM.func1()
    /usr/local/go/src/runtime/signal1_unix.go:279 +0x358
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1998 +0x1

goroutine 37 [chan receive, 11 minutes]:
github.com/elastic/beats/libbeat/service.HandleSignals.func1(0xc8201d50e0, 0xc8201beb40, 0xc8201beb30)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/service/service.go:29 +0x3a
created by github.com/elastic/beats/libbeat/service.HandleSignals
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/elastic/beats/libbeat/service/service.go:32 +0x21e

goroutine 123 [select]:
net/http.(*persistConn).writeLoop(0xc8204d40d0)
    /usr/local/go/src/net/http/transport.go:1273 +0x472
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:854 +0x10cb

goroutine 122 [IO wait]:
net.runtime_pollWait(0x7fe48f96fb78, 0x72, 0xc82015a000)
    /usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc820054290, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820054290, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc820054230, 0xc82015a000, 0x1000, 0x1000, 0x0, 0x7fe48f9af028, 0xc82000a120)
    /usr/local/go/src/net/fd_unix.go:250 +0x23a
net.(*conn).Read(0xc8202de000, 0xc82015a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7fe48f9706c8, 0xc8202de000, 0xc8204d4138, 0xc82015a000, 0x1000, 0x1000, 0x404df3, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1683 +0x67
net/http.(*noteEOFReader).Read(0xc8201b8280, 0xc82015a000, 0x1000, 0x1000, 0xc82026ad1d, 0x0, 0x0)
    <autogenerated>:284 +0xd0
bufio.(*Reader).fill(0xc8201d44e0)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8201d44e0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8204d40d0)
    /usr/local/go/src/net/http/transport.go:1069 +0x177
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:853 +0x10a6

goroutine 941 [select]:
net/http.(*persistConn).writeLoop(0xc8204d41a0)
    /usr/local/go/src/net/http/transport.go:1273 +0x472
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:854 +0x10cb

goroutine 940 [IO wait]:
net.runtime_pollWait(0x7fe48f96fc38, 0x72, 0xc8202cd000)
    /usr/local/go/src/runtime/netpoll.go:160 +0x60
net.(*pollDesc).Wait(0xc820205800, 0x72, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc820205800, 0x0, 0x0)
    /usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8202057a0, 0xc8202cd000, 0x1000, 0x1000, 0x0, 0x7fe48f9af028, 0xc82000a120)
    /usr/local/go/src/net/fd_unix.go:250 +0x23a
net.(*conn).Read(0xc8202df930, 0xc8202cd000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:172 +0xe4
net/http.noteEOFReader.Read(0x7fe48f970578, 0xc8202df930, 0xc8204d4208, 0xc8202cd000, 0x1000, 0x1000, 0x404df3, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1683 +0x67
net/http.(*noteEOFReader).Read(0xc8202d1020, 0xc8202cd000, 0x1000, 0x1000, 0xc8203cfd1d, 0x0, 0x0)
    <autogenerated>:284 +0xd0
bufio.(*Reader).fill(0xc82040be00)
    /usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc82040be00, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8204d41a0)
    /usr/local/go/src/net/http/transport.go:1069 +0x177
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:853 +0x10a6

goroutine 1339 [runnable]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc8201ad300, 0x40, 0xc8200da0c0, 0xc8200da000, 0xc8200da120, 0xffffffffffffffff, 0xc8202de670, 0xc82017cc00)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe4
created by github.com/fsouza/go-dockerclient.(*Client).Stats
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7

goroutine 1327 [runnable]:
github.com/fsouza/go-dockerclient.(*Client).Stats.func3(0xc8201ad240, 0x40, 0xc820523f80, 0xc820523f00, 0xc8200da000, 0xffffffffffffffff, 0xc820134cd8, 0xc8200da240)
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:798 +0xe4
created by github.com/fsouza/go-dockerclient.(*Client).Stats
    /home/cloud-user/.go/src/github.com/ingensi/dockerbeat/Godeps/_workspace/src/github.com/fsouza/go-dockerclient/container.go:804 +0x3a7

Identify container with human readable names

It will allow to understand more easily kibana dashboards...

Wrong cpu type event in README file

Found this line in README:
type: cpu for per process container statistics.
but example type: cpu output don't have anything related to processes

Can you clarify this?
Thanks

Percentage values should be set between 0 and 1

Currently, percentage values are set between 0 and 100, but kibana field format only supports percentages between 0 and 1 out of the box...

CPU stats should show load percentage

Dockerbeat should send cpu load (percentage) instead of raw values (usages).

Should dockerbeat allow to monitor container logs?

The question is: Does the dockerbeat role is to monitor only container statistics or should also ship container logs ?

Gather and/or duplicate metrics container

We have to choose between:

gather all statistics (cpu, blkio, container, etc.) into a unique and big event
remove the container event, and duplicate container information to all other metrics

We should compare to topbeat and packetbeat to see their logics.

Sample Kibana dashboards

It would be very useful if you could provide some Kibana templates for dockerbeat.
Like Elastic does for topbeat: https://github.com/elastic/beats-dashboards

Blkio stats should gather "per second" statistics

Currently, dockerbeat sends blkio data as number of reads/writes/total minus previous tick data. It should divides by the number of seconds to emit a "per second" data (like network and proc data).

Add blkio stats

Dockerbeat should collect blkio statistics. Stats should be human-readable i.e should be read/write Bytes per second.

Sanitize docker labels containing dots

Docker-compose generates some container labels like 'com.docker.compose.container-number' and 'com.docker.compose.project' among others. These are invalid field names since ES 2.0 does not support dot in field names. It would be nice if those names are sanitized using '_' or any other valid character.

I checked docker-compose documentation and source and I wasn't able to find a way to disable that label generation.

Docker version constraint

Currently, development version has the constraint for docker version upper than 1.9. What is the reason of this? I've tried to run it without this constraint and everything works fine.

Doc & release 1.0.0-beta1

Release the 1.0.0-beta1 dockerbeat version

Code comments

Code should includes some comments

Connection to Docker via TCP/TLS

Dockerbeat only allows connection to docker daemon socket. It should also be able to connect to listening TCP daemon.

Improve README content

add a TLDR
rewrite some parts
etc.

panic after 30 minutes running

Hi,

We are trying to run develop branch on our servers, but are getting after 30 minutes panics, and the container stops. Seems a deadlock situation..

We are running this command:
docker run -d -v /var/run/docker.sock:/var/run/docker.sock --link elasticsearch-001:elasticsearch ingensi/dockerbeat:develop

docker version:

Client:
Version: 1.9.1
API version: 1.21
Go version: go1.4.2
Git commit: a34a1d5
Built: Fri Nov 20 13:12:04 UTC 2015
OS/Arch: linux/amd64
Server:
Version: 1.9.1
API version: 1.21
Go version: go1.4.2
Git commit: a34a1d5
Built: Fri Nov 20 13:12:04 UTC 2015
OS/Arch: linux/amd64[/quote]

Attached you find the error log
dockerbeat-fail.zip

Percentage fields should be between 0.0 and 1.0

Currently, percentage fields are between 0.0 and 100.0, to use kibana field formatting properly, it should be between 0.0 and 1.0.

Add dockerbeat to AUR repo

Could we push dockerbeat to AUR repo (for Arch distrib) like topbeat, filebeat and so on ?

Dependency (go-dockerclient) has been updated

The go-dockerclient project have change for the new Docker API (1.22).

The Stat object have change : see here

The impact on Dockerbeat is that it changes the event struct.

Add a "dockerSocket" field

Add the dockerSocket field to all shipped events. It should be usefull when many dockerbeat are launched on the same host to monitor different dockers daemons.

Mapping volume to config directory causes executable to be lost

Hey, I am just getting started with dockerbeat, but I like the way it looks. Ran into one issue right off the bat. In your documentation, it says that to map a custom config file simply mount a volume:

-v /src/beats/dockerbeat/config/:/etc/dockerbeat

However, when you do that, you simply overwrite the executable, and then it cannot launch. Not a huge ordeal, I can simply use COPY in the Dockerfile or change the shared directory and update the entrypoint. Just thought I would let you all know. Thanks

Report Docker label

Hi all,

I would like to use dockerbeat in a multitenant environement.
Do you plan to return Label keys (https://docs.docker.com/engine/userguide/labels-custom-metadata/) ?

That's will be helpfull to differentiate customers

Best regards,
Hug

No network stats

We are running dockerbeat v1.0.0-beta2, and we've found that there are no network statistics, and no any error message.
logs with debug level: https://gist.github.com/hcoa/335c1c26ec011adc4394
docker api version 1.20

Dockerbeat crashes when docker API is not reachable

Related issue: #60

Reproduce: launch dockerbeat monitoring a running docker daemon and then stop the docker daemon.

Solve problem: check nil values in EventGenerator

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.