docker-gen is a file generator that renders templates using docker container meta-data.
It can be used to generate various kinds of files for:
- Centralized logging - fluentd, logstash or other centralized logging tools that tail the containers JSON log file or files within the container.
- Log Rotation - logrotate files to rotate container JSON log files
- Reverse Proxy Configs - nginx, haproxy, etc. reverse proxy configs to route requests from the host to containers
- Service Discovery - Scripts (python, bash, etc..) to register containers within etcd, hipache, etc..
There are three common ways to run docker-gen:
- on the host
- bundled in a container with another application
- separate standalone containers
Download the version you need, untar, and install to your PATH.
wget https://github.com/nginx-proxy/docker-gen/releases/download/0.12.0/docker-gen-linux-amd64-0.12.0.tar.gz
tar xvzf docker-gen-linux-amd64-0.12.0.tar.gz
./docker-genDocker-gen can be bundled inside of a container along-side applications.
nginx-proxy/nginx-proxy trusted build is an example of running docker-gen within a container along-side nginx. jwilder/docker-register is an example of running docker-gen within a container to do service registration with etcd.
It can also be run as two separate containers using the nginx-proxy/docker-gen image, together with virtually any other image.
This is how you could run the official nginx image and
have docker-gen generate a reverse proxy config in the same way that nginx-proxy works. You may want to do
this to prevent having the docker socket bound to a publicly exposed container service.
Start nginx with a shared volume:
docker run -d -p 80:80 --name nginx -v /tmp/nginx:/etc/nginx/conf.d -t nginxFetch the template and start the docker-gen container with the shared volume:
mkdir -p /tmp/templates && cd /tmp/templates
curl -o nginx.tmpl https://raw.githubusercontent.com/nginx-proxy/docker-gen/main/templates/nginx.tmpl
docker run -d --name nginx-gen --volumes-from nginx \
-v /var/run/docker.sock:/tmp/docker.sock:rw \
-v /tmp/templates:/etc/docker-gen/templates \
-t nginxproxy/docker-gen -notify-sighup nginx -watch -only-exposed /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.confStart a container, taking note of any Environment variables a container expects. See the top of a template for details.
docker run --env VIRTUAL_HOST='example.com' --env VIRTUAL_PORT=80 ...$ docker-gen
Usage: docker-gen [options] template [dest]
Generate files from docker container meta-data
Options:
-config value
config files with template directives. Config files will be merged if this option is specified multiple times. (default [])
-endpoint string
docker api endpoint (tcp|unix://..). Default unix:///var/run/docker.sock
-interval int
notify command interval (secs)
-keep-blank-lines
keep blank lines in the output file
-notify restart xyz
run command after template is regenerated (e.g restart xyz)
-notify-output
log the output(stdout/stderr) of notify command
-notify-sighup container-ID
send HUP signal to container.
Equivalent to 'docker kill -s HUP container-ID', or `-notify-container container-ID -notify-signal 1`.
You can pass this option multiple times to send HUP to multiple containers.
-notify-container container-ID
send -notify-signal signal (defaults to 1 / HUP) to container.
You can pass this option multiple times to notify multiple containers.
-notify-filter key=value
container filter for notification (e.g -notify-filter name=foo).
You can pass this option multiple times to combine filters with AND.
https://docs.docker.com/engine/reference/commandline/ps/#filter
-notify-signal signal
signal to send to the -notify-container and -notify-filter. -1 to call docker restart. Defaults to 1 aka. HUP.
All available signals available on the dockerclient
https://github.com/fsouza/go-dockerclient/blob/main/signal.go
-only-exposed
only include containers with exposed ports
-only-published
only include containers with published ports (implies -only-exposed)
-include-stopped
include stopped containers
-tlscacert string
path to TLS CA certificate file (default "~/.docker/machine/machines/default/ca.pem")
-tlscert string
path to TLS client certificate file (default "~/.docker/machine/machines/default/cert.pem")
-tlskey string
path to TLS client key file (default "~/.docker/machine/machines/default/key.pem")
-tlsverify
verify docker daemon's TLS certicate (default true)
-version
show version
-watch
watch for container changes
-wait
minimum (and/or maximum) duration to wait after each container change before triggering
Arguments:
template - path to a template to generate
dest - path to write the template. If not specfied, STDOUT is used
Environment Variables:
DOCKER_HOST - default value for -endpoint
DOCKER_CERT_PATH - directory path containing key.pem, cert.pm and ca.pem
DOCKER_TLS_VERIFY - enable client TLS verification]
If no <dest> file is specified, the output is sent to stdout. Mainly useful for debugging.
Using the -config flag from above you can tell docker-gen to use the specified config file instead of command-line options. Multiple templates can be defined and they will be executed in the order that they appear in the config file.
An example configuration file, docker-gen.cfg can be found in the examples folder.
[[config]]
# Starts a configuration section
dest = "path/to/a/file"
# path to write the template. If not specfied, STDOUT is used
notifycmd = "/etc/init.d/foo reload"
# run command after template is regenerated (e.g restart xyz)
onlyexposed = true
# only include containers with exposed ports
template = "/path/to/a/template/file.tmpl"
# path to a template to generate
watch = true
# watch for container changes
wait = "500ms:2s"
# debounce changes with a min:max duration. Only applicable if watch = true
[config.NotifyContainers]
# Starts a notify container section
containername = 1
# container name followed by the signal to send
container_id = 1
# or the container id can be used followed by the signal to sendPutting it all together here is an example configuration file.
[[config]]
template = "/etc/nginx/nginx.conf.tmpl"
dest = "/etc/nginx/sites-available/default"
onlyexposed = true
notifycmd = "/etc/init.d/nginx reload"
[[config]]
template = "/etc/logrotate.conf.tmpl"
dest = "/etc/logrotate.d/docker"
watch = true
[[config]]
template = "/etc/docker-gen/templates/nginx.tmpl"
dest = "/etc/nginx/conf.d/default.conf"
watch = true
wait = "500ms:2s"
[config.NotifyContainers]
nginx = 1 # 1 is a signal number to be sent; here SIGHUP
e75a60548dc9 = 1 # a key can be either container name (nginx) or IDThe templates used by docker-gen are written using the Go text/template language. In addition to the built-in functions supplied by Go, docker-gen uses sprig and some additional functions to make it simpler (or possible) to generate your desired output. Some templates rely on environment variables within the container to make decisions on what to generate from the template.
Several templates may be parsed at once by using a semicolon (;) to delimit the template value. This can be used as a proxy for Golang's nested template functionality. In all cases, the main rendered template should go first.
[[config]]
template = "/etc/docker-gen/templates/nginx.tmpl;/etc/docker-gen/templates/header.tmpl"
dest = "/etc/nginx/conf.d/default.conf"
watch = true
wait = "500ms:2s"
Within the templates, the object emitted by docker-gen will be a structure consisting of following Go structs:
type RuntimeContainer struct {
ID string
Created time.Time
Addresses []Address
Networks []Network
Gateway string
Name string
Hostname string
Image DockerImage
Env map[string]string
Volumes map[string]Volume
Node SwarmNode
Labels map[string]string
IP string
IP6LinkLocal string
IP6Global string
Mounts []Mount
State State
}
type Address struct {
IP string
IP6LinkLocal string
IP6Global string
Port string
HostPort string
Proto string
HostIP string
}
type Network struct {
IP string
Name string
Gateway string
EndpointID string
IPv6Gateway string
GlobalIPv6Address string
MacAddress string
GlobalIPv6PrefixLen int
IPPrefixLen int
Internal bool
}
type DockerImage struct {
Registry string
Repository string
Tag string
}
type Mount struct {
Name string
Source string
Destination string
Driver string
Mode string
RW bool
}
type Volume struct {
Path string
HostPath string
ReadWrite bool
}
type SwarmNode struct {
ID string
Name string
Address Address
}
type State struct {
Running bool
Health Health
}
type Health struct {
Status string
}
// Accessible from the root in templates as .Docker
type Docker struct {
Name string
NumContainers int
NumImages int
Version string
ApiVersion string
GoVersion string
OperatingSystem string
Architecture string
CurrentContainerID string
}
// Host environment variables accessible from root in templates as .EnvFor example, this is a JSON version of an emitted RuntimeContainer struct:
{
"ID": "71e9768075836eb38557adcfc71a207386a0c597dbeda240cf905df79b18cebf",
"Addresses": [
{
"IP": "172.17.0.4",
"Port": "22",
"Proto": "tcp",
"HostIP": "192.168.10.24",
"HostPort": "2222"
}
],
"Gateway": "172.17.42.1",
"Node": {
"ID": "I2VY:P7PF:TZD5:PGWB:QTI7:QDSP:C5UD:DYKR:XKKK:TRG2:M2BL:DFUN",
"Name": "docker-test",
"Address": {
"IP": "192.168.10.24"
}
},
"Labels": {
"operatingsystem": "Ubuntu 14.04.2 LTS",
"storagedriver": "devicemapper",
"anything_foo": "something_bar"
},
"IP": "172.17.0.4",
"Name": "docker_register",
"Hostname": "71e976807583",
"Image": {
"Registry": "jwilder",
"Repository": "docker-register"
},
"Env": {
"ETCD_HOST": "172.17.42.1:4001",
"PATH": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"DOCKER_HOST": "unix:///var/run/docker.sock",
"HOST_IP": "172.17.42.1"
},
"Volumes": {
"/mnt": {
"Path": "/mnt",
"HostPath": "/Users/joebob/tmp",
"ReadWrite": true
}
}
}- Functions from Go
- Functions from Sprig v3, except for those that have the same name as one of the following functions.
closest $array $value: Returns the longest matching substring in$arraythat matches$valuecoalesce ...: Returns the first non-nil argument.comment $delimiter $string: Returns$stringwith each line prefixed by$delimiter(helpful for debugging combined with SprigtoPrettyJson:{{ toPrettyJson $ | comment "#" }}).contains $map $key: Returnstrueif$mapcontains$key. Takes maps fromstringto any type.dir $path: Returns an array of filenames in the specified$path.exists $path: Returnstrueif$pathrefers to an existing file or directory. Takes a string.eval $templateName [$data]: Evaluates the named template like Go's built-intemplateaction, but instead of writing out the result it returns the result as a string so that it can be post-processed. The$dataargument may be omitted, which is equivalent to passingnil.groupBy $containers $fieldPath: Groups an array ofRuntimeContainerinstances based on the values of a field path expression$fieldPath. A field path expression is a dot-delimited list of map keys or struct member names specifying the path from container to a nested value, which must be a string. Returns a map from the value of the field path expression to an array of containers having that value. Containers that do not have a value for the field path in question are omitted.groupByWithDefault $containers $fieldPath $defaultValue: Returns the same asgroupBy, but containers that do not have a value for the field path are instead included in the map under the$defaultValuekey.groupByKeys $containers $fieldPath: Returns the same asgroupBybut only returns the keys of the map.groupByMulti $containers $fieldPath $sep: LikegroupBy, but the string value specified by$fieldPathis first split by$sepinto a list of strings. A container whose$fieldPathvalue contains a list of strings will show up in the map output under each of those strings.groupByLabel $containers $label: Returns the same asgroupBybut grouping by the given label's value. Containers that do not have the$labelset are omitted.groupByLabelWithDefault $containers $label $defaultValue: Returns the same asgroupBybut grouping by the given label's value. Containers that do not have the$labelset are included in the map under the$defaultValuekey.include $file: Returns content of$file, and empty string if file reading error.intersect $slice1 $slice2: Returns the strings that exist in both string slices.fromYaml $string/mustFromYaml $string: Similar to Sprig'sfromJson/mustFromJson, but for YAML.toYaml $dict/mustToYaml $dict: Similar to Sprig'stoJson/mustToJson, but for YAML.keys $map: Returns the keys from$map. If$mapisnil, anilis returned. If$mapis not amap, an error will be thrown.sortStringsAsc $strings: Returns a slice of strings$stringssorted in ascending order.sortStringsDesc $strings: Returns a slice of strings$stringssorted in descending (reverse) order.sortObjectsByKeysAsc $objects $fieldPath: Returns the array$objects, sorted in ascending order based on the values of a field path expression$fieldPath.sortObjectsByKeysDesc $objects $fieldPath: Returns the array$objects, sorted in descending (reverse) order based on the values of a field path expression$fieldPath.when $condition $trueValue $falseValue: Returns the$trueValuewhen the$conditionistrueand the$falseValueotherwisewhere $items $fieldPath $value: Filters an array or slice based on the values of a field path expression$fieldPath. A field path expression is a dot-delimited list of map keys or struct member names specifying the path from container to a nested value. Returns an array of items having that value.whereNot $items $fieldPath $value: Filters an array or slice based on the values of a field path expression$fieldPath. A field path expression is a dot-delimited list of map keys or struct member names specifying the path from container to a nested value. Returns an array of items not having that value.whereExist $items $fieldPath: Likewhere, but returns only items where$fieldPathexists (is not nil).whereNotExist $items $fieldPath: Likewhere, but returns only items where$fieldPathdoes not exist (is nil).whereAny $items $fieldPath $sep $values: Likewhere, but the string value specified by$fieldPathis first split by$sepinto a list of strings. The comparison value is a string slice with possible matches. Returns items which OR intersect these values.whereAll $items $fieldPath $sep $values: LikewhereAny, except all$valuesmust exist in the$fieldPath.whereLabelExists $containers $label: Filters a slice of containers based on the existence of the label$label.whereLabelDoesNotExist $containers $label: Filters a slice of containers based on the non-existence of the label$label.whereLabelValueMatches $containers $label $pattern: Filters a slice of containers based on the existence of the label$labelwith values matching the regular expression$pattern.
Sprig functions that have the same name as docker-gen function (but different behaviour) are made available with the sprig prefix:
sprigCoalesce ...: Alias for Sprig'scoalesce.sprigContains $string $string: Alias for Sprig'scontains.sprigDir $path: Alias for Sprig'sdir.sprigReplace $old $new $string: Alias for Sprig'sreplace.sprigSplit $sep $string: Alias for Sprig'ssplit.sprigSplitn $sep $count $string": Alias for Sprig'ssplitn.
Some functions are aliases for Go's strings package functions:
parseBool $string: Alias forstrconv.ParseBool. Returns the boolean value represented by$string. It accepts 1, t, T, TRUE, true, True, 0, f, F, FALSE, false, False. Any other value returns an error.replace $string $old $new $count: Alias forstrings.Replace. Replaces up to$countoccurences of$oldwith$newin$string.split $string $sep: Alias forstrings.Split. Splits$stringinto a slice of substrings delimited by$sep.splitN $string $sep $count: Alias forstrings.SplitN. Splits$stringinto a slice of substrings delimited by$sep, with number of substrings returned determined by$count.toLower $string: Alias forstrings.ToLower. Replace capital letters in$stringto lowercase.toUpper $string: Alias forstrings.ToUpper. Replace lowercase letters in$stringto uppercase.
Those have been aliased to Sprig functions with the same behaviour as the original docker-gen function:
json $value: Alias for Sprig'smustToJsonparseJson $string: Alias for Sprig'smustFromJson.sha1 $string: Alias for Sprig'ssha1sum.
- Automated Nginx Reverse Proxy for Docker
- Docker Log Management With Fluentd
- Docker Service Discovery Using Etcd and Haproxy
nginxproxy/nginx-proxy trusted build.
Start nginx-proxy:
docker run -d -p 80:80 -v /var/run/docker.sock:/tmp/docker.sock -t nginxproxy/nginx-proxyThen start containers with a VIRTUAL_HOST (and the VIRTUAL_PORT if more than one port is exposed) env variable:
docker run -e VIRTUAL_HOST=foo.bar.com -e VIRTUAL_PORT=80 -t ...If you wanted to run docker-gen directly on the host, you could do it with:
docker-gen -only-published -watch -notify "/etc/init.d/nginx reload" templates/nginx.tmpl /etc/nginx/sites-enabled/defaultThis template generate a fluentd.conf file used by fluentd. It would then ship log files off the host.
docker-gen -watch -notify "restart fluentd" templates/fluentd.tmpl /etc/fluent/fluent.confThis template is an example of generating a script that is then executed. This template generates a python script that is then executed which register containers in Etcd using its HTTP API.
docker-gen -notify "/bin/bash /tmp/etcd.sh" -interval 10 templates/etcd.tmpl /tmp/etcd.shThis project uses Go Modules for managing 3rd party dependencies.
This means that at least go 1.11 is required.
For go 1.11 and go 1.12 it is additionally required to manually enable support by setting GO111MODULE=on.
For later versions, this is not required.
git clone <your fork>
cd <your fork>
make get-deps
makeMIT
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent