just-containers / s6-overlay Goto Github PK

I'm having a hard time access environment variables within various scripts that I'm running inside the container.

I've read through the source of s6-overlay and found things like with-contenv and import-from-envdir, but I just can't get these to function.

I noted all of the scripts are in the execline language, using #!/usr/bin/execlineb -S0. Do we have to use that to use with-contenv and access the environment variables? We're currently using #!/bin/sh because that is what we're familiar with but can't access any of the environment variables.

There are two scenarios, we want to access environment variables for use within scripts, such as:

if [ -z ${TUTUM_SERVICE_HOSTNAME+x} ]; then
  BIND=`getent hosts ${HOSTNAME} | awk '{print $1}'`
else
  BIND=`getent hosts ${TUTUM_SERVICE_HOSTNAME} | awk '{print $1}'`
fi

And we also want to start things such as Node.js applications with the container environment, maybe something like:

#!/bin/sh

# cd into our directory
cd /app

# start our node.js application
exec with-contenv node server.js;

ideas are welcome :-), coming from #25

So our ADD layer in docker build:

ADD https://github.com/glerchundi/container-s6-overlay-builder/releases/download/v0.1.0/s6-overlay-0.1.0-linux-amd64.tar.gz /s6-overlay.tar.gz

Does not cache. According to the Dockerfile documentation on ADD command, it should check the mtime and not re-download the same file:

https://docs.docker.com/reference/builder/#add

Perhaps it is because the way github releases URL does behind redirects (i tihnk i saw to s3), and this is hiding to docker the real tarball file. And it cannot check the mtime.

I have not found yet such docker issue in docker/docker about it. Still looking.

Hi, I'm considering adding a new feature to current s6-overlay init process.

The idea is to pre-deploy multiple config sets (cont-init.d, cont-finish.d and services.d) into docker image, and to switch into one profile (dev, staging etc) when launching containers with specific environment variables.

Unlike popular config management software Puppet or Chef for example, which has its own init logic thus cannot fit into s6-overlay, this feature is focused on profile switching only. You still have to somehow generate config files, while it helps when you need to change container configs to fit variety environments.

I have updated my fork at master...azhuang:profile-switch .

I'm using S6_CONTAINER_PROFILE to pass profile name into container. To be compatible with previous version, it's default to null so that profile switching is disabled.

Any profile has its own directory at /etc/cont-profile.d/<profile_name>/ . Each of the profile currently has 3 sub directories as cont-init.d, cont-finish.d and services.d . The active profile will be automatically (symbolic) linked to /etc/{cont-init.d, cont-finish.d, services.d}.

The switching process is executed after fix-attr.d so that you can safely update container profile by mounting a volume into /etc/cont-profile.d/ . Then the init process will continue to load init scripts and services.

My work has not been tested thoroughly. I'd like to know what you think about this feature at first.

I run into issue when services falling into errors because of empty env variables, supervisor try's to restart them, but no success, env is not loading during attempts.
Env is exporting into /var/run/s6/container_environment/ via cont-init.d script. Shell started via CMD is getting all variables loaded.
So how can i ensure to not start services until cont-init.d script is done his work?

I have a service script (bash) that sets 2 env vars then runs a small compiled Go program. The script is installed in /etc/services.d/portster/run. (If curious the code for portster is here: https://github.com/gzoller/portster/blob/master/portster.go)

Steps to reproduce my problem:

1. I run the image -it and get into /bin/sh (no S6 running yet)
2. I can execute the run script manually with no problem
3. Kill that, then start /init

This will produce a failure on line 26 of my program:

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0xc8 pc=0x4011ed]

goroutine 1 [running]:
main.main()
    /go/src/github.com/gzoller/portster/portster.go:26 +0x1ed

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /usr/local/go/src/runtime/asm_amd64.s:1696 +0x1

Can you think of any reason why the same program, launched from the same run script would not run under /init? My program is using a library to read the Docker API tcp port 2376.

Contents of /etc/services.d/portster/run here:

#!/bin/sh
export DOCKER_CERT_PATH=/mnt/certs
export DOCKER_HOST=tcp://$HOST_IP:2376
/usr/local/bin/portster

I have a base image that uses s6-overlay:

FROM ubuntu-debootstrap:15.04

<snip>
ENV S6_LOGGING 0
ENV S6_BEHAVIOUR_IF_STAGE2_FAILS 2
COPY s6-overlay-1.14.0.1 /
ENTRYPOINT ["/init"]

I then start a container:

$ docker run --rm -it aris/base bash
<s6-overlay startup logs>
root@76248caedbb2:/# ls
bin   dev  home  lib    media  opt   root  sbin  sys  usr
boot  etc  init  lib64  mnt    proc  run   srv   tmp  var

Now I ctrl+d (or use exit):

root@76248caedbb2:/# exit
bash exited 0
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] syncing disks.
s6-svscanctl: fatal: unable to control /var/run/s6/services: supervisor not listening
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

Check out s6-svscanctl: fatal: unable to control /var/run/s6/services: supervisor not listening. I'm not sure what's causing this?

I have a base Docker containing a small program (portster) I want to run on start-up. I'm using S6 overlay for this and have create a script: /etc/services.d/portster/run (and finish).

So this works well by itself. I can derive trivial Dockers from this base image and so long as I don't define an ENDPOINT or CMD my portster program launches neatly without any assistance.

Now I want to use this base image to derive a new Docker holding a Java application. When run this application needs to accept parameters from the "docker run" command line (to be consumed as args in my Java code). I have a shell script that will launch my Java code (sets variables, classpath, etc.)

How can I configure S6 overlay to launch my Java app, accepting parameters, and still retain the core behavior of auto-launching/managing portster I initially needed (without the new Java Docker having to know about or manage portster)?

My docker container relies on env parameters to configure certain aspects of the main daemon process, which is configured as a service.

If the user does not configure these parameters properly, ideally I want the container to error out, which means that the service/run script exits with an error the container should stop. But with s6-overlay, the service will be relaunched over and over leaving it unusable. Is it possible to return with an error code that will make the container error out and stop?

The README text describes The Docker Way, where if "the thing" fails the container stops. I'm not clear how to configure/differentiate "the thing" process(es) from stuff like cron that should clearly be restarted. How is this done?

Thanks!

I have images that use the s6-overlay /init as the ENTRYPOINT. The init scripts allow running an ad-hoc command that will be supervised by s6 along with the other configured services, but there doesn't seem to be a way to retrieve the exit code of that ad-hoc command (understandably).

However, it seems that it should be possible to capture the exit code of that ad-hoc command (and perhaps write it to a file) and then exit with that code in the shutdown scripts. I'm assuming this without having dug around too much in the overlay code or being all that familiar with S6 in general.

I can work around this by bypassing S6 at runtime with --entrypoint=/bin/bash and then prepending my command with "-c", but this is a bit ugly.

We need to create a good README to help people identifying:

• why this project exists
• what it includes
• how should be used
• make references to our 'recommended' base images.
  • just-containers/base (ubuntu trusty)
  • just-containers/base-slim (alpine)

Whenever S6_LOGGING is set to 1 in my container, I get the following error:
s6-chown: fatal: GID not set

I create a service at /etc/services.d/ and it starts when the container is started. How can I manually stop the service and remove it from the services list so that it is not started again?

provide key for gpg verify and file with hash(es)

many thanks for making it easy to use s6 in Docker!

Hi Gorka,
Sorry to be a nuicance, but i've got a question about logging 2 services to stdout. Is this possible?

I am looking at the documentation here:

https://github.com/just-containers/s6-overlay#writing-a-service-script

But there is no mention of it.

v1.16.0.2
I am getting a fatal error on container shutdown when including a finish script for services as described here: https://github.com/just-containers/s6-overlay#writing-an-optional-finish-script

see also #76

Example

/etc/service.d/nginx/run

#!/usr/bin/execlineb -P
fdmove -c 2 1
nginx

/etc/service.d/nginx/finish

#!/usr/bin/execlineb -S0
s6-svscanctl -t /var/run/s6/services

$ docker logs

[services.d] starting services
[services.d] done.
[30-Nov-2015 04:55:08] ALERT: oops, unknown child (221) exited with code 0. Please open a bug report (https://bugs.php.net).
2015/11/30 04:55:08 [notice] 217#217: using the "epoll" event method
2015/11/30 04:55:08 [notice] 217#217: ngx_http_push_stream_module will not be used with this configuration.
2015/11/30 04:55:08 [notice] 217#217: nginx/1.8.0
2015/11/30 04:55:08 [notice] 217#217: OS: Linux 4.0.9-boot2docker
2015/11/30 04:55:08 [notice] 217#217: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2015/11/30 04:55:08 [notice] 217#217: start worker processes
2015/11/30 04:55:08 [notice] 217#217: start worker process 234
2015/11/30 04:57:02 [notice] 217#217: signal 15 (SIGTERM) received, exiting
2015/11/30 04:57:02 [notice] 234#234: exiting
2015/11/30 04:57:02 [notice] 235#235: exiting
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] syncing disks.
2015/11/30 04:57:02 [notice] 234#234: exit
[s6-finish] sending all processes the TERM signal.
2015/11/30 04:57:02 [notice] 217#217: signal 1 (SIGHUP) received, reconfiguring
2015/11/30 04:57:02 [notice] 217#217: signal 15 (SIGTERM) received, exiting
2015/11/30 04:57:02 [notice] 217#217: signal 17 (SIGCHLD) received
2015/11/30 04:57:02 [notice] 217#217: cache manager process 235 exited with code 0
2015/11/30 04:57:02 [notice] 217#217: signal 29 (SIGIO) received
2015/11/30 04:57:02 [notice] 217#217: signal 17 (SIGCHLD) received
2015/11/30 04:57:02 [notice] 217#217: worker process 234 exited with code 0
2015/11/30 04:57:02 [notice] 217#217: exit
s6-svscanctl: fatal: unable to control /var/run/s6/services: supervisor not listening
[s6-finish] sending all processes the KILL signal and exiting.

I have two suggestions for init system

1. overlay should be able to create directories (recursive) if dont exist when fixing perms
2. and there should be step before fixing perms

My usecase - i have project which heavily uses caches that needs to be cleaned every start (&perms fixed) - the only way to do that is in cont-init.d, leaving native way of fixing perms behind

Hello again!

There is a bit of confusion ATM about where / how to download the s6-overlay tarball build product. (end users). It would be helpful to make a fake Github Project to push the documentation + releases section to.

Then the README docs for this s6-overlay-builder, (and it's releases page) - can stay focussed on the build tool itself, which regular users to not need to see mixed together, or be confused about.

No big hurry. But it seems like a time to suggest when development has mostly stabalized. We just have to keep working and spit / polish the Documentation John has come up with.

@jprjr Wiki is actually harder to find. It not so obvious as a README.md and people don't normally see it very well. I can understand when is already mixed up with builder notes. But I hope we can just move your entire wiki page to be the main README.md file on new fake s6-overlay project. So people can see it / find it. & Many thanks for already doing a big hefty part of this s6-overlay README documentation. It is now much less daunting to finish off. Just be tweaked a little bit more / missing bits added to / improved incrementally. Appreciate it.

@glerchundi It would make sense to move the tarballs to other fake project soon, before too many other people start using those download URLs.

In our Dockerfile, we have version numbers har-coded, in the ADD urls, e.g.

https://github.com/glerchundi/container-s6-overlay-builder/blob/master/Dockerfile#L20

Where the version of fix-attrs package is different than execline package. And s6 package is yet a different version too.

It would be neater if we could set each seperate versions of each upstream source packages in some central manifest.txt file. Or to be downloaded from a .txt webpage on upstream s6-builder repo.

ATM the latest version numbers of each ones are hard-coded into our local Dockerfile.

If we can't use ENV to set them. Then maybe we could instead generate Dockerfile dynamically from a script (simple string substitution). To set the right version numbers.

It's not super important. If 'no' then fine. No big deal. Just a suggestion.

I had a good read through #13, but I couldn't really find the answer I was looking for.

I've used my own Alpine Linux base image, added s6-overlay and used https://github.com/just-containers/nginx-loadbalancer as an example to configure logging (the s6-overlay 'way' I guess). I have my logs going into /var/log/nginx-access-log/current and /var/log/nginx-error-log/current.

But I have some questions. How do I get this to hit stdout, rather than, or concurrently hitting the file as it does now? Also, what is the benefit to setting up the logging as per the nginx-loadbalancer example? I must admit, it was a lot of work to get the logging setup.

Most Docker images out there when run, by default, logging hits stdout. In our case, not just s6 logging, but the main process logging too. Many nginx containers do something like:

RUN ln -sf /dev/stdout /var/log/nginx/access.log && \
    ln -sf /dev/stderr /var/log/nginx/error.log

which is nice and simple (of course, maybe not what everyone would want). There isn't a lot of documentation on the s6-overlay preferred logging at the moment, so I'm a little confused.

I wanted to bring up the script that we all came up with in that thread, that is supposed to bring a container down when a server crashes.

#!/usr/bin/execlineb -S0
if -n { s6-test $# -ne 0 }
if -n { s6-test ${1} -eq 256 }

s6-svscanctl -t /var/run/s6/services

In doing some testing recently, we found that it doesn't actually work. If you take a look at the first line of the script (after the shebang) it reads if the number of arguments passed to the ./finish script is not equal to 0 then stop the ./finish script executing (resulting in the service being restarted). In the instance a script crashes, I would expect their to be a non-zero argument passed to the ./finish script. Therefore the script is not doing it's job. We've rewritten it with a detailed explanation (we're not strong in execline). Can you guys please help us validate it? We've run it, and it actually works when running it.

Rewrite

Desired outcome is to stop the container, if the supervised process exists non-zero (i.e. fails).
If the container was signaled to stop, this script should do nothing.

#!/usr/bin/execlineb -S0
if -n { s6-test ${1} -eq 0 }
if -n { s6-test ${1} -eq 256 }

s6-svscanctl -t /var/run/s6/services

Line 1

s6-test ${1} -eq 0

• If script is run and the first argument is 0, s6-test will exit 0.
• If script is run and the first argument is not 0, s6-test will exit 1.
• Explained: if the supervised process exited non-zero s6-test will exit 1.

if -n { s6-test ${1} -eq 0 }

• If s6-test exits 0 (true), exit 1.
• If s6-test exits 1 (false), goto next line.
• Explained: if the supervised process exited non-zero, goto next line. If the supervised process exited 0, stop execution of the ./finish script.

Line 2

s6-test ${1} -eq 256

• If script is run and the first argument is 256, s6-test will exit 0.
• If script is run and the first argument is not 256, s6-test will exit 1.
• Explained: if the supervised process was signaled s6-test will exit 0.

if -n { s6-test ${1} -eq 256 }

• If s6-test exits 0 (true), exit 1.
• If s6-test exits 1 (false), goto next line.
• Explain: if the supervised process was signalled, stop execution of the ./finish script. If the supervised process was not signalled, goto next line.

because probably in most cases containers are supervised and logged/rotated already, our overlay should propose the best pattern to redirect everything to stdout/stderr instead of /var/log/s6-uncaught-logs.

@skarnet any ideas? lets discuss this here ;-)

Hi again,
I have the following in my dockerfile:

# Install s6-overlay
ADD https://github.com/just-containers/s6-overlay-builder/releases/download/v1.9.1.1/s6-overlay-linux-amd64.tar.gz /tmp/
RUN tar zxf /tmp/s6-overlay-linux-amd64.tar.gz -C / && $_clean
ENV S6_LOGGING="1"
# ENV S6_KILL_GRACETIME="3000"

However now when i try to start my container, it exits, saying:

s6-mkfifo: fatal: unable to mkfifo /var/run/s6/uncaught-logs-fifo: File exists

Things seemed to have worked the first 1-2 time around. After the container was initially created. But not subsequent times after that.

@glerchundi Please let me know if any specific test(s) you want me to do. Or tell me there is nothing we can check up on and is OK to delete this container and start over again. Many thanks.

Hi,

Just started using s6-overlay (coming from supervisord) and love it so far. I do have a few issues / questions. Please let me know if I should submit separate issues.

1. On this line of init-stage3, s6-nuke is called with both -t and -h. This causes both HUP and TERM to be sent to all processes, even though the informative log message mentions only TERM. Shouldn't this be -t only? The HUP causes my process to reload their config, just before getting killed anyway. If there is valid reason for this (which I'd love to learn), I would suggest changing the log message.
2. Also in init-stage3, this stage is sleeping (by default) for 5 seconds, before sending KILL. It seems to do this every time, even if there are no more processes to "nuke". Even if I have a container running services that terminate quickly, docker stop (or restarts) will still take 5 seconds, which seems unnecessary. Any ideas / solutions for this?
3. This might be more of an issue with s6-svscan instead of s6-overlay. With supervisord (running as pid 1), sending a HUP to it (or docker kill -s HUP), would cause the HUP to be "forwarded" to all running services. This in turn would make all services reload their configs, as is a common convention for many daemons. With s6-overlay, this does not happen, instead causing the container to (cleanly) exit. Is there a way to forward signals like supervisord does? Might be useful for other signals as well, like USR1 or USR2. Perhaps this is possible using s6-svscan -s, but if I understand the documentation correctly, this means we have to handle (i.e. properly deal with) all 6 signals in .s6-svscan/ scripts, or else signals we be ignored. Any suggestions?

Thanks!

the stage1 will always output same stuff about fix attrs. Like this:

$ docker run --tty --name s6test dreamcat4/s6test
[fix-attrs] fixing file attributes (ownership & permission)...
[fix-attrs] /etc/fix-attrs.d/00-base: applying...
[fix-attrs] /etc/fix-attrs.d/00-base: done.
[fix-attrs] done.

Might be nice to eventually hide that. Replace it with something else. e.g.;

s6 starting up...
done

Which is useful when using execline incurrs some significant startup delay.

Else we could print nothing (be completely silent). Until quit triggers the reaping (which is always useful to print out).

Or if we could let users set an optional ENV variable. To tell our stage1,2,3 to switch on a full debug output. Not sure if that is possible or might make too complex the script (because it is written in execline).

Hey guys, just noticed this:

If I make a runscript like:

#!/usr/bin/with-contenv /usr/bin/env perl

It's a contrived example, but it looks like with-contenv treats /usr/bin/env perl as a single argument, then dies (since /usr/bin/env perl isn't a valid program).

I'm not super-hot as execline, what's the right way to make sure the rest of the shebang is split up into arguments?

I started writing an app container using v1.13.0.0. Yesterday I upgraded the container to use v1.14.0.3, which I see introduced profiles in #81. Since then, I'm experiencing an interesting problem. The first time I start a specific container, everything is fine. But on subsequent starts, each of the /etc/ directories has had a recursive symlink added to it. This appears to be added during /init on the second time the container is started. These symlinks cause the init scripts to not work properly.

You can see this in action with my base image:

docker run --name=testing -it inklesspen/s6-py3-node:latest /init
# ctrl-c to stop it
docker start -ai testing

Of course, if I remove the container after stopping it, the next run works perfectly.

Hey John,

first of all thx to all contributors for this great package.

I am using alpine linux for a minimalistic approach, and
I saw you are a contributor for alpine linux either.

I got s6-overlay working as a trimmed down package for alpine linux
using their s6 package provided by Laurent Bercot.

apk -U add --allow-untrusted --repository ${ALPINE_EDGE_REPO}  s6 execline s6-portable-utils 

• there is a misalignment between /usr/bin/execlineb and /bin/execlineb
• some execline scripts has absolute path I changed it removing the prefix for several ones.
• also bang paths must be aligned to /bin/execlineb
• I built a new overlay package using the files below without the embedded s6

Question can you build a s6-overlay for alpine linux repository "s6-overlay" with dependencies listed above?

root # tree s6-overlay-amd64-v1.16.0.1-mod.tar.gz
.
|-- etc
|   |-- cont-finish.d
|   |-- cont-init.d
|   |-- fix-attrs.d
|   |-- s6
|   |   |-- init
|   |   |   |-- env
|   |   |   |   `-- PATH
|   |   |   |-- env-stage2
|   |   |   |-- init-stage1
|   |   |   |-- init-stage2
|   |   |   |-- init-stage2-fixattrs.txt
|   |   |   |-- init-stage2-redirfd
|   |   |   `-- init-stage3
|   |   |-- init-catchall
|   |   |   |-- init-stage1
|   |   |   `-- init-stage2
|   |   |-- init-no-catchall
|   |   |   |-- init-stage1
|   |   |   `-- init-stage2
|   |   `-- services
|   |       |-- s6-fdholderd
|   |       |   |-- down
|   |       |   |-- notification-fd
|   |       |   |-- rules
|   |       |   |   `-- uid
|   |       |   |       |-- 0
|   |       |   |       |   |-- allow
|   |       |   |       |   `-- env
|   |       |   |       |       |-- S6_FDHOLDER_GETDUMP
|   |       |   |       |       |-- S6_FDHOLDER_LIST
|   |       |   |       |       |-- S6_FDHOLDER_RETRIEVE_REGEX
|   |       |   |       |       |-- S6_FDHOLDER_SETDUMP
|   |       |   |       |       `-- S6_FDHOLDER_STORE_REGEX
|   |       |   |       `-- default
|   |       |   |           |-- allow
|   |       |   |           `-- env
|   |       |   |               `-- S6_FDHOLDER_LIST
|   |       |   |-- run
|   |       |   `-- supervise
|   |       |-- .s6-svscan
|   |       |   |-- crash
|   |       |   `-- finish
|   |       `-- s6-svscan-log
|   |           `-- run
|   `-- services.d
|-- init
`-- usr
    `-- bin
        |-- fix-attrs
        |-- logutil-newfifo
        |-- logutil-service
        |-- logutil-service-main
        |-- printcontenv
        |-- with-contenv
        `-- with-retries

24 directories, 32 files

Hi. Sorry for the long output. Basically I type CTRL^C on the keyboard (interactive tty). And that sends an interrupt signal (SIGINT) which does not reap the orphans. Wheras docker stop command sends a TERM, which does in fact works.

It would be nice to respong for CTRL^C (SIGINT) too. Other possible signals to listen for are QUIT and HUP.

Anyway, here is all the long testing outputs:

I've been running s6 as process management for Docker for a while. It's been working extremely well. I have a vagrant machine setup to do all of this, including statically building all of the s6 binaries.

https://github.com/smebberson/docker-ubuntu-base

You can see we're making quite a collection of base images with services (redis, mongo, etc) being started by s6. It's all working great.

Can I ask a couple of questions?

It this ready to accept service directory run, and finish scripts? All of my current images are configured this way. Should I be moving to your base image, or keep using mine for now?

Also, when you CTRL+C on a running container, does s6 handle terminating all of the processes? If it does, does it execute the finish script within the service directory, or does that only happen when the process dies?

I'm working on a new image running Consul, and have the following finish script:

#!/usr/bin/env bash

exec s6-setuidgid consul consul leave;

But it never seems to execute. I know the process that was started by s6 (consul), is receiving the signal. I can confirm that, but not that the finish script is executed (I'm testing this by adding echo "finish script" to the above).

Hi,

Sometimes I'm changing service run files and restart them with s6-svc -u /var/run/s6/services/service-name command. If I change a service runfile then I have to copy the file from /etc/services.d/service-name/run to /var/run/s6/services/service-name/.

Should I remove service down files from both folders? or /etc/services.d/service-name folder is enough?

P.S. : I'm changing files after init command and don't want to call init again.

Thanks

Laurent says:

Use execline scripting for init, because init only needs to be run 1 time at system start. However this is not true for containers (or jails, namespaces, or whatever you want to call it). Which uses linux namespaces kernel feature.

The startup time is OK if you have just a few containers. Then it OK and does not seem too slow.

However it is possible to have many more containers running. For example on a production server - to utilize the full host resources.

Then the startup time matter more. For example, let us imagine a XEON host was fully loaded, and it had to start up 100 containers at boot time. If we assume execline based scripts incur a 1 second delay starting inside each container. Then in total that take 100 seconds (additional overhead). For to bring up the server or restart it.

I have not tested yet our init startup time. It is not "1 second". Yet it is not instant.

Maybe in future (eventually) we can write a 'C' program instead to replace the stage1 and stage2. Then such extra delay / overhead can be a lot less. Expecially if not launching out many individual seperate programs (like we are during the execline script).

I created a sudo user devops and use it as the default user.
I observed a difference in container behaviour of finishing a non daemon cmd like:
docker run --rm -it s6base ls /etc/s6

Dockerfile:
USER root
ENTRYPOINT ["/init"]
stdout:
$ [s6-finish] syncing disks.
$ [s6-finish] sending all processes the TERM signal.
$ [s6-finish] sending all processes the KILL signal and exiting.

Dockerfile:
USER devops
ENTRYPOINT ["sudo","/init"]
stdout:
$ [s6-finish] syncing disks.
$ [s6-finish] sending all processes the TERM signal.
$ Hangup

I checked the scripts in /etc/s6 to search for the clue but didn't get it.
if 'Hangup' means '-HUP/-15', it is a grace way to TERM the container.
But not sending the KILL signal concerns me.

I am not sure of this. But can it be we are responding to easily to CTRL^C message from the terminal?

If i run:

• docker attach <container> OR
• docker start -a <container>

Then type CTRL^C. The s6-overlay will catch that signal and stop the cotnainer. That is great in some situations. But can be a problem (how to close the terminal?) when we just want to detach the output and keep the container running.

Does anyone else see this behaviour?

To my memory, from docker attach, pressing CTRL^C without the s6-overlay installed would just detach the session. And the container would keep running until we typed docker stop <container>.

Perhaps pressing CTRL^C twice can ignore the first keypress. Giving docker the oppotunity to catch the first CTRL^C signal itself? Perhaps a new configuration option (ENV var) can be set to decide the CTRL^C handling? Is there an alternative keypress to detach from the docker attach session, without also sending an INT signal to end the program?

Else I'm not really sure what's going on here. Please help if you can.

Hi,

I'm starting getting into s6-overlay (not willing to read the whole s6 project yet) but I did not find a way to define services dependencies. Is it s6-rc integrated in this s6-overlay?

If so, how can I make use of it. Otherwise, any hint about how I can use s6-rc stuff in my docker image is welcome.

Thanks!

Hi,

I found your effort of building this s6-overlay through the thread on supervision mailing list. I have been following it a bit (love it!) and, as a test, am currently migrating my images (based on the Phusion baseimage) to use this overlay.

I know this is work in progress, just wanted to provide some feedback from my tests.

I have some of the images running, but as of the latest build I have some warnings in the docker logs output. I get the following output (when using S6_USE_CATCHALL_LOGGER set to 0):

$ docker logs clacks
[fix-attrs.d] applying owners & permissions fixes...
[fix-attrs.d] applying 00-runscripts...  exited 0
[fix-attrs.d] applying base_scripts...  exited 0
[fix-attrs.d] applying clacks_scripts...  exited 0
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
s6-applyuidgid: usage: s6-applyuidgid [ -z ] [ -u uid ] [ -g gid ] [ -G gidlist ] [ -U ] prog...
s6-applyuidgid: usage: s6-applyuidgid [ -z ] [ -u uid ] [ -g gid ] [ -G gidlist ] [ -U ] prog...
s6-applyuidgid: usage: s6-applyuidgid [ -z ] [ -u uid ] [ -g gid ] [ -G gidlist ] [ -U ] prog...

It seems to be triggered by the /etc/s6/services/s6-svscan-log/run script, but the call inside looks OK to me?

It looks like there is currently no guarantee that all user defined services are actually started before the CMD is executed so we might have a race condition when the CMD depends on a service to be running.
Could we leverage s6-svwait to make sure that services have been started before running the CMD?

In https://github.com/just-containers/s6-overlay/blob/master/builder/overlay-rootfs/etc/s6/init/init-stage2#L136 insert something like this:

        if
        {
          pipeline { s6-ls -0 -- /var/run/s6/etc/services.d }
          forstdin -0 -p -- i
          importas -u i i
          if { s6-svwait -t 1000 -u /var/run/s6/services/${i} }
        }
        if { s6-echo -- "[services.d] done." }

it should be included as a statically linked binary, remove any dependency with go runtime in order to create as lightweight as possible s6-overlay.

Images like busybox:ubuntu-14.04 don't include neither nobody in /etc/passwd nor norgroup in /etc/group and init fails to start outputting this:

s6-envuidgid: fatal: unknown user: nobody
if: fatal: child crashed with signal 13

!!!!!
 init-stage2 failed.
 Logs are in /var/log/s6-uncaught-logs
!!!!!

As our overlay can just deploy static files in the destination image the solution would be to perform these actions in runtime:

• check if nobody/nogroup exists
• create them if they didn't exist.

What do you think?

Hi,

first of all thanks for this great project. It solved so many problems with my homebrew solution.

Here is my question:

I often need to create directories (e.g. /run/xxx) for my docker containers which I implement as a initialization script inside cont-init.d. As the scripts for applying owners & permissions inside fix-attrs.d run before the cont-init.d scripts I have to deal "manually" with the permissions inside the cont-init.d scripts.

Are the any reasons why the fix-attrs.d scripts run before the cont-init.d scripts? I think the cont-init.d scripts could benefit from the fix-attrs.d scripts if we could swap the order.

Thanks and regards

Ingo

I often get messages like this:

Sep 15 22:16:00 some-worker-15.09.15t18.47 docker[29106]: node-harmony exited 0
Sep 15 22:16:00 some-worker-15.09.15t18.47 docker[29106]: [cont-finish.d] executing container finish scripts...
Sep 15 22:16:00 some-worker-15.09.15t18.47 docker[29106]: [cont-finish.d] done.
Sep 15 22:16:00 some-worker-15.09.15t18.47 docker[29106]: [s6-finish] syncing disks.
Sep 15 22:16:00 some-worker-15.09.15t18.47 kernel: s6-pause[29159]: segfault at 0 ip 00000000004005d7 sp 00007ffcd581b980 error 4 in s6-pause[400000+2000]
Sep 15 22:16:00 some-worker-15.09.15t18.47 systemd-coredump[29848]: Failed to get EXE.
Sep 15 22:16:00 some-worker-15.09.15t18.47 docker[29106]: [s6-finish] sending all processes the TERM signal.
Sep 15 22:16:00 some-worker-15.09.15t18.47 systemd-coredump[29848]: Process 27 (kblockd) of user 0 dumped core.

I have node.js program that runs in docker container via systemd (coreos/fleet). It does some work and gracefully exits (node-harmony exited 0). In s6 layer there is only one cont-init.d script and nothing else. ExecStop and ExecStopPost in systemd don't run for this units. Unit fails with failed state because.

full log

s6-overlay: v1.14.0.4
docker version 1.7.1, build 2c2c52b-dirty
base image: alpine:3.2
CoreOS beta: 766.3.0

1. Don't build four tarballs by default. Make no debug linux the default, and check environment variables for build the portable or debug versions. Also, document the differences between these versions.

2. Don't include all s6 utilities by default, only the subset actually used by the init scripts.

3. It's a bit confusing that you're bringing in a package from your other skaware-builder repo. I'm assuming this is just the s6 source? Could you pull it from an official location instead?

4. Why does /init wrap /etc/s6/init/init-stage1? It's an extra layer of indirection, can this be replaced with a symlink?

5. The flow from /etc/s6/init/init-stage1 -> /etc/s6/init/init-{catchall,no-catchall}/init-stage1 -> /etc/s6/init/init-{catchall,no-catchall}/init-stage2 -> /etc/s6/init/init-stage1 is confusing. Perhaps the middle steps could renamed stage1-post and stage2-pre, or conditionally imported into the main scripts?

6. What does the "vital fifo" /var/run/s6/services/s6-fdholderd/supervise/control do?

7. Why are some services in /etc/s6/services and others in /etc/services.d? Can they be merged to a single location?

8. I assume no-catch-all is inaccurately named, and actually does catch all, but logs everything to stdout/stderr?

9. I'm going to attempt to boil down your scripts to the simplest possible version (mentioned this on the mailing list). I'd really appreciate it if you could review them when I'm done!

10. Why is only init-stage2-redirfd broken out into its own separate file? Either all of the sections of stage2 should broken out, or none.

11. Could init-stage3 be moved to .s6-svscan/finish?

So to recap. The Docker USER directive in Dockerfile. It is a part of the official Docker feature set. The directive provides a simple way to allow people to run their apps with better security by not being the root user. There are other ways people can write containers to achieve the same thing. By starting as root then dropping privileges to the target user before running their program. Or have the program do that itself when it starts up.

However it would provide best and most complete support for Docker community - if we can support USER directive. Not sure how much messing about is required to do that.

The problem presented isn't just about modifying our scripts to avoid doing the [env]setuidgid on processes. But rather it may be difficult to some of these helper processes (logging and such) to be permitted to write to certain directories, when beforehand the overlay tarball does not know which the process UID will be.

Another possible solution might be to assign a single fixed PID number (we choose in the 32000 range again). And tell users they must configure their docker USER to be only that particular fixed uid:gid. Which means we can know it beforehand, and (again) hard-code directories to be owned by that single pre-determined UID number.

Hey guys, what do you think about allowing cont-init.d scripts to echo to stdout?

What is your version strategy. It has 4 numbers, so I wanted to clarify exactly what it is...

Thanks!

The logutil-service writes logs as user nobody or with 32768:32768 if that user is not present. The logutil-newfifo script does have a owner option to do this, but it would be nice to have that ability in logutil-service as well.

Right... with newest v1.8.2 we are always outputting certain things during the startup and shutdown. That is good for debugging problems, and OK when running things in daemon mode. However for single-shot cmdline operation, is not desirable.

I wonder what would be the best way to do this in the stageN scripts ?

Here is a real-world example:

s6-overlay v1.8.2 output*

s$ docker run dreamcat4/tvheadend-uk-atlas --help
[fix-attrs.d] applying owners & permissions fixes...
[fix-attrs.d] applying 00-runscripts...  exited 0
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
/usr/bin/tvheadend -u hts -g video -c /config --help
Usage: /usr/bin/tvheadend [OPTIONS]

Generic Options

  -h, --help                  Show this page
  -v, --version               Show version infomation

Service Configuration

  -c, --config                Alternate config path
  -B, --nobackup              Do not backup config tree at upgrade
  -f, --fork                  Fork and run as daemon
  -u, --user                  Run as user
  -g, --group                 Run as group
  -p, --pid                   Alternate pid path
  -C, --firstrun              If no user account exists then create one with
                              no username and no password. Use with care as
                              it will allow world-wide administrative access
                              to your Tvheadend installation until you edit
                              the access-control from within the Tvheadend UI
  -U, --dbus                  Enable DBus
  -e, --dbus_session          DBus - use the session message bus instead system one
  -a, --adapters              Only use specified DVB adapters (comma separated)
      --satip_xml             URL with the SAT>IP server XML location

Server Connectivity

  -6, --ipv6                  Listen on IPv6
  -b, --bindaddr              Specify bind address
      --http_port             Specify alternative http port
      --http_root             Specify alternative http webroot
      --htsp_port             Specify alternative htsp port
      --htsp_port2            Specify extra htsp port
      --useragent             Specify User-Agent header for the http client
      --xspf                  Use xspf playlist instead M3U

Debug Options

  -d, --stderr                Enable debug on stderr
  -s, --syslog                Enable debug to syslog
  -l, --logfile               Enable debug to file
      --debug                 Enable debug subsystems
      --trace                 Enable trace subsystems
      --fileline              Add file and line numbers to debug
      --threadid              Add the thread ID to debug
      --uidebug               Enable webUI debug (non-minified JS)
  -A, --abort                 Immediately abort
  -D, --dump                  Enable coredumps for daemon
      --noacl                 Disable all access control checks
      --nobat                 Disable DVB bouquets
  -j, --join                  Subscribe to a service permanently

TODO: testing

      --tsfile_tuners         Number of tsfile tuners
      --tsfile                tsfile input (mux file)

For more information please visit the Tvheadend website:
  https://tvheadend.org

/entrypoint.sh exited 0. Stopping the supervision tree.
[s6-finish] syncing disks.
if: fatal: child crashed with signal 11
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

Desired output: - s6-overlay is silent

$ docker run --entrypoint="tvheadend" dreamcat4/tvheadend-uk-atlas --help
Usage: tvheadend [OPTIONS]

Generic Options

  -h, --help                  Show this page
  -v, --version               Show version infomation

Service Configuration

  -c, --config                Alternate config path
  -B, --nobackup              Do not backup config tree at upgrade
  -f, --fork                  Fork and run as daemon
  -u, --user                  Run as user
  -g, --group                 Run as group
  -p, --pid                   Alternate pid path
  -C, --firstrun              If no user account exists then create one with
                              no username and no password. Use with care as
                              it will allow world-wide administrative access
                              to your Tvheadend installation until you edit
                              the access-control from within the Tvheadend UI
  -U, --dbus                  Enable DBus
  -e, --dbus_session          DBus - use the session message bus instead system one
  -a, --adapters              Only use specified DVB adapters (comma separated)
      --satip_xml             URL with the SAT>IP server XML location

Server Connectivity

  -6, --ipv6                  Listen on IPv6
  -b, --bindaddr              Specify bind address
      --http_port             Specify alternative http port
      --http_root             Specify alternative http webroot
      --htsp_port             Specify alternative htsp port
      --htsp_port2            Specify extra htsp port
      --useragent             Specify User-Agent header for the http client
      --xspf                  Use xspf playlist instead M3U

Debug Options

  -d, --stderr                Enable debug on stderr
  -s, --syslog                Enable debug to syslog
  -l, --logfile               Enable debug to file
      --debug                 Enable debug subsystems
      --trace                 Enable trace subsystems
      --fileline              Add file and line numbers to debug
      --threadid              Add the thread ID to debug
      --uidebug               Enable webUI debug (non-minified JS)
  -A, --abort                 Immediately abort
  -D, --dump                  Enable coredumps for daemon
      --noacl                 Disable all access control checks
      --nobat                 Disable DVB bouquets
  -j, --join                  Subscribe to a service permanently

TODO: testing

      --tsfile_tuners         Number of tsfile tuners
      --tsfile                tsfile input (mux file)

For more information please visit the Tvheadend website:
  https://tvheadend.org

You'll get messages like this when you try to run the container since it is invoking the /usr/bin/import binary in all the s6-overlay scripts that call import.

$ docker run -it --rm test
import: unable to open X server `' @ error/import.c/ImportImageCommand/369.

Any ideas on what the best way would be to fix this?

Is it currently possible to change the user (account) and the group via a fix-attrs.d file?

I tried:

/usr/sbin/ssmtp false root:ssmtp 2755 0750
/etc/ssmtp true root:ssmtp 0640 0750

but it raises an error (but still exits with 0):

[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] 00-ssmtp: applying...
s6-envuidgid: fatal: unknown user: root:ssmtp
s6-envuidgid: fatal: unknown user: root:ssmtp
s6-envuidgid: fatal: unknown user: root:ssmtp
s6-envuidgid: fatal: unknown user: root:ssmtp
[fix-attrs.d] 00-ssmtp: exited 0.
[fix-attrs.d] done.