Docker in Docker - probably harmful? about kythe

[danielmoy-google]

astroilov points out correctly that the other alternative approach here is to excise the internal docker bits from current extractrepo binary.

Right now it's fairly baked in, and pulling it out wouldn't be pretty. In particular given the dynamic generation of the ephemeral dockerfile it produces, we'd need to have separate running steps there: running docker image that just takes config as input and generates an ephemeral dockerfile, and then something to run that resulting dockerfile.

I don't see a super clean approach either way.

[danielmoy-google]

Behind door A, investigating the extent to which dind will work for us:

FROM docker:dind

ADD extractrepo extractrepo

ENTRYPOINT ["/extractrepo"]

I'm having some normal docker mechanical difficulties in dealing with input/output, but my initial guess is that's purely because I don't know how to hold this tool properly, not because of any problems with dind.

[danielmoy-google]

^ Nope actually I'm wrong. My problem is still dind, not because of holding input/output:

$ docker run -i -t -v ~/code/testrepo:/inputrepo -v /tmp/dockertest:/outputrepo test-extract -local /inputrepo --output /outputrepo
2018/06/14 19:35:55 Using configuration file: ""
2018/06/14 19:35:55 Failed to clean up docker image: exit status 1
2018/06/14 19:35:55 Failed to extract repo: building docker image: exit status 1
Command output time="2018-06-14T19:35:55Z" level=error msg="failed to dial gRPC: cannot connect to the Docker daemon. Is 'docker daemon' running on this host?: dial unix /var/run/docker.sock: connect: no such file or directory"
context canceled

dang

[danielmoy-google]

Ok well:

$ docker run -i -t -v ~/code/testrepo:/inputrepo -v /tmp/dockertest:/outputrepo -v /var/run/docker.sock:/var/run/docker.sock test-extract -local /inputrepo --output /outputrepo
2018/06/14 19:40:19 Using configuration file: ""
2018/06/14 19:40:21 Failed to extract repo: extracting repo: exit status 1
Command output: cp: cannot stat 'pom.xml': No such file or directory

That might be much closer (probably just failed to copy the repo in properly because this is literally the first time I've ever tried to pass data into docker.

[danielmoy-google]

Ok update - docker in docker sort of works. There's complications in mapping 1st-layer container volumes to the 2nd-layer innermost container. That can be hacked around with e.g.:

$ - outside
/ # - inside first container
# - inside innermost container
(/tmp/testdocker/Dockerfile is just a dummy docker file to show proof of concept, it would be replaced with the extractrepo generated Dockerfile)
 
$ docker run --entrypoint "sh" -i -t --volume /tmp/repodir -v ~/code/craigtest:/inputrepo -v /tmp/dockertest:/output -v/tmp/testdocker:/buildstuff -v /var/run/docker.sock:/var/run/docker.sock test-extract
 
/ # docker build -t bad-test /buildstuff
 
/ # echo 'fake repo' > /tmp/repodir/notarealrepo
 
/ # export UNSPEAKABLETHINGS=`cat /proc/self/cgroup | grep 'docker/' | tail -1 | awk -F "/" '{print $3}' | cut -c 1-12`
 
/ # docker run -i -t --entrypoint "sh" --volumes-from $UNSPEAKABLETHINGS bad-test
 
# cat /tmp/repodir/notarealrepo
fake repo

This will need a modification to extractrepo binary to configure the temp repo dir from output, since the specific volume has to be mapped at container create time. However we have flexibility here and can just name it whatever we want.

Also I think we're still strongly leaning towards not doing docker-in-docker for medium term. This is just a temporary hack.