tiangolo / docker-with-compose Goto Github PK

Hi everyone
I just tried to use this useful image but getting error trying a simple docker-compose build ?

.gitlab-ci.ymlseems to me like copy-pasted from official docs, not sure I'm missing something? pretty new to GitLab builds, so I could miss something obvious.

Output from CI:

thanks
nicola

Currently, I have to install make everytime I want to run make commands. IMO, a good amount of people may be using make to run docker-compose commands.

Is it out of the scope to add the make installation to this package?

This is my current .gitlab-ci.yml file

image: tiangolo/docker-with-compose

services:
  - docker:dind

before_script:
  - apk add --update make

stages:
  - lint
  - test

lint:
  stage: lint
  script:
    - make ci-lint

test:
  stage: test
  script:
    - make ci-test

Hi,

I have a docker-compose.yaml that runs the Keycloak SSO server and Postgresql. The Keycloak HTTP server listens on port 8080 and so I have

-ports
  8080:8080

in the docker-compose.yaml.

In my .gitlab-ci.yml I do: docker-compose -f path/to/docker.compose.yaml up -d. I then start a watchdog so that I know when Keycloak is available by doing a curl -I http://localhost:8080 in a loop-with-sleep checking curl's exit status.

However, I get connection refused even when I know the Keycloak is up and running. I'm sure there's something I don't understand here. What is it?

The Dockerfile fails to build now because apt-get install cargo is required for docker-compose. See:

https://docs.docker.com/compose/install/#alternative-install-options

I had your image (thanks very much) set up with a private gitlab instance, but it just stopped working. Though there are no obvious changes on the server where it is executed nor in your repo. But maybe you have some idea?
config.toml and erros message below.

Running with gitlab-runner 11.11.2 (ac2a293c)
  on homeserver something
Using Docker executor with image tiangolo/docker-with-compose ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image sha256:6ce0d31cf4d670f84b0f74b1b15910f9e3fde4105232e14b0d360a6a49c05362 for docker:dind ...
Waiting for services to be up and running...
Pulling docker image tiangolo/docker-with-compose ...
Using docker image sha256:cf0fa05056110e8c9866200f6879b615ea55be3e54f558826aab5b0882537f70 for tiangolo/docker-with-compose ...
Running on runner-something-0 via ubuntu...
DEPRECATION: this GitLab server doesn't support refspecs, gitlab-runner 12.0 will no longer work with this version of GitLab
Fetching changes...
Checking out 13232142 as somebranch...

Skipping Git submodules setup
$ docker-compose up -d
Couldn't connect to Docker daemon at http://docker:2375 - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.
ERROR: Job failed: exit code 1

config.toml

concurrent = 1
check_interval = 0

[session_server]
session_timeout = 1800

[[runners]]
name = "homeserver"
url = "https://gitlab.something.eu/"
token = "<sometoken>"
executor = "docker"
[runners.custom_build_dir]
[runners.docker]
tls_verify = false
image = "tiangolo/docker-with-compose"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
[runners.cache]
  [runners.cache.s3]
  [runners.cache.gcs]

The latest push did break some of our deployments.

Initially posted in #32 but does not per se have the same issue.

Before:

FROM tiangolo/docker-with-compose:2021-09-18
WORKDIR /tmp
COPY docker-compose.yml ./
CMD ['docker-compose', '-f', 'docker-compose.yml', 'config']

After:

FROM tiangolo/docker-with-compose:latest
WORKDIR /tmp
COPY docker-compose.yml ./
CMD ['docker-compose', '-f', 'docker-compose.yml', 'config']

Basic file input docker-compose.yml

version: '3.8'
 
services:
  db:
    image: postgres:14-alpine
    environment:
      POSTGRES_USER: api
      POSTGRES_PASSWORD: azerty
      POSTGRES_DB: api
    volumes:
      - app-db:/var/lib/postgresql/data
 
volumes:
  app-db:

Output before:

services:
  db:
    environment:
      POSTGRES_DB: api
      POSTGRES_PASSWORD: azerty
      POSTGRES_USER: api
    image: postgres:14-alpine
    volumes:
    - app-db:/var/lib/postgresql/data:rw
version: '3.8'
volumes:
  app-db: {}

Output now:

name: tmp
services:
  db:
    environment:
      POSTGRES_DB: api
      POSTGRES_PASSWORD: azerty
      POSTGRES_USER: api
    image: postgres:14-alpine
    networks:
      default: null
    volumes:
    - type: volume
      source: app-db
      target: /var/lib/postgresql/data
      volume: {}
networks:
  default:
    name: tmp_default
volumes:
  app-db:
    name: tmp_app-db

• version: '3.8' is not present on the generated stack.yml (the target host fallback to version: 1 which is problematic. even though it is on our (input) files.
• volumes descriptions have changed unexpectedly so there are re-created (not found)
• it seems that this image generates a compose-compliant (v2) schema file while the command docker stack is not ready.

from remote, not actually the docker-with-compose image

docker stack deploy -c docker-stack.yml --with-registry-auth "xxxx"
unsupported Compose file version: 1.0

hint: the docker-stack.yml seems to be generated using compose v2 instead of docker-compose legacy version.

/usr/local/bin/docker-entrypoint.sh: exec: line 61: /bin/bash: not found

When I run docker-compose -version on my Mac I get:

1.26.2, build eefe0d31

Inside the tiangolo/docker-with-compose container I get:

1.26.0

I have an issue using contexts that only repros in the container, and I assume it's from the version difference. When I built the container manually, i.e. ran docker:latest then ran apk add --no-cache py3-pip python3-dev libffi-dev openssl-dev curl gcc libc-dev make && pip3 install docker-compose my issue disappeared.

Can you rebuild your docker image, to get the newest docker-compose included?

This image wasn't working for me in gitlab. (I'm running tests for the fastAPI project generator library) I was using the following:

image: tiangolo/docker-with-compose

before_script:
  - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
  - docker info
  - pip install docker-auto-labels

variables:
  DOCKER_TLS_CERTDIR: ""

stages:
  - test

tests:
  stage: test
  script:
    - sh ./scripts/test.sh

But would get the following error: error during connect: Post http://docker:2375/v1.40/auth: dial tcp: lookup docker on 169.254.169.254:53: no such host (see the job here)

Any ideas why this image isn't working? Is it due to the TLS changes with docker:dind?

Using the official docker/compose image, and changing the entrypoint, is working for me:

image:
  name: docker/compose:1.25.1-rc1
  entrypoint: ["/bin/sh", "-c"]

variables:
  DOCKER_TLS_CERTDIR: ""
  DOCKER_HOST: tcp://docker:2375/
  DOCKER_DRIVER: overlay2

services:
  - docker:dind

before_script:
  - docker info
  - docker-compose version

stages:
  - test

tests:
  stage: test
  script:
    - sh ./scripts/test.sh

Error -

Couldn't connect to Docker daemon at http+docker://localhost - is it running?

If it's at a non-standard location, specify the URL with the DOCKER_HOST environment variable.

In full-stack-fastapi-postgresql cookiecutter repo docker-auto-labels were used, and I think that including this package could speedup each CI pipeline a bit.

Running apk to install nodejs no longer installed npm depdendency.

Note that nghttp2-libs is installed, and nodejs-npm is not.

Left: Using current image (tiangolo/docker-with-compose:latest)
Right: Using image from a day ago

According to the docker-with-compose build details, the build was just updated.

Different versions of alpine are noted in the fetch comments.

can you add curl installation? may with same tag on docker image

FROM docker:latest

RUN apk add --no-cache curl
RUN apk add --no-cache py-pip
RUN pip install docker-compose

First of all, thanks a lot for this project.

Just wanted to let you (and others) know that the latest version (latest / 2022-11-10) doesn't work in Bitbucket Pipelines anymore:

+ docker-compose build
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 2.6s done
#1 creating container buildx_buildkit_default 0.0s done
#1 ERROR: Error response from daemon: authorization denied by plugin pipelines: --privileged=true is not allowed
------
 > [internal] booting buildkit:
------
Error response from daemon: authorization denied by plugin pipelines: --privileged=true is not allowed

I switched back to version 2021-09-18, which works fine.
Not sure if this is anything you can fix at all.

Now that compose has been added into the official docker image (docker-library/docker#361) what is the purpose of this repository anymore?

Is it time to tell folks to stop using it? Should it become a plain mirror for docker:latest?

gettext allows me to use envsubst which is nice for configuring rendering docker-compose.yml with external secrets, networks etc as their names better to be stored as CI variables rather as hardcoded value in docker-compose.yml.

Inside the Description section of the README.md file, there is a typo:

The main purpose of this image is to help in Continuous Integration environments that need the docker binary, the docker-compose binary and posibly (possibly) require doing other things, like running Bash scripts.

Document how to mount the Docker sock as a volume so that Docker Compose inside the container can have access to the local Docker service/daemon.

This has to be done instead of Docker-in-Docker: http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/

Probably with the ideas here: https://medium.com/better-programming/about-var-run-docker-sock-3bfd276e12fd

The README says:

This image allows running arbitrary commands like Bash scripts,

But the image does not contain bash. I propose we apk add bash.

docker-compose has some annoying bugs running on python2, e.g. not seeing log output of services that have been idle for some time

Furthermore, docker-compose is considering deprecating using it on Python 2 altogether.

We should use a base image with Python 3 and run docker-compose on that instead of Python 2.