Traefik couldn't deploy new stacks , 404 error about dockerswarm.rocks

[stratosgear]

Ok, I found a solution in my case.

In my previous installation Traefik was talking HTTPS with the proxied service (Zulip server). It seems that under swarm.rocks installation this does not work.

Using DISABLE_HTTPS in the Zulip server, and making Traefik talk to traefik.port: 80 solved my problem. All is working fine now.

Still a bit weird why under the current installation that would cause a problem, whereas this used to work, but since I got this working now, I'll leave it at that. It's not less secure anyways, since the traffik between traefik and zulip server is internal communications only (I think/hope)

I'll let myself out now. Sorry for "polluting" this issue.

[tiangolo]

Yes, that's correct @noobcoder1983 .

[rayrrr]

@noobcoder1983 yes, that is correct, at least in my experience...it looks like every container is assigned a unique "internal IP" as well, which prevents such port conflicts.

Ah alright good to know, this makes so many things much easier.

[pattonwebz]

Check and see what is inside of the DOMAIN and TRAEFIK_PUBLIC_TAG environment variables. Maybe they are not set to the correct values? If the traefik.tags value is wrong when this is deployed then the way Traefik is setup in this repo it would ignore those containers startups.

[tiangolo]

Thanks for your help here @pattonwebz ! Good advice.

Also, if your Wordpress is expecting to receive requests at one host name (for example, localhost), it might not respond to requests that come to a different host name (for example, yourdomain.com). I'm not sure how the official Wordpress image is configured, but that could be the case.

[MerNat]

Documentation in the site is fully there. But

you can see this example:
https://gist.github.com/MerNat/8ee69c5eeaa4ac909b0044f48e9569b0#file-gistfile1-txt

[stratosgear]

I have a similar experience with a different docker-compose for the Zulip Chat app (not sure it is exactly the same, but the behavior is suspiciously similar)

The docker compose was copied and pasted from another Portainer installation (runnning sans swarm.rocks installation instructions, in a non swarm mode), and it was working.

Trying to start it up from the Portainer of swarm.rocks (and trying many tweaks to make it work) always gives me, 404s or 502s

Here is the docker-compose:

version: '3'

services:
  database:
    image: "zulip/zulip-postgresql"
    environment:
      POSTGRES_DB: zulip
      POSTGRES_USER: zulip
      POSTGRES_PASSWORD: pg_zulip
    networks:
      - default
    volumes:
      - "zulip2_psql_data:/var/lib/postgresql/data:rw"

  memcached:
    image: "quay.io/sameersbn/memcached:latest"
    networks:
      - default

  rabbitmq:
    image: "rabbitmq:3.7.7"
    hostname: zulip-rabbit
    environment:
      RABBITMQ_DEFAULT_USER: "zulip"
      RABBITMQ_DEFAULT_PASS: "rb_zulip"
    networks:
      - default
    volumes:
      - "zulip2_rabbitmq_data:/var/lib/rabbitmq:rw"

  redis:
    image: "quay.io/sameersbn/redis:latest"
    networks:
      - default
    volumes:
      - "zulip2_redis_data:/var/lib/redis:rw"

  zulip:
    image: "zulip/docker-zulip:2.0.2-0"
    expose:
      - 443
    #ports:
    #  - "9990:80"
    #  - "7443:443"
    environment:
      DB_HOST: "database"
      DB_HOST_PORT: "5432"
      DB_USER: "zulip"
      SSL_CERTIFICATE_GENERATION: "self-signed"
      SETTING_MEMCACHED_LOCATION: "memcached:11211"
      SETTING_RABBITMQ_HOST: "rabbitmq"
      SETTING_REDIS_HOST: "redis"
      SECRETS_email_password: "xxxxxxxx"
      SECRETS_rabbitmq_password: "rb_zulip"
      SECRETS_postgres_password: "pg_zulip"
      SECRETS_secret_key: "XXXXXXXX"
      SECRETS_google_oauth2_client_secret: "XXXXX"
      SECRETS_social_auth_github_secret: "XXXXXX"
      SETTING_EXTERNAL_HOST: "xxx.xxxxx.com"
      SETTING_ZULIP_ADMINISTRATOR: "[email protected]"
      SETTING_EMAIL_HOST: "smtp.gmail.com"  # e.g. smtp.example.com
      SETTING_EMAIL_HOST_USER: "[email protected]"
      SETTING_EMAIL_PORT: "587"
      # It seems that the email server needs to use ssl or tls and can't be used without it
      SETTING_EMAIL_USE_SSL: "False"
      SETTING_EMAIL_USE_TLS: "True"
      ZULIP_AUTH_BACKENDS: "EmailAuthBackend,GoogleMobileOauth2Backend,GitHubAuthBackend"
      SETTING_GOOGLE_OAUTH2_CLIENT_ID: "XXXXXXXXXX-XXXXXXXX.apps.googleusercontent.com"
      SETTING_SOCIAL_AUTH_GITHUB_KEY: "XXXXXXX"
      #DISABLE_HTTPS: "True"
      # Uncomment this when configuring the mobile push notifications service
      SETTING_PUSH_NOTIFICATION_BOUNCER_URL: 'https://push.zulipchat.com'
    networks:
      - default
      - traefik-public
    volumes:
      - "zulip2_app_data:/data:rw"
    deploy:
      placement:
        constraints:
          - node.role == manager    
      labels:
        traefik.frontend.rule: Host:xxx.xxx.com
        traefik.enable: 'true'
        traefik.backend: zulip
        traefik.default.protocol: https
        traefik.port: '443'
        traefik.tags: traefik-public
        traefik.docker.network: traefik-public
        # Traefik service that listens to HTTP
        traefik.redirectorservice.frontend.entryPoints: http
        traefik.redirectorservice.frontend.redirect.entryPoint: https
        # Traefik service that listens to HTTPS
        traefik.webservice.frontend.entryPoints: https


networks:
  traefik-public:
    external: true
    
volumes:
  zulip2_psql_data:
    external: true
  zulip2_rabbitmq_data:
    external: true  
  zulip2_app_data:
    external: true  
  zulip2_redis_data:
    external: true  

The installation of swarm.rocks and the swarmpit and portainer stacks seems to be working correctly with no issue. I just cannot bring up anything else.

[MerNat]

you should include these labels to the adminer service.
- traefik.redirectorservice.frontend.entryPoints=http
- traefik.redirectorservice.frontend.redirect.entryPoint=https
- traefik.webservice.frontend.entryPoints=https

[tiangolo]

When you get 5xx errors, it normally means that whatever is the application that is handling it is having an error.

In that case, I suggest you check the logs of that specific service, e.g. in Swarmpit.

When you get 404 errors, it's quite possible that the public Traefik is not being able to communicate to whatever service it needs to talk (it might be the intra-stack Traefik proxy). Or that an intra-stack Traefik proxy is not being able to communicate with the final service that handles that request.

About HTTPS and handling communication between different services, there would probably be only one single point in the chain that does HTTPS, has the certificates, etc. In this case (DockerSwarm.rocks) it would be the public Traefik proxy. For more details on all this HTTPS stuff, check: https://fastapi.tiangolo.com/deployment/#https

Hello, Sebastián,

we e-mailed a few days ago. Unfortunately I didn't get any further to start thelounge (https://hub.docker.com/r/thelounge/thelounge/), but like colleagues here I get the 404 error from traefik.

My thelounge.yml file looks like this:

version: '3.3'

services:
  thelounge:
    image: thelounge/thelounge:latest
    volumes:
      - thelounge:/data
      #- ~/data/thelounge:/var/opt/thelounge
    ports:
        - "4000:4000"
    labels:
      # - traefik.backend=thelounge
      - traefik.enable=true
      - traefik.frontend.rule=Host:${DOMAIN}
      - traefik.port=4000
      - traefik.docker.network=traefik-public
      - traefik.tags=traefik-public
      # Traefik service that listens to HTTP
      - traefik.redirectorservice.frontend.entryPoints=http
      - traefik.redirectorservice.frontend.redirect.entryPoint=https
      # Traefik service that listens to HTTPS
      - traefik.webservice.frontend.entryPoints=https      
    networks:
      #- web
      - traefik-public

networks:
  traefik-public:
    external: true

volumes:
  thelounge:

What did I do wrong? Can anyone please help me with my problem?

Edit: I also tried:

version: '3.3'

services:
  thelounge:
    image: thelounge/thelounge:latest
    volumes:
      - thelounge:/data
    labels:
      # - traefik.backend=thelounge
      - traefik.enable=true
      - traefik.frontend.rule=Host:${DOMAIN}
      - traefik.port=4000
      - traefik.docker.network=traefik-public
      - traefik.tags=traefik-public
      # Traefik service that listens to HTTP
      - traefik.redirectorservice.frontend.entryPoints=http
      - traefik.redirectorservice.frontend.redirect.entryPoint=https
      # Traefik service that listens to HTTPS
      - traefik.webservice.frontend.entryPoints=https      
    networks:
      #- web
      - traefik-public

networks:
  traefik-public:
    external: true

volumes:
  thelounge:

Also the 404 error appears. :-(

[tiangolo]

@noobcoder1983 check the logs for your service thelounge.

They might show an error somewhere.

If the logs show as if everything is working properly, make sure your service is up, that is hasn't exited.

Yeah I tried that. Here is the output:

3wnmgec 2019-05-10 17:10:37 [INFO] Configuration file created at /var/opt/thelounge/config.js.
3wnmgec 2019-05-10 17:10:38 [INFO] The Lounge v3.0.1 (Node.js 10.15.1 on linux x64)
3wnmgec 2019-05-10 17:10:38 [INFO] Configuration file: /var/opt/thelounge/config.js
3wnmgec 2019-05-10 17:10:38 [INFO] Available at http://:::9000/ in private mode
3wnmgec 2019-05-10 17:10:38 [INFO] New VAPID key pair has been generated for use with push subscription.
3wnmgec 2019-05-10 17:10:38 [INFO] There are currently no users. Create one with thelounge add <name>.

In my .yml file I defined the port to 4000 but with no effort. And @ my host there is no /var/opt/thelounge/config.js

And the service runs at port 9000. What I could do to solve that?

[rayrrr]

I am yet another user getting 404s when trying to bring up additional stacks for the websites I actually want to run. Not sure how to proceed.

[rayrrr]

Actually I was able to get this fixed for my use case...try putting your Traefik-related labels inside a deploy block like this where you currently have them:

    deploy:
      replicas: 1
      labels:
        - traefik.backend=thelounge
        - traefik.frontend.rule=Host:${DOMAIN}
        - traefik.enable=true
        - traefik.port=4000
        - traefik.tags=traefik-public
        - traefik.docker.network=traefik-public
        # Traefik service that listens to HTTP
        - traefik.redirectorservice.frontend.entryPoints=http
        - traefik.redirectorservice.frontend.redirect.entryPoint=https
        # Traefik service that listens to HTTPS
        - traefik.webservice.frontend.entryPoints=https

Ah ok I understand, but in this cases what makes the difference? The order of declarations?

[rayrrr]

What's most important is the indentation level. It has semantic meaning here, just like in Python.

[tiangolo]

Thanks @rayrrr for your help!

@noobcoder1983 in Docker Compose, under a specific service, there's a key labels that Traefik can use, ONLY in Docker Compose, let's say, at services.thelounge.labels.

For using Docker Swarm, those labels are read from services.thelounge.deploy.labels. With the deploy additional hierarchical level, as @rayrrr was explaining.

Also, you seem to have:

- traefik.port=4000, that tells Traefik to communicate with the thelounge service at port 4000, but by your logs, that service seems to be running at port 9000. So, I suspect you should have - traefik.port=9000.

Ok thanks so better to set port 9000 to avoid problems. Also portainer runs at port 9000, so I better change portainer port at portainer.yml file?

[tiangolo]

The - traefik.port=9000 tells Traefik at which port the container is listening to. But it doesn't publish that port outside. Traefik then exposes it at port 80 for standard HTTP and port 443 for HTTPS, each under its own domain.

You can have many containers/services that each listen at a specific port internally, as long as you don't publish/bind the same ports in the host.

Hmm so I can 14 containers behind traefik with for example the 9000 port without any problems? So portainer and thelounge will run in traefik both on port 9000?

[tiangolo]

I think we could close this issue now, right @vimal0777 ?

[github-actions]

Assuming the original issue was solved, it will be automatically closed now. But feel free to add more comments or create new issues.