Coder Social home page Coder Social logo

nginx-certbot's Introduction

Boilerplate for nginx with Let’s Encrypt on docker-compose

This repository is accompanied by a step-by-step guide on how to set up nginx and Let’s Encrypt with Docker.

init-letsencrypt.sh fetches and ensures the renewal of a Let’s Encrypt certificate for one or multiple domains in a docker-compose setup with nginx. This is useful when you need to set up nginx as a reverse proxy for an application.

Installation

  1. Install docker-compose.

  2. Clone this repository: git clone https://github.com/wmnnd/nginx-certbot.git .

  3. Modify configuration:

  • Add domains and email addresses to init-letsencrypt.sh
  • Replace all occurrences of example.org with primary domain (the first one you added to init-letsencrypt.sh) in data/nginx/app.conf
  1. Run the init script:

     ./init-letsencrypt.sh
    
  2. Run the server:

     docker-compose up
    

Got questions?

Feel free to post questions in the comment section of the accompanying guide

License

All code in this repository is licensed under the terms of the MIT License. For further information please refer to the LICENSE file.

nginx-certbot's People

Contributors

alexanderdushkin avatar jhyub avatar mestrogov avatar michal-wrzosek avatar pixep avatar wm222au avatar wmnnd avatar ynixon avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nginx-certbot's Issues

Where shall I put the html content ?

Sorry if this is a super dumb question but I failed to find where shall i put the html pages...
I tried on ./data/certbot/www after generating the certificates.

When I try to open in the browser it fails with "try to clean your cookies". I tried with 3 different web browsers and in incognito mode.
When I use curl to https site I get a 301 site moved permanently.
I reviewed all files, even carefully compared with the git repo...
Do you have any idea what's going on?
Site scores A at SSL Labs test anyways... but no webpage is shown.

only first domain works

Hi,

I tried the init script with 6 domains. Only the first one works because the dummy cert is only created for the first domain.

The script does this:

`echo "### Creating dummy certificate for $domains ..."
path="/etc/letsencrypt/live/$domains"
mkdir -p "$data_path/conf/live/$domains"
docker-compose run --rm --entrypoint "
openssl req -x509 -nodes -newkey rsa:1024 -days 1
-keyout '$path/privkey.pem'
-out '$path/fullchain.pem'
-subj '/CN=localhost'" certbot
echo

`

but should include a for i in loop of some kind like the script does use later

domain_args="" for domain in "${domains[@]}"; do domain_args="$domain_args -d $domain" done

unless I am mistaken of course

Connection refused

I know it is a duplicate of #10 .
But what "described" in that issue looks like not a valid solution.
Anyway this is the error

The following errors were reported by the server:

   Domain: mydomain.org
   Type:   connection
   Detail: Fetching
   http://mydomain.org/.well-known/acme-challenge/y1TyeXK2Il0hfnqROt-3Arn_6Oq28ffL4kshJZ50pic:
   Connection refused

this is all the log

matteo@domain:~/domain$ ./init-letsencrypth.sh
Existing data found for domain.com. Continue and replace existing certificate? (y/N) n
matteo@domain:~/domain$ ./init-letsencrypth.sh
Existing data found for domain.com. Continue and replace existing certificate? (y/N) y
### Creating dummy certificate for domain.com ...
Creating network "domain_default" with the default driver
Generating a RSA private key
..+++++
.............+++++
writing new private key to '/etc/letsencrypt/live/domain.com/privkey.pem'
req: Can't open "/etc/letsencrypt/live/domain.com/privkey.pem" for writing, No such file or directory

### Starting nginx ...
Creating domain ... done

### Deleting dummy certificate for domain.com ...

### Requesting Let's Encrypt certificate for domain.com ...
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for admin.domain.com
http-01 challenge for domain.com
http-01 challenge for www.admin.domain.com
http-01 challenge for www.domain.com
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. www.domain.com (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain ...

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: www.domain.com
   Type:   connection
   Detail: Fetching
   http://www.domain.com/.well-known/acme-challenge/CksLQlZjt7xYEf683LHRV87PRAnF9KglD3Yhpw9DESk:
   Connection refused

   Domain: admin.domain.com
   Type:   connection
   Detail: Fetching
   http://admin.domain.com/.well-known/acme-challenge/o6XfSqjxuzupQiiQ_DJ-mUUWmMTgZvyLZKFrZ1RAd7s:
   Connection refused

   Domain: www.admin.domain.com
   Type:   connection
   Detail: Fetching
   http://www.admin.domain.com/.well-known/acme-challenge/QmlFWKvP8L_OE3R5tI0BPG3fjcOwodFpNOT0beNJitE:
   Connection refused

   Domain: domain.com
   Type:   connection
   Detail: Fetching
   http://domain.com/.well-known/acme-challenge/R3RlQ30GFfQQ4yck1l3sA252w-mLy2tNo0DJa_s_wvc:
   Connection refused

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

### Reloading nginx ...
2019/03/27 02:55:14 [notice] 7#7: signal process started

I have notice this, maybe it is the source of the error

req: Can't open "/etc/letsencrypt/live/domain.com/privkey.pem" for writing, No such file or directory

If you need more info I could give it to you.
Thanks

[Question] standalone mode

I recently found out about the certbot standalone mode and was thinking if the nginx setup is a bit over engineered?

How to init cert on local machine?

Im testing your script to build my react app.
Is it possible to setup certbot on local machine (or CI server)?
Currently I'v got error Challenge failed for domain MY_DOMAIN

`IMPORTANT NOTES:

  • The following errors were reported by the server:

    Domain: MY_DOMAIN
    Type: unauthorized
    Detail: Invalid response from
    https://MY_DOMAIN/.well-known/acme-challenge/eGeqlLCWOxrzVHX-QrW8kUvkfS7iE9E0ck2vzTdDFSk
    [194.61.1.107]: "\n<html lang="en">\n\n<meta
    charset="utf-8">\n<title>Error</title>\n\n\n

    Cannot
    GET /.well-known/"

    To fix these errors, please make sure that your domain name was
    entered correctly and the DNS A/AAAA record(s) for that domain
    contain(s) the right IP address.
    `

ERR_TOO_MANY_REDIRECTS

Hi, I finished following your tutorial but when I go to my URL, I get the ERR_TOO_MANY_REDIRECTS issue. Did someone already encounter this issue and could fix it?

I don't know which files I should share with you, so don't hesitate to ask ;).

Thanks in advance,
Best,

Why use "sleep 6h & wait $${!}"?

In the service startup commands, there is:
command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"

Two questions:

  • Why are there two dollar signs? Is the first one needed to escape the second for some reason?
  • Why not just do "sleep 6h" directly instead of sleep and then wait?

What if the docker-compose file has a different name

Hi Philipp,

I followed your tutorial about getting the certificates up and running, and it's great. But I spent quite some time figuring out why it did not work for me initially. Turned out that it was because I have two different docker-compose files, "docker-compose.yml" for development and "docker-compose.prod.yml" for production. This made the init-letsencrypt.sh fail, as nginx and certbot does not exist in my development environment. So I had to change all the "docker-compose" commands to "docker-compose -f docker-compose.prod.yml" in init-letsencrypt.sh.

So I would propose that it is briefly mentioned alongside the following section to save others the headache I got, what's your thoughts?

Edit the script to add in your domain(s) and your email address. If you’ve changed the directories of the shared Docker volumes, make sure you also adjust the data_path variable as well.

ERROR: No container found for nginx_1

This problem just came up when I modified the nginx configuation app.conf then I ran sudo ./init-letsencrypt.sh and get this respond

### Reloading nginx ...
ERROR: No container found for nginx_1

Too see container status I ran sudo docker ps -a and it seem like all of them are exited. Do you have any suggestion?

CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS                      PORTS                        NAMES
edbc1c9a955a        nginx:1.15-alpine   "nginx -g 'daemon of…"   About a minute ago   Exited (1) 58 seconds ago                                nginx-certbot_nginx_1
b83cc25a7a82        certbot/certbot     "certbot"                About a minute ago   Exited (1) 57 seconds ago                                nginx-certbot_certbot_1

Attempting to renew cert (domain.com) from /etc/letsencrypt/renewal/domain.com.conf produced an unexpected error:

When expire the cert, the certbot try renew them and show this error:

Cert is due for renewal, auto-renewing...
Non-interactive renewal: random delay of 190 seconds
Plugins selected: Authenticator webroot, Installer None
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for domain.com
Cleaning up challenges
Attempting to renew cert (domain.com) from /etc/letsencrypt/renewal/domain.com.conf produced an unexpected error: Missing command line flag or config entry for this setting:
Input the webroot for domain.com:. Skipping.

This is working but when expired not renew more...
I suppose the problem is with webroot dir

[2.x, 1.x?] Certbot shuffled files

diff --git a/init-letsencrypt.sh b/init-letsencrypt.sh
index f282b37..154c48e 100755
--- a/init-letsencrypt.sh
+++ b/init-letsencrypt.sh
@@ -1,5 +1,5 @@
 #!/bin/bash
-set -e
+set -ex
 
 domains=(example.com www.example.com) # Specify domains here or use the -d argument
 data_path="./data/certbot" # Specify data path here or use the --data-path argument
@@ -80,8 +80,8 @@ fi
 if [ ! -e "$data_path/conf/options-ssl-nginx.conf" ] || [ ! -e "$data_path/conf/ssl-dhparams.pem" ]; then
   echo "### Downloading recommended TLS parameters ..."
   mkdir -p "$data_path/conf"
-  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/tls_configs/options-ssl-nginx.conf > "$data_path/conf/options-ssl-nginx.conf"
-  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot/ssl-dhparams.pem > "$data_path/conf/ssl-dhparams.pem"
+  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/_internal/tls_configs/options-ssl-nginx.conf > "$data_path/conf/options-ssl-nginx.conf"
+  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot/certbot/ssl-dhparams.pem > "$data_path/conf/ssl-dhparams.pem"
   echo
 fi

http-01 challenge failed

### Deleting dummy certificate for myrealdomain.com ...

### Requesting Let's Encrypt certificate for myrealdomain.com ...
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for myrealdomain.com
http-01 challenge for www.myrealdomain.com
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Challenge failed for domain myrealdomain.com
Challenge failed for domain www.myrealdomain.com
http-01 challenge for myrealdomain.com
http-01 challenge for www.myrealdomain.com
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: myrealdomain.com
   Type:   connection
   Detail: Fetching
   http://myrealdomain.com/.well-known/acme-challenge/Xs94VXCMCvMdGrz6QEWDSWrRH86ISG4x3FA6yetMRyw:
   Connection refused

   Domain: www.myrealdomain.com
   Type:   connection
   Detail: Fetching
   http://www.myrealdomain.com/.well-known/acme-challenge/QW9SYinn_8V6y5apuh4lQwa4ny7R3bcWadWIePNIaI4:
   Connection refused

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

### Reloading nginx ...
Error response from daemon: Container fa728312c6a2d9e5958505f3f8b973aafb46f01969ae9134cc0e33977060b86e is restarting, wait until the container is running

I created and successfully ran my flask app container, and am able to access the app through myrealdomain.com:5000, but not myrealdomain.com. I'm pretty sure the domain and the DNS A/AAAA record(s) is correct.

nginx conf

server {
    listen 80;
    server_name myrealdomain.com www.myrealdomain.com;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name myrealdomain.com www.myrealdomain.com;
    server_tokens off;

    ssl_certificate /etc/letsencrypt/live/example.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.org/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    location / {
        proxy_pass http://web:5000;  # flask service name: web
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

I have no idea how to debug this issue. What could possibly go wrong in this case?

Problem on .tirol Domain

Hello,

i have installed you script and it works perfekt for my .info domain. The problem on the .tirol domain is that the folder structere for the certificate is not created.

I have used exactly the same file with the difference.

The failor is:

Reloading nginx ...

2019/09/11 19:43:43 [emerg] 9#9: cannot load certificate "/etc/letsencrypt/live/moses.tirol/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/moses.tirol/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
nginx: [emerg] cannot load certificate "/etc/letsencrypt/live/moses.tirol/fullchain.pem": BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/moses.tirol/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

Dummy key size is too small

I haven't tracked down when this was introduced, but the tutorial is currently broken because the generated dummy key is 1024 bit and nginx gives a 'key too small' error. Probably that parameter should just reference the rsa_key_size variable so they stay in sync.

It'd also be nice if there was some logic to check the nginx container didn't crash before the challenges are attempted, this took me a bit to diagnose.

Unable to generate new certificate

It won't pass the time to generate new certifications. Doing from my computer, everything works but, since it's not the server, it fails to check the acme-challenge, but that's ok.

From my machine in ec2, I get the following output:

HTTPSConnectionPool(host='acme-v02.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fc130980d30>: Failed to establish a new connection: [Errno -3] Try again'))

initLetsencrypt.sh downloads recommended settings twice

Hello,

the initLetsencrypt.sh downloads the recommended settings twice from github. First in Line 25 and then again in Line 37.

Wouldn't it make sense to download them only once to a temporary directory and then copy them over?
Or at least print the same info message for both?
Or is there any difference that I have overlooked?

Kind Regards

Add ability to force nginx to re-read configs

hi Philipp, thank you for this repo and article !

Is it possible to force nginx to re-read configs?
Before I added
command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"
to docker-compose nginx, I was able to force nginx to re-read configs using this command line
docker kill -s HUP nginx
But now it doesn't work. I have to restart the container.

Don't affect linked containers

When have multiples container with 'links' in docker-compose don't have necessary recreat all containers, only the nginx. If you find it interesting, you can add parameter --no-deps in step 'Starting nginx', with purpose to don't recreat the linked containers.

multiple subdomains question

Been struggling all day with getting this setup to work with two different subdomains which both need to accept https traffic (for a flask site/development site for same) - need to see if anyone has any insight. I'm getting the following errors when the first challenge is attempted while running the script, with some modifications.

On the first cert verification attempt:

Waiting for verification...
Challenge failed for domain my.example.domain
http-01 challenge for my.example.domain
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: my.example.domain
   Type:   connection
   Detail: Fetching
   http://my.example.domain/.well-known/acme-challenge/Z*****************0:
   Connection refused

	... blah blah A records, etc ...

### Reloading nginx ...
cannot exec in a stopped state: unknown

In docker-compose up output - the reason nginx fails above, presumably?

nginx_1 | <timestamp-here> [emerg] 1#1: open() "/etc/letsencrypt/options-ssl-nginx.conf" failed (2: No such file or directory) in /etc/nginx/conf.d/app.conf:22

nginx_1 | nginx: [emerg] open() "/etc/letsencrypt/options-ssl-nginx.conf" failed (2: No such file or directory) in /etc/nginx/conf.d/app.conf:22

My Desired Setup:

  • NGINX reverse proxy to accept/redirect all http/s traffic to SSL to both: "my.example.com" and "my-dev.example.com"
  • two basically identical containers, "web" and "web-dev", which are flask apps running on 5000:80 and 5001:80.
  • I want to bring it all up at once with the script and then docker-compose up, issuing separate certs to each (I'm fine with changing it to a single cert for both if that's possible/easier for some reason)

I've tried everything I could find from searching anything remotely related to the issue and suspect it's more a fundamental "not understanding NGINX config" issue on my part than a bug, but I hope someone can help and this question can be a reference for anyone else similarly stuck. I think there might be a solution in the nginx.conf file, having to do with adding some upstream entries, possibly, but I don't quite understand where to begin.

All my code is below, thank you in advance!

my nginx.conf

server {
    listen 80;
    server_name my.example.domain;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name my.example.domain;
    server_tokens off;

    ssl_certificate /etc/letsencrypt/live/my.example.domain/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/my.example.domain/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    location / {
        proxy_pass  http://web:5000;
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    }
}

server {
    listen 80;
    server_name my-dev.example.domain;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name my-dev.example.domain;
    server_tokens off;

    ssl_certificate /etc/letsencrypt/live/my-dev.example.domain/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/my-dev.example.domain/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    location / {
        proxy_pass  http://web-dev:5001;
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    }
}

my init-letsencrypt.sh

#!/bin/bash

domains=(my.example.domain my-dev.example.domain)
rsa_key_size=4096
data_path="./data/certbot"
email="[email protected]" # Adding a valid address is strongly recommended
staging=1 # Set to 1 if you're testing your setup to avoid hitting request limits

if [ -d "$data_path" ]; then
  read -p "Existing data found for $domains. Continue and replace existing certificate? (y/N) " decision
  if [ "$decision" != "Y" ] && [ "$decision" != "y" ]; then
    exit
  fi
fi


if [ ! -e "$data_path/conf/options-ssl-nginx.conf" ] || [ ! -e "$data_path/conf/ssl-dhparams.pem" ]; then
  echo "### Downloading recommended TLS parameters ..."
  mkdir -p "$data_path/conf"
  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/tls_configs/options-ssl-nginx.conf > "$data_path/conf/options-ssl-nginx.conf"
  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot/ssl-dhparams.pem > "$data_path/conf/ssl-dhparams.pem"
  echo
fi

echo "### Creating dummy certificate for $domains ..."
path="/etc/letsencrypt/live/$domains"
mkdir -p "$data_path/conf/live/$domains"
docker-compose run --rm --entrypoint "\
  openssl req -x509 -nodes -newkey rsa:1024 -days 1\
    -keyout '$path/privkey.pem' \
    -out '$path/fullchain.pem' \
    -subj '/CN=localhost'" certbot
echo


echo "### Starting nginx ..."
docker-compose up --force-recreate -d nginx #tried this both ways
echo

echo "### Deleting dummy certificate for $domains ..."
docker-compose run --rm --entrypoint "\
  rm -Rf /etc/letsencrypt/live/$domains && \
  rm -Rf /etc/letsencrypt/archive/$domains && \
  rm -Rf /etc/letsencrypt/renewal/$domains.conf" certbot
echo


echo "### Requesting Let's Encrypt certificate for $domains ..."
#Join $domains to -d args
domain_args=""
for domain in "${domains[@]}"; do
  domain_args="$domain_args -d $domain"
done

# Select appropriate email arg
case "$email" in
  "") email_arg="--register-unsafely-without-email" ;;
  *) email_arg="--email $email" ;;
esac

# Enable staging mode if needed
if [ $staging != "0" ]; then staging_arg="--staging"; fi

docker-compose run --rm --entrypoint "\
  certbot certonly --webroot -w /var/www/certbot \
    $staging_arg \
    $email_arg \
    $domain_args \
    --rsa-key-size $rsa_key_size \
    --agree-tos \
	--eff-email \  ##"I added this and it should have no weird effect"
    --force-renewal" certbot
echo

echo "### Reloading nginx ..."
docker-compose exec nginx nginx -s reload

my docker-compose

version: '3'

services:
  nginx:
    image: nginx:stable-alpine
    restart: unless-stopped
    volumes:
      - ./data/nginx:/etc/nginx/conf.d
      - ./data/certbot/conf:/etc/letsencrypt
      - ./data/certbot/www:/var/www/certbot
      - ./data/nginx/file_size.conf:/etc/nginx/file_size.conf # I added this, unrelated?
    ports:
      - "80:80"
      - "443:443"
    links:
      - web
      - web-dev
    command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"


  certbot:
    image: certbot/certbot
    restart: unless-stopped
    volumes:
      - ./data/certbot/conf:/etc/letsencrypt
      - ./data/certbot/www:/var/www/certbot
    entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"


  web:
    build:
      context: .
      dockerfile: prod.Dockerfile
    restart: unless-stopped
    volumes:
      - ./data/web:/data/web
    ports:
      - "5000:80"


  web-dev:
    build:
      context: .
      dockerfile: dev.Dockerfile
    restart: unless-stopped
    volumes:
      - ./data/web-dev:/data/web-dev
    ports:
      - "5001:80"
    environment:
      FLASK_ENV: development

my 'web' container dockerfile

FROM python:3.7-slim-buster

# handle static files
ENV STATIC_URL /static

# if making changes this should be an absolute path
ENV STATIC_PATH /data/web/static

# set working directory and import /data/web/*
COPY ./data/web /data/web
WORKDIR /data/web

# Make /data/web* available to be imported by Python globally
ENV PYTHONPATH=/data/web

ENV FLASK_APP /data/web/app.py
ENV FLASK_RUN_HOST 0.0.0.0

# install requirements 
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .
CMD ["flask", "run"]

my 'web-dev' container dockerfile

FROM python:3.7-slim-buster

# handle static files
ENV STATIC_URL /static

# if making changes this should be an absolute path
ENV STATIC_PATH /data/web-dev/static

# set working directory and import /data/web-dev/*
COPY ./data/web-dev /data/web-dev
WORKDIR /data-dev/web

# Make /data/web-dev* available to be imported by Python globally
ENV PYTHONPATH=/data/web-dev

ENV FLASK_APP /data/web-dev/app.py
ENV FLASK_RUN_HOST 0.0.0.0

# install requirements 
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

COPY . .
CMD ["flask", "run"]

Connection refused

Successfully built 584500ec72e1
Successfully tagged flask-docker-container-boilerplate_web:latest
WARNING: Image for service web was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating flask-docker-container-boilerplate_web_1 ... done

### Deleting dummy certificate for meem.no ...

### Requesting Let's Encrypt certificate for meem.no ...
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for meem.no
http-01 challenge for www.meem.no
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Challenge failed for domain meem.no
Challenge failed for domain www.meem.no
http-01 challenge for meem.no
http-01 challenge for www.meem.no
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: meem.no
   Type:   connection
   Detail: Fetching
   http://meem.no/.well-known/acme-challenge/qPqfKmnb_0Khm89bPyQnywU7D1ufzMp0JUQ7nX6m69w:
   Connection refused

   Domain: www.meem.no
   Type:   connection
   Detail: Fetching
   http://www.meem.no/.well-known/acme-challenge/axhzfLr7JZtePGupyRW-zNPwuNYxb99txR7WKZXY8D8:
   Connection refused

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

### Reloading web ...
nginx: [emerg] BIO_new_file("/etc/letsencrypt/live/meem.no/fullchain.pem") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/meem.no/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)

There were too many requests of a given type :: Error creating new order :: too many certificates already issued for exact set of domains

I encounter after changing the staging from 1 to 0, do you happen to know how to fix this because I only request once.

### Requesting Let's Encrypt certificate for snickies.com ...
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
An unexpected error occurred:
There were too many requests of a given type :: Error creating new order :: too many certificates already issued for exact set of domains: snickies.com,www.snickies.com: see https://letsencrypt.org/docs/rate-limits/
Please see the logfiles in /var/log/letsencrypt for more details.

Did not work with multiple domains; here's a fix

I did not find that the init script worked if I had multiple domains; also I found that I needed to parameterize it to reflect staging/production environments that use different docker-compose files and domains.

Here's what I came up with -- if you like I can make a PR.

#!/bin/bash
die () {
    echo >&2 "$@"
    exit 1
}

[ "$#" -ge 2 ] || die "usage: init-letsencrypt.sh [staging | production] <domain1 domain2 ...> "

domains=(${@:2})
rsa_key_size=4096
data_path="/etc/certbot"
email="[email protected]" # Adding a valid address is strongly recommended
staging=0 # Set to 1 if you're testing your setup to avoid hitting request limits


if [ -d "$data_path/conf/live/" ]; then
  read -p "Existing data found. Continue and replace existing certificates? (y/N) " decision
  if [ "$decision" != "Y" ] && [ "$decision" != "y" ]; then
    exit
  fi
fi

if [ ! -e "$data_path/conf/options-ssl-nginx.conf" ] || [ ! -e "$data_path/conf/ssl-dhparams.pem" ]; then
  echo "### Downloading recommended TLS parameters ..."
  mkdir -p "$data_path/conf"
  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/options-ssl-nginx.conf > "$data_path/conf/options-ssl-nginx.conf"
  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot/ssl-dhparams.pem > "$data_path/conf/ssl-dhparams.pem"
  echo
fi

for domain in "${domains[@]}"; do
  echo "### Removing old certificate for $domain ..."
  docker-compose -f docker-compose.yml -f docker-compose.$1.yml run --rm --entrypoint "\
    rm -Rf /etc/letsencrypt/live/$domain && \
    rm -Rf /etc/letsencrypt/archive/$domain && \
    rm -Rf /etc/letsencrypt/renewal/$domain.conf" certbot
  echo
done

for domain in "${domains[@]}"; do
  echo "### Creating dummy certificate for $domain ..."
  path="/etc/letsencrypt/live/$domain"
  mkdir -p "$data_path/conf/live/$domain"
  docker-compose -f docker-compose.yml -f docker-compose.$1.yml run --rm --entrypoint "\
    openssl req -x509 -nodes -newkey rsa:1024 -days 1\
      -keyout "$path/privkey.pem" \
      -out "$path/fullchain.pem" \
      -subj '/CN=localhost'" certbot
  echo
done

echo "### Starting nginx ..."
docker-compose -f docker-compose.yml -f docker-compose.$1.yml up --force-recreate -d nginx
echo

for domain in "${domains[@]}"; do
  echo "### Removing dummy certificate for $domain ..."
  docker-compose -f docker-compose.yml -f docker-compose.$1.yml run --rm --entrypoint "\
    rm -Rf /etc/letsencrypt/live/$domain" certbot
  echo
done

echo "### Requesting Let's Encrypt certificates ..."

# Select appropriate email arg
case "$email" in
  "") email_arg="--register-unsafely-without-email" ;;
  *) email_arg="--email $email" ;;
esac

# Enable staging mode if needed
if [ $staging != "0" ]; then staging_arg="--staging"; fi

for domain in "${domains[@]}"; do
  docker-compose -f docker-compose.yml -f docker-compose.$1.yml run --rm --entrypoint "\
    certbot certonly --webroot -w /var/www/certbot \
      $staging_arg \
      $email_arg \
      -d $domain \
      --rsa-key-size $rsa_key_size \
      --agree-tos \
      --force-renewal" certbot
  echo
done

echo "### Reloading nginx ..."
docker-compose -f docker-compose.yml -f docker-compose.$1.yml exec nginx nginx -s reload

certbot.errors.AuthorizationError: Some challenges have failed for review.

Hello,

Thank you for the detailed tutorial, but I'm having a weird issue.
After a lot of tries, I managed to get this verbose log:

Existing data found for xxx.lorem.com. Continue and replace existing certificate? (y/N) y
### Creating dummy certificate for xxx.lorem.com ...
Generating a RSA private key
..........................................+++++
...............+++++
writing new private key to '/etc/letsencrypt/live/xxx.lorem.com/privkey.pem'
-----

### Starting nginx ...
Recreating xxx-nginx ... 
Recreating xxx-nginx ... done

### Deleting dummy certificate for xxx.lorem.com ...

### Requesting Let's Encrypt certificate for xxx.lorem.com ...
Root logging level set at 10
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requested authenticator webroot and installer None
Single candidate plugin: * webroot
Description: Place files in webroot directory
Interfaces: IAuthenticator, IPlugin
Entry point: webroot = certbot.plugins.webroot:Authenticator
Initialized: <certbot.plugins.webroot.Authenticator object at 0x7f4e79936e10>
Prep: True
Selected authenticator <certbot.plugins.webroot.Authenticator object at 0x7f4e79936e10> and installer None
Plugins selected: Authenticator webroot, Installer None
Picked account: <Account(RegistrationResource(body=Registration(key=None, contact=(), agreement=None, status=None, terms_of_service_agreed=None, only_return_existing=None, external_account_binding=None), uri='https://acme-staging-v02.api.letsencrypt.org/acme/acct/11519950', new_authzr_uri=None, terms_of_service=None), 803e21903d9d8cfa20499d0bb2bd2586, Meta(creation_dt=datetime.datetime(2019, 11, 8, 15, 32, 18, tzinfo=<UTC>), creation_host='13b6b54e68c0'))>
Sending GET request to https://acme-staging-v02.api.letsencrypt.org/directory.
Starting new HTTPS connection (1): acme-staging-v02.api.letsencrypt.org:443
https://acme-staging-v02.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 724
Received response:
HTTP 200
Server: nginx
Date: Fri, 08 Nov 2019 16:33:48 GMT
Content-Type: application/json
Content-Length: 724
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "21IRDSaUJLI": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
  "keyChange": "https://acme-staging-v02.api.letsencrypt.org/acme/key-change",
  "meta": {
    "caaIdentities": [
      "letsencrypt.org"
    ],
    "termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf",
    "website": "https://letsencrypt.org/docs/staging-environment/"
  },
  "newAccount": "https://acme-staging-v02.api.letsencrypt.org/acme/new-acct",
  "newNonce": "https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce",
  "newOrder": "https://acme-staging-v02.api.letsencrypt.org/acme/new-order",
  "revokeCert": "https://acme-staging-v02.api.letsencrypt.org/acme/revoke-cert"
}
Obtaining a new certificate
Generating key (4096 bits): /etc/letsencrypt/keys/0015_key-certbot.pem
Creating CSR: /etc/letsencrypt/csr/0015_csr-certbot.pem
Requesting fresh nonce
Sending HEAD request to https://acme-staging-v02.api.letsencrypt.org/acme/new-nonce.
https://acme-staging-v02.api.letsencrypt.org:443 "HEAD /acme/new-nonce HTTP/1.1" 200 0
Received response:
HTTP 200
Server: nginx
Date: Fri, 08 Nov 2019 16:33:49 GMT
Connection: keep-alive
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 0002tUBK5GOg9sYqNGtZVk8sAu5CvI2kCg6CZ2ubFK7OzKU
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800


Storing nonce: 0002tUBK5GOg9sYqNGtZVk8sAu5CvI2kCg6CZ2ubFK7OzKU
JWS payload:
b'{\n  "identifiers": [\n    {\n      "type": "dns",\n      "value": "xxx.lorem.com"\n    }\n  ]\n}'
Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/new-order:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8xMTUxOTk1MCIsICJub25jZSI6ICIwMDAydFVCSzVHT2c5c1lxTkd0WlZrOHNBdTVDdkkya0NnNkNaMnViRks3T3pLVSIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9uZXctb3JkZXIifQ",
  "signature": "eUu7ijFJznEuhHIbsGZXEIFMVnVGQ7q3B_afLKLkdEeHreuggZq9GgBycDZrsbMHxojVkwwRdFV59tSL6uJ_PLYnsat9nsRgU9EDnFpCgUlXwvPiek43_yXVczB2VHtgYdx8bL1jxnAWUCwlJVmfMy-Y1Jptjn10-Jvolsemr5-jQrn5P7Be1-ibVqi7iqA8rYY8ZeWw7YJdpaF3lR_i1VBQsaLcnE4-fGIn7CTDMdOKqZGlTY4ac-41dDpHtzZ7yQV3yU6nBaXakenBoubInvctfc5T6jQ_HVchKW1R2Qoysy1Hki-j24t2cJaet7BZwyNV-mF3lN6-BGWfbz7qQzfPdoFFzDaPNmPBsnu1y1sBmtTBgHCJ1YjQQTbYmACwIvcTRzGTUbqLFS0avjCXvZOsK08j6yaDP3uSvpPCB9qaV2v_7_Lqlcir66i0--EiVHEKvXiFVe345EudGg6Q9_sjw93FqpMo3TAKpOuLYRZQIZJxfnpeCCTt8jCREcNUUFp05Q17pCf03pv25F2EK_g61HLdrDUQNva8IvtNMEzm6HQUE23HfNRyFHogYODUlAGMB2ygAdFUhje9SA4YgWXwCvDKSrct-EN4bODp8hfLjfhHbx1Gp5__-8IkKPNNyKAnxvggI9iJYSnazOwsog0gquCrljmjNzDwctZNsPI",
  "payload": "ewogICJpZGVudGlmaWVycyI6IFsKICAgIHsKICAgICAgInR5cGUiOiAiZG5zIiwKICAgICAgInZhbHVlIjogInByb2pvLm9jdXMuY29tIgogICAgfQogIF0KfQ"
}
https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/new-order HTTP/1.1" 201 355
Received response:
HTTP 201
Server: nginx
Date: Fri, 08 Nov 2019 16:33:49 GMT
Content-Type: application/json
Content-Length: 355
Connection: keep-alive
Boulder-Requester: 11519950
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Location: https://acme-staging-v02.api.letsencrypt.org/acme/order/11519950/60509153
Replay-Nonce: 0002d_thn50OPYUbaGIfCqJFbisnxyD1Hy19kLbjXooClNg
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "status": "pending",
  "expires": "2019-11-15T16:33:49.73815071Z",
  "identifiers": [
    {
      "type": "dns",
      "value": "xxx.lorem.com"
    }
  ],
  "authorizations": [
    "https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/19815850"
  ],
  "finalize": "https://acme-staging-v02.api.letsencrypt.org/acme/finalize/11519950/60509153"
}
Storing nonce: 0002d_thn50OPYUbaGIfCqJFbisnxyD1Hy19kLbjXooClNg
JWS payload:
b''
Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/19815850:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8xMTUxOTk1MCIsICJub25jZSI6ICIwMDAyZF90aG41ME9QWVViYUdJZkNxSkZiaXNueHlEMUh5MTlrTGJqWG9vQ2xOZyIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei12My8xOTgxNTg1MCJ9",
  "signature": "KZ7llK_j4mZIYamX7xhVkHSUnavih30MWEY4Kl_mzcYdNHtOd_hGSNG0FFU3ZsVFuu4XIzmfUyIW3esdSIHllOM41DdC_HVe8d0sIxuN75V7cs59ltAPMCZoeiG64XBwxJDsBtDV2WdJ35DVv7UPqCfcX9Ntw0T9LkooCQaRGJCstAPlx2MiVb2uf6gJIPqY8-O4thKUA-_DbzMDeew0PDMtlHYHa_qQZOvaajgY4vTFtF188Q8DsfEyLASpkony3f_Hlw7S6IuDDrZhLUyAbvKgwJDeTkYOOyiizxzf8JnEvEG4JyKNZsfLhMCxst1W4dDYlbESYXAKQSY_bBhbFaDCzVAOQ_T5--Cc3iq8MSVITyB4h0vGq7dJHVJloobTBDg7azuNN3ERnsZLuIKO0_wFQ7HoGyFP_2AQ8VPKgCDP6MXf34Whv9O9InciBuAO8QKkNrpxBmLlrXD8ndcDQ53pD6gtlStvBIADHuee2L9zxoqIu5sOxmbmAWCHIPCQDYDIOTbad3WPYS6N6aiIdjoNwTbtN-bXys5CzYfeSw9nRugyWz9geTg9MhpYqo_HIWqFQfVMW-3ER2g7kwc-IYTb_jZt2MdE5IW_ydJEq7Q7M_vcEvGOLuYfpZjE74deaHl5NSiw2JUW6mYtSylWxXgxRVdChhRZtjh5UlspHWU",
  "payload": ""
}
https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/19815850 HTTP/1.1" 200 810
Received response:
HTTP 200
Server: nginx
Date: Fri, 08 Nov 2019 16:33:50 GMT
Content-Type: application/json
Content-Length: 810
Connection: keep-alive
Boulder-Requester: 11519950
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 0002pQ68YQCYousjJoEJZ1sTVmzmaAAtuPCBHN7FLfOiQU4
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "xxx.lorem.com"
  },
  "status": "pending",
  "expires": "2019-11-15T16:33:49Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/19815850/PmKNmQ",
      "token": "dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/19815850/xRpPTQ",
      "token": "dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U"
    },
    {
      "type": "tls-alpn-01",
      "status": "pending",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/19815850/-6VPpg",
      "token": "dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U"
    }
  ]
}
Storing nonce: 0002pQ68YQCYousjJoEJZ1sTVmzmaAAtuPCBHN7FLfOiQU4
Performing the following challenges:
http-01 challenge for xxx.lorem.com
Using the webroot path /var/www/certbot for all unmatched domains.
Creating root challenges validation dir at /var/www/certbot/.well-known/acme-challenge
Attempting to save validation to /var/www/certbot/.well-known/acme-challenge/dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U
Waiting for verification...

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Challenges loaded. Press continue to submit to CA. Pass "-v" for more info about
challenges.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Press Enter to Continue
JWS payload:
b'{\n  "resource": "challenge",\n  "type": "http-01"\n}'
Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/19815850/PmKNmQ:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8xMTUxOTk1MCIsICJub25jZSI6ICIwMDAycFE2OFlRQ1lvdXNqSm9FSloxc1RWbXptYUFBdHVQQ0JITjdGTGZPaVFVNCIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9jaGFsbC12My8xOTgxNTg1MC9QbUtObVEifQ",
  "signature": "FDYkuxb4R5dc1ZBRJfezgMw2D_OSt7y4nAqoplXs4F0D2N8Esvnl9hiVFhd3MlbbYLdsVz-_EQ1vbnc0NPUzVVK1MtFI_w-3aMNuIJnGIAgn1coIffZZKfha8mQD9gRffhuIFUP8xVDMNR8rcpsN9WNiHnDeuz_pUJqOGGDiqGy0-SITfiCff9cTKqYOykbgsPsfBO4taY1KHq1nGP2H6pQvlboEu4_ALKzH7UUP3UaHxejq5FLLrHYLqHMsVqLSegJU4OjxrxCMl3g6_ygrNunrO36ctgiIgN9WNgld4I5mOOEGkfz6UVjvmfGmKwlWl7fcqsGyy-9jBgG_bgvPbBFX7YRsahdBvlb7nJaLNs5lBqFWUCgnGLO0PuMbDgVTqFUwrcGM91UerwV66GK2AItNxvV9DAcXRGorVnYEindt22p7OsbXA8iO50X-Hegp0ficXfLuBKnbc5IxW0R_o5ew8eU3nQ27170DpEE5wh65exlQJgm1-XYItbDL00XdpxkVcGkvE3jhQpuvwSWv1MGgxkuVU2uV3eSPw6wB0iQxHHzk6vl5kD9GbtC7Lqx9TaadkLHWQm1rl7PjHjJS5O5cYvIWqlUaDFVOfyABub-J5Dd_FHIWHjKNZz8hFIW8AyNOsxP50VotU9xN5ypc5m4OkLfJWFuXSuOCBqwRdp0",
  "payload": "ewogICJyZXNvdXJjZSI6ICJjaGFsbGVuZ2UiLAogICJ0eXBlIjogImh0dHAtMDEiCn0"
}
https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/chall-v3/19815850/PmKNmQ HTTP/1.1" 200 191
Received response:
HTTP 200
Server: nginx
Date: Fri, 08 Nov 2019 16:35:08 GMT
Content-Type: application/json
Content-Length: 191
Connection: keep-alive
Boulder-Requester: 11519950
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index", <https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/19815850>;rel="up"
Location: https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/19815850/PmKNmQ
Replay-Nonce: 0002uVhZwVzdVbFWDpcL52c_gOmusK7spQSTfRTrB1N4I9o
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "type": "http-01",
  "status": "pending",
  "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/19815850/PmKNmQ",
  "token": "dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U"
}
Storing nonce: 0002uVhZwVzdVbFWDpcL52c_gOmusK7spQSTfRTrB1N4I9o
JWS payload:
b''
Sending POST request to https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/19815850:
{
  "protected": "eyJhbGciOiAiUlMyNTYiLCAia2lkIjogImh0dHBzOi8vYWNtZS1zdGFnaW5nLXYwMi5hcGkubGV0c2VuY3J5cHQub3JnL2FjbWUvYWNjdC8xMTUxOTk1MCIsICJub25jZSI6ICIwMDAydVZoWndWemRWYkZXRHBjTDUyY19nT211c0s3c3BRU1RmUlRyQjFONEk5byIsICJ1cmwiOiAiaHR0cHM6Ly9hY21lLXN0YWdpbmctdjAyLmFwaS5sZXRzZW5jcnlwdC5vcmcvYWNtZS9hdXRoei12My8xOTgxNTg1MCJ9",
  "signature": "EIKVD-rBIyCXBwsMk-w7Ejs_VgAbcp5ry2oKvSFKnP9T6ze66aq-rk-SAMZAWyrOObarYAB7jMZGNCJzk2-ODIIVQBACY2LXxvwfWj42IItIf77NyFc6dIUgw17r-3fPLMpWCR7meTyzTVDzVjzqJsn5bQw4pGTlXvAPl_6BtH8MB4d9qKokOYWd19qVSRRFpygm6ITLCTaqryl-t-lb5XFRFVrYtkChRetdyhr8WXHgEbO8D59wuoYYLN_W3R2MOlbb_qBKYmTs6ekxH00ofD-nx13HHi-lqskbFT0sjG6Mtqe8yM4ui8KWGipG5Gacwme3_YW_Lf2cJDgHStSyVJacegDQa05fePwYEiZfHCEw6fkChNyxT0iGKDOdTUIf7tYLpMFIBi341zjdN5qEAShqx4hFoqSlYHv0K1OWvKgv5S4DvJXPwH2zMKIVJDIGFI6J69HH-Hjru_XhLp9mN62LLaeRsVyfiiRgT5PKwo7uLm_mRiysa_m9BwciCalZKEhQlMoswiUOkCnSJaeXTEGdJtdE-VWk7uPblNQUTgL-yM7x_tAuftiwNbZCi56jHXjeVkEEx_NKfptihb_hkjAj_C0D6IX3G0d_8alwnl9BtwsJgPAhHx5s349nQ8B3oX808cEIkeSZPixKY3KIYCc1wtYE99kUCq1BDroZqlE",
  "payload": ""
}
https://acme-staging-v02.api.letsencrypt.org:443 "POST /acme/authz-v3/19815850 HTTP/1.1" 200 1407
Received response:
HTTP 200
Server: nginx
Date: Fri, 08 Nov 2019 16:35:10 GMT
Content-Type: application/json
Content-Length: 1407
Connection: keep-alive
Boulder-Requester: 11519950
Cache-Control: public, max-age=0, no-cache
Link: <https://acme-staging-v02.api.letsencrypt.org/directory>;rel="index"
Replay-Nonce: 0002m0OY7qa_ElRdf4p40LyJeqReobHWSx3G8FMn4I0ganA
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800

{
  "identifier": {
    "type": "dns",
    "value": "xxx.lorem.com"
  },
  "status": "invalid",
  "expires": "2019-11-15T16:33:49Z",
  "challenges": [
    {
      "type": "http-01",
      "status": "invalid",
      "error": {
        "type": "urn:ietf:params:acme:error:connection",
        "detail": "Fetching http://xxx.lorem.com/.well-known/acme-challenge/dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U: Connection refused",
        "status": 400
      },
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/19815850/PmKNmQ",
      "token": "dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U",
      "validationRecord": [
        {
          "url": "http://xxx.lorem.com/.well-known/acme-challenge/dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U",
          "hostname": "xxx.lorem.com",
          "port": "80",
          "addressesResolved": [
            "157.230.115.5"
          ],
          "addressUsed": "157.230.115.5"
        }
      ]
    },
    {
      "type": "dns-01",
      "status": "invalid",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/19815850/xRpPTQ",
      "token": "dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U"
    },
    {
      "type": "tls-alpn-01",
      "status": "invalid",
      "url": "https://acme-staging-v02.api.letsencrypt.org/acme/chall-v3/19815850/-6VPpg",
      "token": "dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U"
    }
  ]
}
Storing nonce: 0002m0OY7qa_ElRdf4p40LyJeqReobHWSx3G8FMn4I0ganA
Challenge failed for domain xxx.lorem.com
http-01 challenge for xxx.lorem.com
Reporting to user: The following errors were reported by the server:

Domain: xxx.lorem.com
Type:   connection
Detail: Fetching http://xxx.lorem.com/.well-known/acme-challenge/dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U: Connection refused

To fix these errors, please make sure that your domain name was entered correctly and the DNS A/AAAA record(s) for that domain contain(s) the right IP address. Additionally, please check that your computer has a publicly routable IP address and that no firewalls are preventing the server from communicating with the client. If you're using the webroot plugin, you should also verify that you are serving files from the webroot path you provided.
Encountered exception:
Traceback (most recent call last):
  File "/opt/certbot/src/certbot/auth_handler.py", line 91, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/src/certbot/auth_handler.py", line 180, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.

Calling registered functions
Cleaning up challenges
Removing /var/www/certbot/.well-known/acme-challenge/dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U
All challenges cleaned up
Exiting abnormally:
Traceback (most recent call last):
  File "/usr/local/bin/certbot", line 11, in <module>
    load_entry_point('certbot', 'console_scripts', 'certbot')()
  File "/opt/certbot/src/certbot/main.py", line 1378, in main
    return config.func(config, plugins)
  File "/opt/certbot/src/certbot/main.py", line 1265, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "/opt/certbot/src/certbot/main.py", line 121, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/opt/certbot/src/certbot/client.py", line 417, in obtain_and_enroll_certificate
    cert, chain, key, _ = self.obtain_certificate(domains)
  File "/opt/certbot/src/certbot/client.py", line 348, in obtain_certificate
    orderr = self._get_order_and_authorizations(csr.data, self.config.allow_subset_of_names)
  File "/opt/certbot/src/certbot/client.py", line 396, in _get_order_and_authorizations
    authzr = self.auth_handler.handle_authorizations(orderr, best_effort)
  File "/opt/certbot/src/certbot/auth_handler.py", line 91, in handle_authorizations
    self._poll_authorizations(authzrs, max_retries, best_effort)
  File "/opt/certbot/src/certbot/auth_handler.py", line 180, in _poll_authorizations
    raise errors.AuthorizationError('Some challenges have failed.')
certbot.errors.AuthorizationError: Some challenges have failed.
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: xxx.lorem.com
   Type:   connection
   Detail: Fetching
   http://xxx.lorem.com/.well-known/acme-challenge/dkzqQxktAdggktPN5t4gT3ckwh70o3hmSc28g_yhE8U:
   Connection refused

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

### Reloading nginx ...
2019/11/08 16:35:12 [notice] 10#10: signal process started

xxx stands for my subdomain - lorem for my domain

I double checked my subdomain with nslookup and it seems to be fine.
Also, I tried on a different subdomain, same issue.

In order to give you all the inputs, please find my different files:

default.conf

server {
    listen 80;
    server_name xxx.lorem.com;

    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }    
    
    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name xxx.lorem.com;
    server_tokens off;

    ssl_certificate /etc/letsencrypt/live/xxx.lorem.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/xxx.lorem.com/privkey.pem;

    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
    
    location / {
        proxy_pass http://xxx.lorem.com;
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    }
}

init-letsencrypt.sh

#!/bin/bash

if ! [ -x "$(command -v docker-compose)" ]; then
  echo 'Error: docker-compose is not installed.' >&2
  exit 1
fi

domains=(xxx.lorem.com)
rsa_key_size=4096
data_path="./data/certbot"
email="[email protected]" # Adding a valid address is strongly recommended
staging=1 # Set to 1 if you're testing your setup to avoid hitting request limits

if [ -d "$data_path" ]; then
  read -p "Existing data found for $domains. Continue and replace existing certificate? (y/N) " decision
  if [ "$decision" != "Y" ] && [ "$decision" != "y" ]; then
    exit
  fi
fi


if [ ! -e "$data_path/conf/options-ssl-nginx.conf" ] || [ ! -e "$data_path/conf/ssl-dhparams.pem" ]; then
  echo "### Downloading recommended TLS parameters ..."
  mkdir -p "$data_path/conf"
  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/tls_configs/options-ssl-nginx.conf > "$data_path/conf/options-ssl-nginx.conf"
  curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot/ssl-dhparams.pem > "$data_path/conf/ssl-dhparams.pem"
  echo
fi

echo "### Creating dummy certificate for $domains ..."
path="/etc/letsencrypt/live/$domains"
mkdir -p "$data_path/conf/live/$domains"
docker-compose run --rm --entrypoint "\
  openssl req -x509 -nodes -newkey rsa:1024 -days 1\
    -keyout '$path/privkey.pem' \
    -out '$path/fullchain.pem' \
    -subj '/CN=localhost'" certbot
echo


echo "### Starting nginx ..."
docker-compose up --force-recreate -d nginx
echo

echo "### Deleting dummy certificate for $domains ..."
docker-compose run --rm --entrypoint "\
  rm -Rf /etc/letsencrypt/live/$domains && \
  rm -Rf /etc/letsencrypt/archive/$domains && \
  rm -Rf /etc/letsencrypt/renewal/$domains.conf" certbot
echo


echo "### Requesting Let's Encrypt certificate for $domains ..."
#Join $domains to -d args
domain_args=""
for domain in "${domains[@]}"; do
  domain_args="$domain_args -d $domain"
done

# Select appropriate email arg
case "$email" in
  "") email_arg="--register-unsafely-without-email" ;;
  *) email_arg="--email $email" ;;
esac

# Enable staging mode if needed
if [ $staging != "0" ]; then staging_arg="--staging"; fi

docker-compose run --rm --entrypoint "\
  certbot certonly --webroot -v --debug-challenges -w /var/www/certbot \
    $staging_arg \
    $email_arg \
    $domain_args \
    --rsa-key-size $rsa_key_size \
    --agree-tos \
    --force-renewal" certbot
echo

echo "### Reloading nginx ..."
docker-compose exec nginx nginx -s reload

docker-compose.yml

version: "3"

services:
  nginx:
    image: nginx:1.15-alpine
    container_name: xxx-nginx
    restart: unless-stopped
    volumes:
      - ./data/nginx:/etc/nginx/conf.d
      - ./data/certbot/conf:/etc/letsencrypt
      - ./data/certbot/www:/var/www/certbot
    ports:
      - "80:80"
      - "443:443"
    command: '/bin/sh -c ''while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g "daemon off;"'''

  certbot:
    image: certbot/certbot
    container_name: xxx-certbot
    restart: unless-stopped
    volumes:
      - ./data/certbot/conf:/etc/letsencrypt
      - ./data/certbot/www:/var/www/certbot
    entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"

  nuxt:
    build: ./app/
    container_name: xxx-app
    restart: always
    ports:
      - "3333:3333"
    command: "npm run start"

I'm really desesperate.
Any advice, tip or idea is more than welcome.

Auto renew doesn't work...

Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait 31944{!}; done;'": stat /bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait 31944{!}; done;': no such file or directory": unknown

Running on Container Optimized OS.

Ports take up by certbot

Both the init script and docker-compose up fail. As much I can tell nginx container can't start because the certbot container already uses ports 80 and 443.

Output of init script

### Downloading recommended TLS parameters ...

### Creating dummy certificate for dicedraw.me ...
Generating a RSA private key
........................+++++
............+++++
writing new private key to '/etc/letsencrypt/live/dicedraw.me/privkey.pem'
-----

### Starting nginx ...
Building nginx
Step 1/4 : FROM nginx:alpine
 ---> 0476319fbdad
Step 2/4 : COPY /build /usr/share/nginx/html
 ---> 46e1e3ae3428
Step 3/4 : COPY /compose/nginx.conf /etc/nginx/conf.d/default.conf
 ---> ac6ab3e5a67f
Step 4/4 : CMD ["nginx", "-g", "daemon off;"]
 ---> Running in 0c5af44c0668
Removing intermediate container 0c5af44c0668
 ---> e0dbfc383fb9
Successfully built e0dbfc383fb9
Successfully tagged server_nginx:latest
WARNING: Image for service nginx was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating server_nginx_1 ... error

ERROR: for server_nginx_1  Cannot start service nginx: driver failed programming external connectivity on endpoint server_nginx_1 (4df10f029d22d1d37ec45f74ead73e1d83b63ee0c433fd2a0342ae39f8789e83): Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use

ERROR: for nginx  Cannot start service nginx: driver failed programming external connectivity on endpoint server_nginx_1 (4df10f029d22d1d37ec45f74ead73e1d83b63ee0c433fd2a0342ae39f8789e83): Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use
ERROR: Encountered errors while bringing up the project.

### Deleting dummy certificate for dicedraw.me ...

Output of docker-compose up

Creating server_certbot_1 ... 
Starting server_nginx_1   ... error

Creating server_certbot_1 ... done
rting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use

ERROR: for nginx  Cannot start service nginx: driver failed programming external connectivity on endpoint server_nginx_1 (60bec82974a55f55ffefe5adeeb6c0a2a64b9b0b3b488dca1977f2bed813be95): Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use
ERROR: Encountered errors while bringing up the project.

Running on ubuntu 18.04
Docker version 18.09.4, build d14af54266
docker-compose version 1.21.2, build a133471

Permission denied writing to volume when using CentOS 7 as host

When using CentOS 7 as a host, with only the domain changed in the init-letsencrypt.sh script, the following happens:

$ ./init-letsencrypt.sh
++ command -v docker-compose
+ '[' -x /bin/docker-compose ']'
+ domains=(domain.com)
+ rsa_key_size=4096
+ data_path=./data/certbot
+ email=
+ staging=0
+ '[' -d ./data/certbot ']'
+ read -p 'Existing data found for domain.com. Continue and replace existing certificate? (y/N) ' decision
Existing data found for domain.com. Continue and replace existing certificate? (y/N) y
+ '[' y '!=' Y ']'
+ '[' y '!=' y ']'
+ '[' '!' -e ./data/certbot/conf/options-ssl-nginx.conf ']'
+ '[' '!' -e ./data/certbot/conf/ssl-dhparams.pem ']'
+ echo '### Creating dummy certificate for domain.com ...'
### Creating dummy certificate for domain.com ...
+ path=/etc/letsencrypt/live/domain.com
+ mkdir -p ./data/certbot/conf/live/domain.com
+ docker-compose run --rm --entrypoint '  openssl req -x509 -nodes -newkey rsa:1024 -days 1    -keyout '\''/etc/letsencrypt/live/domain.com/privkey.pem'\''     -out '\''/etc/letsencrypt/live/domain.com/fullchain.pem'\''     -subj '\''/CN=localhost'\''' certbot
Generating a RSA private key
...+++++
.....................+++++
writing new private key to '/etc/letsencrypt/live/domain.com/privkey.pem'
req: Can't open "/etc/letsencrypt/live/domain.com/privkey.pem" for writing, Permission denied

Changing the Docker Compose volumes to named volumes, instead of shared directories, fixes the problem for /etc/letsencrypt, but still has Permission denied errors when creating directories in /var/www/certbot, which makes absolutely no sense to me.

This seems to be an issue with the certbot Docker image, or with the way Docker volumes works with the latest Docker build.

Unable to create certificates for multiple domains

Hi,

I would like to create SSL certificate for both a domain and a subdomain but it failed :

Performing the following challenges:
http-01 challenge for api.charles.cool
http-01 challenge for charles.cool
http-01 challenge for www.api.charles.cool
http-01 challenge for www.charles.cool
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Challenge failed for domain api.charles.cool
Challenge failed for domain charles.cool
Challenge failed for domain www.api.charles.cool
Challenge failed for domain www.charles.cool

There error is the following for the 4 challenges :

Domain: api.charles.cool
Type: connection
Detail: Fetching
http://api.charles.cool/.well-known/acme-challenge/sfqbe7iENRpKWgMpGAd_yk3J1GB2KZJHsU2QDwK-P3o:
Connection refused

That is weird because when I first execute the script with domains set to (api.charles.cool www.api.charles.cool) and then set to (charles.cool www.charles.cool) it works fine.

I don't know where the error comes from...

Do you have any idea ?

Have a nice day

Does not link with Dependent Containers

I'm running a Docker configuration where I have:

  • Django (web application framework)
  • Postgres
  • Nginx
  • Certbot (as per your guide)

The Nginx container depends on the Django container for the web app, which worked on my HTTP only implementation. When I modified my Nginx to include Certbot, I get everything saying that the certification works, but the Nginx container no longer links to my Django container.

Below is the relevant docker-compose.yml section

#############################################################################
  # Nginx Configurations
  nginx:
    container_name: 'nginx'
    image: nginx:1.15-alpine
    restart: unless-stopped
    volumes:
      - ./data/nginx:/etc/nginx/conf.d
      - ./data/certbot/conf:/etc/letsencrypt
      - ./data/certbot/www:/var/www/certbot
      - static_volume:/usr/src/personal_website/static
    ports:
      - "80:80"
      - "443:433"
    depends_on:
      - django
    command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"

  #############################################################################
  # Certbot Configurations (SSL)
  certbot:
    container_name: 'certbot'
    image: certbot/certbot
    restart: unless-stopped
    volumes:
      - ./data/certbot/conf:/etc/letsencrypt
      - ./data/certbot/www:/var/www/certbot
    entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"

Connection refused

sudo ./init-letsencrypt.sh
.
.
.
.
.
### Requesting Let's Encrypt certificate for mydomain.org ...
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing to share your email address with the Electronic Frontier
Foundation, a founding partner of the Let's Encrypt project and the non-profit
organization that develops Certbot? We'd like to send you email about our work
encrypting the web, EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for mydomain.org
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. mydomain.org (http-01): urn:ietf:params:acme:error:connection :: The server could not connect to the client to verify the domain :: Fetching http://mydomain.org/.well-known/acme-challenge/y1TyeXK2Il0hfnqROt-3Arn_6Oq28ffL4kshJZ50pic: Connection refused

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: mydomain.org
   Type:   connection
   Detail: Fetching
   http://mydomain.org/.well-known/acme-challenge/y1TyeXK2Il0hfnqROt-3Arn_6Oq28ffL4kshJZ50pic:
   Connection refused

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.

### Reloading nginx ...
Error response from daemon: Container 67f553b92423babaf66bc1aeea04c62dfc8ebcec58c586001f48b619f3433d12 is restarting, wait until the container is running

Error report Challenge failed for domain

nginx.conf:

upstream fastcgi_backend {
    # use tcp connection
    server  php-fpm:9000;
    # or socket
    # server   unix:/run/php/php7.3-fpm.sock;
}


server {
    listen 80 default;
    server_name xx.com;
    server_tokens off;
#    client_max_body_size 108M;

    access_log /var/log/nginx/application.access.log;
    error_log /var/log/nginx/application.error.log  error;

    #root /application/public/pub;
    set $MAGE_ROOT /application/public;
    # include /application/public/nginx.conf.sample;
    #index index.php;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }

#    if (!-e $request_filename) {
#        rewrite ^.*$ /index.php last;
#    }

#    location ~ \.php$ {
#        fastcgi_pass php-fpm:9000;
#        fastcgi_index index.php;
#        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
#        fastcgi_param PHP_VALUE "error_log=/var/log/nginx/application_php_errors.log";
#        fastcgi_buffers 16 16k;
#        fastcgi_buffer_size 32k;
#        include fastcgi_params;
#    }
    
}

server {
    listen 443 ssl;
    server_name xx.com;
    server_tokens off;

    ssl_certificate /etc/letsencrypt/live/example.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.org/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    location / {
        proxy_pass  http://xx.com;
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    }
}


image

Thanks for the enthusiastic reply.Thank you

@wmnnd @michal-wrzosek
@mestrogov @Pixep @ynixon

[Question] How do you refer to other docker containers?

The host runs nginx-certbot.
It also runs a Wordpress docker.

In the data/nginx/app.conf, on the proxy_pass directive, what host/port do you recommend to use? How do you forward the request to another dockerized service?
Obviously something like http://127.0.0.1:12345 doesn't work, as these two on different (docker) networks.

How do you solve this?

Fails with no such file error

nginx_1 | 2019/10/13 01:17:59 [emerg] 1#1: open() "/etc/letsencrypt/options-ssl-nginx.conf" failed (2: No such file or directory) in /etc/nginx/conf.d/app.conf:22
nginx_1 | nginx: [emerg] open() "/etc/letsencrypt/options-ssl-nginx.conf" failed (2: No such file or directory) in /etc/nginx/conf.d/app.conf:22
certbot_1 | Saving debug log to /var/log/letsencrypt/letsencrypt.log
certbot_1 |
certbot_1 | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
certbot_1 |
certbot_1 | No renewals were attempted.
certbot_1 | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Permission errors on Ubuntu

After stopping docker and making sure the certbot directory does not exist, these are the errors I get when trying to run the init script. I've tried to make this work for the past hour to no avail.

(my nginx image is called "proxy")

moving source_videoprocessing_1 ... done
Removing source_torrents_1        ... done
Removing source_db_1              ... done
Removing source_redis_1           ... done
Removing network source_default
root@homeserver:/var/homeserver/source# ./init-letsencrypt.sh 
### Downloading recommended TLS parameters ...

### Creating dummy certificate for home.nicolasbouliane.com ...
Creating network "source_default" with the default driver
Generating a RSA private key
..........+++++
.............+++++
writing new private key to '/etc/letsencrypt/live/home.nicolasbouliane.com/privkey.pem'
req: Can't open "/etc/letsencrypt/live/home.nicolasbouliane.com/privkey.pem" for writing, Permission denied

### Starting nginx ...
Creating source_torrents_1        ... done
Creating source_videoprocessing_1 ... done
Creating source_db_1              ... done
Creating source_torrents_1        ... 
Creating source_backend_1         ... done
Creating source_videoprocessing_1 ... 
Creating source_frontend_1        ... done
Creating source_frontend_1        ... 
Creating source_proxy_1           ... done

### Deleting dummy certificate for home.nicolasbouliane.com ...
rm: can't remove '/etc/letsencrypt/live/home.nicolasbouliane.com': Permission denied

### Requesting Let's Encrypt certificate for home.nicolasbouliane.com ...
Saving debug log to /var/log/letsencrypt/letsencrypt.log
The following error was encountered:
[Errno 13] Permission denied: '/etc/letsencrypt/.certbot.lock'
Either run as root, or set --config-dir, --work-dir, and --logs-dir to writeable paths.

### Reloading nginx ...
2019/02/22 11:03:06 [notice] 12#12: signal process started
root@homeserver:/var/homeserver/source# 

Do you have any idea of why Docker can't write to a directory it created?

Wildcard certificate?

How do you create a wildcard certificate for all subdomains in a domain?

BTW - There is another issue but it's about multiple specified domains and not wildcards.

Nginx setup for two domains (App/API)

First, thanks for your project. SSL works beautifully.

However, I am having trouble mapping my javascript api running on http://localhost:4000 to https://api.example.io. While SSL works, for both my app and api, and even though in production the frontend is up, the frontend however can't connect to api.

Can you see anything wrong with my current nginx.conf.

The error I get is:

OPTIONS https://api.example.io/api/v1/users 502 (Bad Gateway)

Access to XMLHttpRequest at 'https://api.example.io/api/v1/users' from origin 'https://client.example.io' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

But I'm pretty sure its a nginx.conf issue and not a cors issue.

Cors setup:

...

const corsOptions = {
  origin: [
    'https://client.example.io',
    'http://localhost:3006'
  ]
};

const app = express();
app.use(cors(corsOptions));
app.use(logger);
app.options('*', cors());
...

 // working ok 
server {
    listen 80; 
    server_name  client.example.io;
    server_tokens off;

    root /usr/share/nginx/html;
    index index.html index.htm;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }
     
    location / { 
   
     try_files $uri /index.html =404;
    
    }    
    
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
}

server { 
    listen 443 ssl;
    server_name client.example.io;
    server_tokens off;

    
    ssl_certificate /etc/letsencrypt/live/client.example.io/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/client.example.io/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    location / {
        proxy_pass  http://client.example.io;
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    }
}

// below is sort of working, ssl works but can't access api
server {
    listen 80; 
    server_name  api.example.io;
    server_tokens off;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }
    
    location / {
        proxy_pass https://$host$request_uri;
    }

}

server { 
    listen 443 ssl;
    server_name api.example.io;
    server_tokens off;

    ssl_certificate /etc/letsencrypt/live/api.example.io/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/api.example.io/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    location / {
        proxy_pass      http://localhost:4000;
        proxy_set_header    Host                $http_host;
        proxy_set_header    X-Real-IP           $remote_addr;
        proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;        
    }
}

Thank you for your time and consideration. I would appreciate any tips

Firewall problem

Problem

  • When trying the script it doesn't work because of a firewall problem
  • How do I adjust the script for multiple domains and subdomains

Output of running the script:

svl@Dinkleberg:~/projects/dinkleberg$ sudo ./init-letsencrypt.sh
Existing data found for dinkleberg.weslynouse.com. Continue and replace existing certificate? (y/N) y

Creating dummy certificate for dinkleberg.weslynouse.com ...

Generating a RSA private key
......+++++
.......................................+++++
writing new private key to '/etc/letsencrypt/live/dinkleberg.weslynouse.com/privkey.pem'

failed to resize tty, using default size

Starting nginx ...

ERROR: No such service: nginx

Deleting dummy certificate for dinkleberg.weslynouse.com ...

Requesting Let's Encrypt certificate for dinkleberg.weslynouse.com ...

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for dinkleberg.weslynouse.com
http-01 challenge for www.dinkleberg.weslynouse.com
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Challenge failed for domain dinkleberg.weslynouse.com
Challenge failed for domain www.dinkleberg.weslynouse.com
http-01 challenge for dinkleberg.weslynouse.com
http-01 challenge for www.dinkleberg.weslynouse.com
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:

Reloading nginx ...

ERROR: No such service: nginx
svl@Dinkleberg:/projects/dinkleberg$
svl@Dinkleberg:
/projects/dinkleberg$ sudo ./init-letsencrypt.sh | cat >> output.txt
Existing data found for dinkleberg.weslynouse.com. Continue and replace existing certificate? (y/N) y
ERROR: No such service: nginx
ERROR: No such service: nginx
svl@Dinkleberg:~/projects/dinkleberg$ cat output.txt

Creating dummy certificate for dinkleberg.weslynouse.com ...

Generating a RSA private key
........+++++
..+++++
writing new private key to '/etc/letsencrypt/live/dinkleberg.weslynouse.com/privkey.pem'

Starting nginx ...

Deleting dummy certificate for dinkleberg.weslynouse.com ...

Requesting Let's Encrypt certificate for dinkleberg.weslynouse.com ...

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for dinkleberg.weslynouse.com
http-01 challenge for www.dinkleberg.weslynouse.com
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Challenge failed for domain dinkleberg.weslynouse.com
Challenge failed for domain www.dinkleberg.weslynouse.com
http-01 challenge for dinkleberg.weslynouse.com
http-01 challenge for www.dinkleberg.weslynouse.com
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:

Reloading nginx ...

app.conf settings:

svl@Dinkleberg:~/projects/dinkleberg$ cat data/nginx/app.conf
server {
listen 80;
server_name dinkleberg.weslynouse.com;
server_tokens off;

location /.well-known/acme-challenge/ {
    root /var/www/certbot;
}

location / {
    return 301 https://$host$request_uri;
}

}

server {
listen 443 ssl;
server_name dinkleberg.weslynouse.com;
server_tokens off;

ssl_certificate /etc/letsencrypt/live/dinkleberg.weslynouse.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/dinkleberg.weslynouse.com/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

location / {
    proxy_pass  http://dinkleberg.weslynouse.com;
    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
}

}

docker-compose.yml settings:

svl@Dinkleberg:~/projects/dinkleberg$ cat docker-compose.yml
version: '3'

services:
base:
build: ./base
image: nginx:latest
restart: unless-stopped
volumes:
- ./data/nginx:/etc/nginx/conf.d
- ./data/certbot/conf:/etc/letsencrypt
- ./data/certbot/www:/var/www/certbot
ports:
- "80:80"
- "443:443"
command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g "daemon off;"'"
certbot:
image: certbot/certbot:latest
restart: unless-stopped
volumes:
- ./data/certbot/conf:/etc/letsencrypt
- ./data/certbot/www:/var/www/certbot
entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"

[Support] Challenge failed for domain docker.xyz.com

Steps Which I performed

  1. Create a new instance
  2. Added IP of that instance to A record
  3. Opened ports for 80,22,443
  4. Cloned project
  5. installed docker and docker-compose
  6. Ran ./init-letsencrypt.sh file
  7. throws error Challenge failed for domain docker.xyz.com

please guide I am stuck here and do let me know for any of your findings

Error response from daemon, wait until the container is running

I follow the instruction. I changed domains and mail in init-letsencrypt.sh and app.conf.
When I run ./init-letsencrypt.sh, I get error:

### Creating dummy certificate for kosyachniy.com ...
Generating a RSA private key
.....+++++
.............................+++++
writing new private key to '/etc/letsencrypt/live/kosyachniy.com/privkey.pem'
-----

### Starting nginx ...
Recreating nginx-certbot_nginx_1 ... done

### Deleting dummy certificate for kosyachniy.com ...

### Requesting Let's Encrypt certificate for kosyachniy.com ...
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for kosyachniy.com
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Challenge failed for domain kosyachniy.com
http-01 challenge for kosyachniy.com
Cleaning up challenges
Some challenges have failed.

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: kosyachniy.com
   Type:   connection
   Detail: Fetching
   http://kosyachniy.com/.well-known/acme-challenge/BMxxYdv7natNOi8C6C29sGc9Y7SUY9OFmlhaGxA_R2o:
   Connection refused

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address. Additionally, please check that
   your computer has a publicly routable IP address and that no
   firewalls are preventing the server from communicating with the
   client. If you're using the webroot plugin, you should also verify
   that you are serving files from the webroot path you provided.

### Reloading nginx ...
Error response from daemon: Container d91885e4419fed65d3539f8ec9f0ed880b3a4f2eb82f1468e3f825349dde727d is restarting, wait until the container is running

How can I fix it?

docker-machine error

Hi! I use docker-machine to connect to the remote (digitalocean) environment:
eval $(docker-machine env my_env_name)
and then I try to run ./init-letsencrypt.sh. The result is an error:
writing new private key to '/etc/letsencrypt/live/example.com/privkey.pem' req: Can't open "/etc/letsencrypt/live/example.com/privkey.pem" for writing, No such file or directory
example.com I have changed to real domain name...

Can You help me? What I do wrong?

404: Not Found

$ curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/tls_configs/options-ssl-nginx.conf                                                              
404: Not Found

Originally posted by @dw9694 in #39 (comment)

certbot is not running as a container

Following Phillips' instruction, I got the nginx proxy working with encryption, but I don't see certbot running as a container, not even an exited container. Is this a normal situation?

Dalton

Unable to successfully run "init-letsencrypt.sh"

I followed the article and changed the domain name to localhost and then when I ran the script using sudo ./init-letsencypt.sh
it shows the output

> ➜ sudo ./init-letsencrypt.sh
Password:
Existing data found for localhost. Continue and replace existing certificate? (y/N) y
### Creating dummy certificate for localhost ...
Generating a RSA private key
........+++++
............+++++
writing new private key to '/etc/letsencrypt/live/localhost/privkey.pem'
req: Can't open "/etc/letsencrypt/live/localhost/privkey.pem" for writing, Permission denied

### Starting nginx ...
Recreating nginx-certbot_nginx_1 ... done

### Deleting dummy certificate for localhost ...
rm: can't remove '/etc/letsencrypt/live/localhost': Permission denied

### Requesting Let's Encrypt certificate for localhost ...
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Obtaining a new certificate
An unexpected error occurred:
Error creating new order :: DNS name does not have enough labels
Please see the logfiles in /var/log/letsencrypt for more details.

### Reloading nginx ...
Error response from daemon: Container 9c922778b71621777e238612b37444fbecd6852820ca4517c85feb82c46f5bf2 is restarting, wait until the container is running

I simply want to host a static file on server

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.