A whitelist proxy that uses socat. ๐๐ผ
tl;dr: To workaround moby/moby#36174.
Basically, Docker supports internal networks; but when you use them, you simply cannot open ports from those services, which is not very convenient: you either have full or none isolation.
This proxy allows some whitelist endpoints to have network connectivity. It can be used for:
- Allowing connection only to some APIs, but not to the rest of the WWW.
- Exposing ports from a container while still not letting the container access the WWW.
Use these environment variables:
Required. It's the host name where the incoming connections will be redirected to.
Default: tcp
Set to udp
to proxy in UDP mode.
Default: 100
Limits the maximum number of accepted connections at once per port.
For each port and open connection a subprocess is spawned. Setting a number too high might make your host system unresponsive and prevent you from logging in to it. So be very careful with setting this setting to a large number.
The typical linux system can handle up to 32768 so if you need a lot more parallel open connections make sure to also set the corresponding variables on your host system. See https://stackoverflow.com/questions/6294133/maximum-pid-in-linux for reference. And divide this number by at least the number of ports you are running through docker-whitelist.
docker-whitelist basically starts socat
so the behaviour is the same. In case no more
subprocesses can be forked:
- UDP mode: You won't see a difference on the connecting side. But no more packets are forwarded for new connections until the number of connections for this port is reduced.
- TCP mode: docker-whitelist no longer accepts the connection and your connection will wait until the number of connections for this port is reduced. Your connection may time out.
Default: 208.67.222.222 8.8.8.8 208.67.220.220 8.8.4.4
to use OpenDNS and Google DNS
resolution servers by default.
Only used when pre-resolving is enabled.
Default: 80 443
. If you're proxying HTTP/S services, no need to specify!
The port where this service will listen, and where the target service is expected to be listening on also.
Default: 0
Set to 1
to force using the specified nameservers to resolve the
target before proxying.
This is especially useful when using a network alias to whitelist an external API.
Default: 1
1
means the process will wait for an answer from the server before the forked child
process terminates (until this happens the connection counts towards the connection
limit). Set to 0
if no answers are expected from the server, this prevents
subprocesses waiting for an answer indefinitely.
Setting to 0
is recommended if you are using this to connect to a syslog server like
graylog.
Default: 0
Set to 1
to log all connections.
So say you have a production app called coolapp
that sends and reads emails, and uses
Google Font APIs to render some PDF reports.
It is defined in a docker-compose.yaml
file like this:
# Production deployment
version: "2.0"
services:
app:
image: Tecnativa/coolapp
ports:
- "80:80"
environment:
DB_HOST: db
depends_on:
- db
db:
image: postgres:alpine
volumes:
- dbvol:/var/lib/postgresql/data:z
volumes:
dbvol:
Now you want to set up a staging environment for your QA team, which includes a fresh copy of the production database. To avoid the app to send or read emails, you put all into a safe internal network:
# Staging deployment
version: "2.0"
services:
proxy:
image: traefik
networks:
default:
public:
ports:
- "8080:8080"
volumes:
# Here you redirect incoming connections to the app container
- /etc/traefik/traefik.toml
app:
image: Tecnativa/coolapp
environment:
DB_HOST: db
depends_on:
- db
db:
image: postgres:alpine
networks:
default:
internal: true
public:
Now, it turns out your QA detects font problems. Logic! app
cannot contact
fonts.google.com
. Yikes! What to do? ๐คท
tecnativa/whitelist
to the rescue!! ๐ช๐ค
# Staging deployment
version: "2.0"
services:
fonts_googleapis_proxy:
image: tecnativa/whitelist
environment:
TARGET: fonts.googleapis.com
PRE_RESOLVE: 1 # Otherwise it would resolve to localhost
networks:
# Containers in default restricted network will ask here for fonts
default:
aliases:
- fonts.googleapis.com
# We need public access to "open the door"
public:
fonts_gstatic_proxy:
image: tecnativa/whitelist
networks:
default:
aliases:
- fonts.gstatic.com
public:
environment:
TARGET: fonts.gstatic.com
PRE_RESOLVE: 1
proxy:
image: traefik
networks:
default:
public:
ports:
- "8080:8080"
volumes:
# Here you redirect incoming connections to the app container
- /etc/traefik/traefik.toml
app:
image: Tecnativa/coolapp
environment:
DB_HOST: db
depends_on:
- db
db:
image: postgres:alpine
networks:
default:
internal: true
public:
And voilร ! app
has fonts, but nothing more. โ๐ฎ
All the dependencies you need to develop this project (apart from Docker itself) are managed with poetry.
To set up your development environment, run:
pip install pipx # If you don't have pipx installed
pipx install poetry # Install poetry itself
poetry install # Install the python dependencies and setup the development environment
To run the tests locally, add --prebuild
to autobuild the image before testing:
poetry run pytest --prebuild
By default, the image that the tests use (and optionally prebuild) is named
test:docker-whitelist
. If you prefer, you can build it separately before testing, and
remove the --prebuild
flag, to run the tests with that image you built:
docker image build -t test:docker-whitelist .
poetry run pytest
If you want to use a different image, pass the --image
command line argument with the
name you want:
# To build it automatically
poetry run pytest --prebuild --image my_custom_image
# To prebuild it separately
docker image build -t my_custom_image .
poetry run pytest --image my_custom_image