Comments (29)
I suspect the reason is indeed because of that linked commit, however, it's impossible to know for sure without more info, I think curl -v will give us that information.
By default, the admin endpoint only allows requests to the host to which the listener binds its socket; the default is localhost
(as a special case, allowing localhost
or ::1
or 127.0.0.1
, etc) -- if you bind to something like foobar:2019
, then the only allowed Host header is foobar
. Or if you bind to 10.1.2.3:2019
then the only allowed Host header is 10.1.2.3
. I guess if you bind to :2019
(all interfaces) or 0.0.0.0:2019
(all IPv4 interfaces), or [::1]:2019
(all IPv6 interfaces), then we need some special logic to accommodate that as well.
To clarify, this is all about the default permissions. As a workaround, you can set the "origins"
property next to "listen"
with a list of hostnames you will be accessing the admin API from. (The docs are slightly outdated, I will update them soon. You don't have to enable origin checking.)
I'll need to decide if we change the current behavior or leave it as-is. If I change it, the only change I can think of would be to disable the Host header checking if binding to a wildcard interface, which is (mildly) less secure. If I leave it as-is, then you still get better security and can simply set the "origins"
field yourself with known, trusted Host values.
Does that make sense? Any thoughts?
(Of course, this is all assuming my hunch is correct; hopefully curl -v will help us know for sure.)
from caddy-docker.
I set enforce_origin to false, and I can't reach the admin API at any address other than the listening address.
After further testing, I take this back. I can reach the admin API at the hosts listed in "origins".
This resolves my issue, since I can whitelist the hostname I want to use to reach Caddy (which seems entirely reasonable to me, from a security perspective.) Thanks, and sorry for the misunderstanding!
from caddy-docker.
@coatezy I think you want this fix: caddyserver/caddy@f5ccb90 which hasn't been released yet, but should be next week.
from caddy-docker.
@b3ntly you need to change the listen address to 0.0.0.0:2019
.
For next time, please ask your usage questions on the Caddy community forums. We prefer to keep the GitHub issue board for bugs and feature requests. Don't forget to fill out the thread template so we can help you!
from caddy-docker.
Can you give more details? What's your full Caddyfile or JSON config? How did you run Docker (docker run
command or docker-compose.yml
file)? What version were you using before you rolled back exactly? caddy:2.0.0-rc.3
or caddy/caddy:2.0.0-rc.3
?
from caddy-docker.
I think this is the relevant section of the JSON config:
"admin":{"listen":"0.0.0.0:2019"}
Before the rollback I was running the caddy/caddy:latest
tag released 2 hours ago: https://hub.docker.com/layers/caddy/caddy/latest/images/sha256-81b8ed2e47aeed5ced4addf3993884bed4766b09d891a9fa671e2ec7c448689d?context=explore
I don't know what numbered version that corresponds to. I only noticed because the image updated itself at some point and suddenly my other containers couldn't reach the Caddy admin API.
I'll see if I can reproduce locally with minimal configuration before sharing the specifics of my deployment. I don't think it will require any special configuration, if I understand the issue correctly
from caddy-docker.
FYI, from now on you should use the caddy
docker image instead of caddy/caddy
, we now have an official image!
Clear reproduction steps would be helpful, with curl -v
, etc.
from caddy-docker.
same issue here.
one console:
echo hello > index.html
docker run -p 80:80 -p 2019:2019 -v $PWD/index.html:/usr/share/caddy/index.html -v caddy_data:/data caddy
{"level":"info","ts":1587030105.4372017,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"info","ts":1587030105.4415984,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
{"level":"info","ts":1587030105.441695,"logger":"http","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
{"level":"info","ts":1587030105.4417708,"logger":"tls","msg":"cleaned up storage units"}
2020/04/16 09:41:45 [INFO][cache:0xc0006062d0] Started certificate maintenance routine
{"level":"info","ts":1587030105.44193,"msg":"autosaved config","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1587030105.4419374,"msg":"serving initial configuration"}
anohter console:
curl localhost:2019/config/
curl: (56) Recv failure: Connection reset by peer
enter caddy container:
apk add curl
curl localhost:2019/config/
success get json config
from caddy-docker.
I'm fairly certain that this is due to some changes in RC3:
Enhanced default security of admin endpoint
The commit in question seems to be caddyserver/caddy@a3bdc22
@postgetme: I don't think you're having the same issue as @collinbachi - @collinbachi has the "admin":{"listen":"0.0.0.0:2019"}
config set, which is important when running in Docker (since the container's localhost
is not the same as the host's localhost
, unless you're sharing the host's network namespace).
@collinbachi If your config isn't working, I have a feeling that the issues is with Caddy RC3 itself, not in the Docker image.
@mholt I feel like the issue is in the allowedOrigins
func somwhere, but I'm not clear on where (caddyserver/caddy@a3bdc22#diff-39b065607d3d0839bdc97f997d6a430cR145)
from caddy-docker.
I set enforce_origin to false, and I can't reach the admin API at any address other than the listening address.
@collinbachi can you clarify what you mean by that? which address specifically are you trying to reach the API at, and which which exact command?
from caddy-docker.
@collinbachi maybe post the output of a curl -v
command to the endpoint?
from caddy-docker.
@collinbachi and your full Caddy configuration would be useful too š
from caddy-docker.
I guess if you bind to
:2019
(all interfaces) or0.0.0.0:2019
(all IPv4 interfaces), or[::1]:2019
(all IPv6 interfaces), then we need some special logic to accommodate that as well.
Yeah, that's what I had wondered, except shouldn't the enforce_origin: false
option have disabled all of that checking?
from caddy-docker.
Yeah, that's what I had wondered, except shouldn't the enforce_origin: false option have disabled all of that checking?
No, that only affects CORS enforcement. The default is false
anyway.
from caddy-docker.
@mholt we should spend more time looking to improve the unix socket situation for admin (especially on Linux, Windows does seem to be a bit of a dead end there unfortunately). I think using a unix socket for the admin endpoint instead of 0.0.0.0:2019
when running in Docker makes a lot of sense.
E.g. if you specify a unix socket, when Caddy is stopped the unix socket file sticks around. Running Caddy again, it complains that the socket file already exists - Caddy should remove the socket when stopped.
Also, as mentioned by someone else previously, the socket doesn't always have the right permissions on it. I'm pretty sure Caddy should set those to something sensible after starting.
from caddy-docker.
Ah! ok, maybe we need an enforce_host
option?
from caddy-docker.
I think using a unix socket for the admin endpoint instead of
0.0.0.0:2019
when running in Docker makes a lot of sense.
It'd help with network security, to be sure, but sharing sockets can be a bit tricky with Docker. It's been a
long time since I've needed to do that, but I remember it being a pain... Plus it makes the API more awkward to access - users just playing around with curl would need to know to use --unix-socket
and how to craft the URL as well.
from caddy-docker.
If we do change things, one possibility is to disable Host checking if binding to a wildcard interface and then emitting a warning in the logs. If you're allowing all network interfaces to connect to your admin endpoint, then you better know what you're doing, I guess?
we should spend more time looking to improve the unix socket situation for admin (especially on Linux, Windows does seem to be a bit of a dead end there unfortunately). I think using a unix socket for the admin endpoint instead of 0.0.0.0:2019 when running in Docker makes a lot of sense.
I believe the admin endpoint does let you bind to a Unix socket. It should work in rc3, if it doesn't, let me know. But in my testing (on a mac) it works fine.
from caddy-docker.
I believe the admin endpoint does let you bind to a Unix socket. It should work in rc3, if it doesn't, let me know. But in my testing (on a mac) it works fine.
Yes, the issues I'm describing is specifically related to that. It works on first run, but subsequent runs don't without manually deleting the socket
from caddy-docker.
That's weird... is that true of all unix sockets, and does that happenf or listen addresses for the HTTP server too? Might be worth an issue in the main repo to track that. (I guess I don't know much about how unix sockets work.)
from caddy-docker.
Re unix sockets opened an issue: caddyserver/caddy#3269
from caddy-docker.
No worries, I just pushed a change that disables that check if you listen on the wildcard interface. I think it's reasonable if you go to the trouble to listen on an open interface (we also emit a warning in the logs).
from caddy-docker.
I was wondering if this was resolved or is there still work to be done upstream? Iām seeing the same issue using the latest build. Using docker-compose I can connect via 0.0.0.0 once listen
has been updated to 0.0.0.0:2019 and I can also connect via a given hostname if set in the list of origins, but if I deploy the same image and config to AWS ECS, the connection to the admin API simply fails to connect, although I can connect to port 80 and 443.
from caddy-docker.
Thanks @mholt. You are right, it was late and had been hitting my head against the desk for a good hour or so. I woke up this morning and realised you had implemented a fix. I confirmed that my setup did work with the RC2 release as mentioned in comments above and have since got everything working with RC3. It is good to know that the fix is being released soon.
I now need to work out how to pump all logs to an HTTP endpoint so I can work out when certificates have been generated successfully or not, but thats another thing.
from caddy-docker.
I now need to work out how to pump all logs to an HTTP endpoint so I can work out when certificates have been generated successfully or not, but thats another thing.
That'll probably get easier with a later version of Caddy in the 2.x tree.
from caddy-docker.
My current workaround is to do something like this:
CaddyConfig.json
{
"admin": {
"listen": "0.0.0.0:2019"
}
}
Dockerfile
FROM caddy:2-alpine
COPY CaddyConfig.json /etc/caddy/
EXPOSE 80
EXPOSE 443
EXPOSE 2019
CMD caddy run --config /etc/caddy/CaddyConfig.json --resume
Put then in the same folder then build my docker image.
I don't know if is the best approach but works for me.
from caddy-docker.
Yep @DrecDroid that's a perfectly reasonable approach.
FYI, those EXPOSE lines don't actually do anything, they're essentially just comments to the reader. You still need to use -p
when running (or the equivalent docker compose yml).
from caddy-docker.
Yes, of course. Then I have to run as usual but now I have the CaddyConfig.json as I needed.
from caddy-docker.
Still a little confused on this one. I'm attempting to pull metrics from localhost:2019/metics from a separate docker container on the same docker network - so the origin address is dynamic. What settings should I use in my caddyfile to enable access to that particular endpoint? If not could I use a proxy directive to achieve the same effect?
Current Caddyfile:
{
debug
admin {
origins *
}
}
localhost {
reverse_proxy golang:4200
}
localhost:9090 {
reverse_proxy prometheus:9090
}
docker-compose
version: '3.8'
services:
caddy:
container_name: caddy
image: 'caddy:latest'
networks:
- global
ports:
- '80:80'
- '443:443'
- '9090:9090'
volumes:
- caddy-data:/data
- caddy-config:/config
- ./Caddyfile:/etc/caddy/Caddyfile
prometheus:
container_name: prometheus
image: prom/prometheus
networks:
- global
volumes:
- prometheus-storage:/prometheus
- ./prometheus.yml:/etc/prometheus/prometheus.yml
volumes:
caddy-data:
caddy-config:
prometheus-storage:
networks:
global:
external: true
and finally prometheus.yaml for good measure
global:
scrape_interval: 15s
scrape_timeout: 10s
evaluation_interval: 15s
scrape_configs:
- job_name: caddy
honor_timestamps: true
scrape_interval: 15s
scrape_timeout: 10s
metrics_path: /metrics
scheme: http
static_configs:
- targets:
- caddy:2019
from caddy-docker.
Related Issues (20)
- Caddy frequently failing to start correctly on Windows containers HOT 3
- Caddy volume increasing like crazy until the disk is full HOT 9
- No linux/amd64 builds for caddy 2.7.5, 2.7.5-alpine, 2.7.5-builder HOT 4
- Caddy 2.7.5-builder is missing specific architectures HOT 2
- No linux amd64 image in docker hub with caddy 2.7.5 HOT 1
- CVE-2023-45142 on otelhttp HOT 2
- Builder tag not friendly for variables HOT 4
- Pass arguments to Caddy without config file HOT 3
- healthcheck always shows unhealthy HOT 2
- No arm32 build for caddy 2.7.6 HOT 2
- Update to golang v1.22 for builder base HOT 7
- caddy:2.7.6: getting missing instance.uuid file warning HOT 2
- Dockerhub 2.8.0 image is missing linux build HOT 1
- Missing Caddy v2.8.2 and v2.8.4 HOT 2
- Warning in caddy logs HOT 2
- Missing image for 2.8.4 on docker.io HOT 1
- Build amd64 and arm are broken HOT 4
- Docker images for arm64 are not working after 2.8.1 HOT 12
- request container image on ghcr
- Current 2.8.4 image contain critical security vulnerability HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
š Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ššš
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ā¤ļø Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from caddy-docker.