Docker CLI hangs when launching PWD containers in non detached mode about play-with-docker

[marcosnils]

Found the reason. When creating a container the CLI tries to hijack the connection by telling the server that it'll upgrade the connection to TCP to stream the container output:

https://github.com/docker/docker/blob/d8d33148d809ec97794bbd492617b1be677cf820/client/hijack.go#L36-L80

PWD doesn't support TCP proxying ATM, so I'm not sure how/if we should handle this.

any ideas @jpetazzo @xetorthio @alexellis @akalipetis @so0k ?

[jpetazzo]

Ideas:

• support TCP proxying (doh!)
• have each DinD instance expose port 2376, and give that port number when specifying DOCKER_HOST (requires SSL certs)
• use ngrok in TCP mode (maybe?)

[marcosnils]

support TCP proxying (doh!)

Yes, but how?. We can't do TCP/IP reverse proxy as we have no way to know which instance the client is trying to talk to as it's only ip:port.

have each DinD instance expose port 2376, and give that port number when specifying DOCKER_HOST (requires SSL certs)

Not sure how this helps with the hang thing. Whenever the CLI tries to upgrade the connection to TCP, it will still fail, right?

use ngrok in TCP mode (maybe?)

Yes, but this won't work for PWD machine driver :(. As ngrok address will differ from the original endpoint.

[xetorthio]

We could use a prefixed port for each DinD as a way to handle TCP proxying to the docker daemon

…

On Wed, Jan 25, 2017, 10:18 Marcos Nils ***@***.***> wrote: support TCP proxying (doh!) Yes, but how?. We can't do TCP/IP reverse proxy as we have no way to know which instance the client is trying to talk to as it's only ip:port. have each DinD instance expose port 2376, and give that port number when specifying DOCKER_HOST (requires SSL certs) Not sure how this helps with the hang thing. Whenever the CLI tries to upgrade the connection to TCP, it will still fail, right? use ngrok in TCP mode (maybe?) Yes, but this won't work for PWD machine driver :(. As ngrok address will differ from the original endpoint. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#78 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIhp9J0npo32HYfvY3LXwfohFAAI_4lks5rV0usgaJpZM4LcqI6> .

[marcosnils]

We could use a prefixed port for each DinD as a way to handle TCP proxying
to the docker daemon

so basically, each time you create an instance (dind) you allocate a port on the host that will serve as TCP proxy?. But that will require the instance to have several daemon ports wide open right?. If this is the case we'll have to restrict daemons to SSL which will invalidate the ability to set DOCKER_HOST manually to control PWD daemons.

[xetorthio]

Why this would restrict daemons to SSL?

…

On Wed, Jan 25, 2017 at 11:29 AM Marcos Nils ***@***.***> wrote: We could use a prefixed port for each DinD as a way to handle TCP proxying to the docker daemon so basically, each time you create an instance (dind) you allocate a port on the host that will serve as TCP proxy?. But that will require the instance to have several daemon ports wide open right?. If this is the case we'll have to restrict daemons to SSL which will invalidate the ability to set DOCKER_HOST manually to control PWD daemons. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#78 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIhp-NV_8KowhHt0WuvBmwA-7JIO7d1ks5rV1xUgaJpZM4LcqI6> .

[marcosnils]

Why this would restrict daemons to SSL?

Because there are hundreds of bots out there looking for open daemons to mess with the system. Right now it's "more" to do because it'd have to guess the URL of the instance to access. But if we do TCP reverse proxy through the host ip: it'd be super easy to detect and exploit.

[xetorthio]

Oh... OK. So the solution I proposed will work when using docker-machine. But to be able to set your host manually you'd have to set a few extra parameters, as we would use SSL. Doesn't seem like a big price to pay, right?

…

On Wed, Jan 25, 2017 at 11:45 AM Marcos Nils ***@***.***> wrote: Why this would restrict daemons to SSL? Because there are hundreds of bots out there looking for open daemons to mess with the system. Right now it's "more" to do because it'd have to guess the URL of the instance to access. But if we do TCP reverse proxy through the host ip: it'd be super easy to detect and exploit. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#78 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIhp83vGhzF3nXimmOvTUEdqyl68MYiks5rV1__gaJpZM4LcqI6> .

[marcosnils]

But to be able to set your host manually you'd have to set a few extra
parameters, as we would use SSL.
Doesn't seem like a big price to pay, right?

Well.. it's not that easy. You'd need to get the certificates somehow to make it work manually. When using machine these are provided by machine itself.

[xetorthio]

The play-with-docker website can give you a nice link to download the certificates and even tell you how to configure your CLI to work with the daemon. Still seems like a very corner case. My guess is that people will either use play-with-docker web interface and/or docker-machine. Which means that optimizing for these two is the right move. The last option would be configuring you CLI manually, and pwd site can help you with that too.

…

On Wed, Jan 25, 2017 at 11:52 AM Marcos Nils ***@***.***> wrote: But to be able to set your host manually you'd have to set a few extra parameters, as we would use SSL. Doesn't seem like a big price to pay, right? Well.. it's not that easy. You'd need to get the certificates somehow to make it work manually. When using machine these are provided by machine itself. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#78 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIhp3TFqGIQrj-pbJmswJOJ2LuTc7wwks5rV2GagaJpZM4LcqI6> .

[marcosnils]

The play-with-docker website can give you a nice link to download the
certificates and even tell you how to configure your CLI to work with the
daemon.

It's not exactly how it works. The user needs to generate some certs and then make the CA (the server in this case) sign them. So "just downloading" them is not an option.

Still seems like a very corner case. My guess is that people will either
use play-with-docker web interface and/or docker-machine. Which means that
optimizing for these two is the right move

Agree. I propose to just drop the single CLI configuration for simplicity and security aspects and only allow docker-machine configurations.

[xetorthio]

It's not exactly how it works. The user needs to generate some certs and then make the CA (the server in this case) sign them. So "just downloading" them is not an option.

That depends. In our case client validates that the server uses a valid certificate signed by some known and trusted CA. So by default the DinD, when created, can offer a certificate and the CA for the client to use. This would solve the manual CLI configuration.

When using docker-machine, those keys are overwritten by the ones machine sends (this is already how it works now).

Which means that all scenarios would be possible with a very small change.

[marcosnils]

So by default the DinD, when created, can offer a certificate and the CA for the client to use. This would solve the manual CLI configuration.

Right. Not sure if it makes a lot of sense though. Now the certs will depend on each instance you create instead of downloading them directly from the PWD site.

I guess my vote goes for just ignore this use-case and focus on machine + tcp ssl proxy for the moment.

[xetorthio]

Now the certs will depend on each instance you create instead of downloading them directly from the PWD site.

Not sure I understand this. Can you explain this further?

[marcosnils]

Seems like now it's not even possible to run detached containers. I think something changed in the daemon that's preventing this :(. @jpetazzo any hints?

[xetorthio]

What we are going to do is to proxy tcp encapsulated in http. This will fix this issue and might also make http and websocket proxy totally useless. We'll try and report back soon!

[marcosnils]

Fixed by 75f3c93 🎉