play-with-docker / play-with-docker

You know it, you use it, now it's time to improve it. PWD!.

play-with-docker

Play With Docker gives you the experience of having a free Alpine Linux Virtual Machine in the cloud where you can build and run Docker containers and even create clusters with Docker features like Swarm Mode.

Under the hood DIND or Docker-in-Docker is used to give the effect of multiple VMs/PCs.

A live version is available at: http://play-with-docker.com/

Requirements

• Docker 18.06.0+
• Go (stable release)

Development

# Clone this repo locally
git clone https://github.com/play-with-docker/play-with-docker
cd play-with-docker

# Verify the Docker daemon is running
docker run hello-world

# Load the IPVS kernel module. Because swarms are created in dind,
# the daemon won't load it automatically
sudo modprobe xt_ipvs

# Ensure the Docker daemon is running in swarm mode
docker swarm init

# Get the latest franela/dind image
docker pull franela/dind

# Optional (with go1.14): pre-fetch module requirements into vendor
# so that no network requests are required within the containers.
# The module cache is retained in the pwd and l2 containers so the
# download is a one-off if you omit this step.
go mod vendor

# Start PWD as a container
docker-compose up

Now navigate to http://localhost and click the green "Start" button to create a new session, followed by "ADD NEW INSTANCE" to launch a new terminal instance.

Notes:

• There is a hard-coded limit of 5 Docker playgrounds per session. After 4 hours sessions are deleted.
• If you want to override the DIND version or image then set the environmental variable i.e. DIND_IMAGE=franela/docker<version>-rc:dind. Take into account that you can't use standard dind images, only franela ones work.

Port forwarding

In order for port forwarding to work correctly in development you need to make *.localhost to resolve to 127.0.0.1. That way when you try to access pwd10-0-0-1-8080.host1.localhost, then you're forwarded correctly to your local PWD server.

You can achieve this by setting up a dnsmasq server (you can run it in a docker container also) and adding the following configuration:

address=/localhost/127.0.0.1

Don't forget to change your computer's default DNS to use the dnsmasq server to resolve.

FAQ

How can I connect to a published port from the outside world?

If you need to access your services from outside, use the following URL pattern http://ip<hyphen-ip>-<session_jd>-<port>.direct.labs.play-with-docker.com (i.e: http://ip2-135-3-b8ir6vbg5vr00095iil0-8080.direct.labs.play-with-docker.com).

Why is PWD running in ports 80 and 443? Can I change that?

No, it needs to run on those ports for DNS resolve to work. Ideas or suggestions about how to improve this are welcome

Hints

How can I use Copy / Paste shortcuts?

Ctrl + insert : Copy
shift + insert : Paste

play-with-docker's Issues

Dockerize app

We need a dockerfile that builds this app 😄

Store rows*cols bytes of every instance terminal

This is useful when new clients connect to a session, they will get this buffer and can see the current state of the terminal.

Add swarm mode info in frontend

Objective:

If I create a swarm (one or many) it'd be amazing if the app shows the swarm ID as well as each node role in the swarm (managers (leaders and followers) and workers).

This one is a bit tricky the way the app currently works because there's no centralized database to query that info from.

Depends on #8

Access-Control-Allow-Origin fails when docker container sends "*" as Header Value

The play-with-docker containers will override the value with the container's URL. If the project inside the container returns "*", it will fail in Chrome and Firefox, because sending the header two times with different values is not allowed as per specification.

Example error in Chrome:

XMLHttpRequest cannot load http://pwd10_0_9_3-8080.play-with-docker.com:4848/api/project/index. The 'Access-Control-Allow-Origin' header contains multiple values 'http://pwd10_0_9_3-8080.play-with-docker.com, *', but only one is allowed. Origin 'http://pwd10_0_9_3-8080.play-with-docker.com' is therefore not allowed access.

The header is added by play-with-docker and not overwritten which means there will be two times the Access-Control-Allow-Origin header transmitted over the network. You can see that in Chrome via the Network Tab's Header Inspector:

Add loading state to action buttons

"Add instance" and "Delete" should have a spinner or something that indicates that the action started and ended. Sometimes "Add instance" fails because of a 1.13rc networking bug and the session needs to be recreated.

Add a button to close the session

Currently we have a button to "delete" an instance.
We should probably add a button to close the entire playground session. We should delete all instances and the session immediately.

Add usage metrics

We could publish metrics so prometheus can collect them and keep track of things like:

• Number of sessions opened
• Number of instances opened

We can eventually add more important metrics :)

Access service from public internet with ngrok

I tried running ngrok but it crashed the screen and gave an error about $HOME.. this would be really cool for connecting to the live service from your PC.

👍 on this project.. very cool proof of concept 💯 %

Limit the amount of processes

Just an idea - maybe limit the amount of processes?

/ # ulimit -p
524288

Could maybe do make a mess if someone fork bomb'd 5x524288 processes?

ssh: command not found

I would like to ssh into another node.

But whenever I type ssh root@IP_of_other_instance, I get the following error.

bash: ssh: command not found

Also apt-get dosen't work.

How can I ssh into another node?

Add central database to store app metadata

The idea to horizontally scale nodes (docker daemons and webapp). In order to do this we need a centralized DB ( maybe redis?) where session information and node state is stored and shared across nodes.

There are several //TODO's in the code to improve this.

Docker CLI hangs when launching PWD containers in non detached mode

If you point your docker cli using DOCKER_HOST to a pwd daemon and try to run a non detached container, the CLI will hang and the container will be left in "Created" state

Check play-with-docker/docker-machine-driver-pwd#1 for more info

Add OpenGraph support

List of exposed ports should be stable (probably also be sorted)

When I run

docker run -d -p 80:80 -p 443:443 nginx

Then ports 80 and 443 are opened.
These exposed ports are also displayed in the UI next to the IP.
There nearly each time the script checks the exposedd ports the order in which they are displayed changes.

Wouldn't it be nicer if these portlists are stable, probably also sorted?
Maybe by port number and/or by time the container with the exposed port has been started.

The current flickering just indicates a change, but in fact nothing changed.

Getting errors when running on Docker4Mac beta31

I'm trying to run PWD locally on my Mac in D4M beta31 (Docker 1.13.0-rc2).

I get to the main webpage and accept the captcha and will be redirected to a blank page and getting the following errors on the console:

2016/12/01 05:53:56 NewSession id=[171c7cec-349c-47f7-95fc-a63e3b090cbf]
2016/12/01 05:53:56 Starting session err [Error response from daemon: datastore for scope "global" is not initialized]
2016/12/01 05:53:56 ERROR NETWORKING
2016/12/01 05:53:56 Error response from daemon: datastore for scope "global" is not initialized
[negroni] Completed 0  in 550.306066ms

Would be nice to get some help or at least some hints where to dig deeper to analyse the problem. Running PWD in D4M could be pretty easy to set up and demo w/o the need to install and maintain additional hypervisors. at all.

Keep online and github versions in sync

This is not an issue, but more of an possible improvement. I saw that the on-line variant was using a newer version of dind, with Docker 1.13.0-rc3, while the Dockerfile in this github was based on 1.13.0-rc2. THis version is also the version on the Docker hub.

Is it possible to keep the code on github, the image on Docker hub and the on-line variant on play-with-docker.com in sync? The code is already there, I think as it is running on play-with-docker.com. Maybe on a seperate (develop/unstable) branch?

Add the option for restarting the timer when it runs out with persistence for active instances.

Is there any possibility of giving the option for saving the state of instance when the timer runs out. And give an extra option to the user to restart the timer along with the previously active instances and work.

Or maybe give an option for extending the timer and session when 10 or so minutes are left before timer runs down.

Consider the below scenario:

1. A user have n number of active instances for m hours.
2. Now, when m = 3, the session will be closed regardless of the number of active instances.
3. But what if the user wanted to continue his work beyond 3 hours?
4. Is there any possibility of doing the provision for restarting the time left but persisting the active work.

Just a thought.

Avoid race conditions by locking on every service call

I think the way we implemented services is a bit racy.
If 2 clients are deleting the same instance at the same time, etc, it could crash or lead to weird results.
Since this is not traffic intensive, we could always lock on every service call, to make sure everything that happens in the session in synchronized and safe.

Does PWD can run on a Docker Swarm ?

level=error msg="'overlay2' is not supported over overlayfs"

Hello,

I try to use the play-with-docker on ubuntu 16.04 with docker with overlay2.

When building image from Dockerfile.dind, then launching it via PWD, I got this error :

level=error msg="'overlay2' is not supported over overlayfs"

I had similair error with

level=error msg="'overlay2' is not supported over aufs"

Which driver to you uses for you PWD instance ?

Thanks

Error response from daemon: driver name: if driver is specified name is required

I did a go get and go build with go 1.7.1 and Docker 1.12.2-rc1. This is what happened when I hit "localhost:3000/"

Commenting out the Driver portion let me work around the issue but used bridge for the driver.

[negroni] Started GET /
2016/10/09 19:06:17 NewSession id=[dc341e92-970c-44ef-aa18-0b80d60e4ed9]
2016/10/09 19:06:17 Starting session err [Error response from daemon: driver name: if driver is specified name is required]
2016/10/09 19:06:17 Error response from daemon: driver name: if driver is specified name is required
[negroni] Completed 0  in 8.6795ms

Store state on disk

We can store all session information on disk so whenever we restart the API, we can still have all the sessions.
We need to check if there is something we can do to not loose our terminals.

Error with WebSockets and UTF-8 encodings

If within an instance you write curl www.google.com you'll see that the javascript console shows: WebSocket connection to 'ws://play-with-docker.com/sessions/952dc23e-c830-40e2-ba0d-35ec548d0127/instances/goofy_keller/attach' failed: Could not decode a text frame as UTF-8. and terminal stops working

Alert when PWD doesn't work

The idea is to write a simple script that:

1. Creates a new session
2. Creates an instance
3. Closes the session

We will run this script every few minutes on every PWD server. If it works, nothing happens. If it doesn't work, we will send a message to a slack channel.

Use cookies?

If you close your tab accidentally and you didn't save your session URL you're done. Should we add a sessionID cookie?

Add instanceId hash to URL

This way when sharing a link the receiver page will open on the same instance that the sender shared.

Add google captcha when creating a session

This way we prevent possible bots from exploiting the site.

Add host redirection when creating session

In order to make this thing scale we need to redirect to <host>.play-with-docker.com when creating the session. This was once the host is picked by the load balancer, any following request will go directly to the host that was chosen for the session.

Allow to upload files and edit them in realtime

Add tutorials

It'd be amazing to have something like golang (https://tour.golang.org/welcome/1) using play-with-docker.

We definitely need to structure in a way that's easy for collaborators to add new sections and content.

unable to get to exposed services when running locally

Is there something else that needs to be configured or setup for service discovery/proxying?

From the README:

If you need to access your services from outside, use the following URL pattern http://ip<underscore_ip>-<port>.play-with-docker.com (i.e: http://ip10_2_135_3-80.play-with-docker.com/).

However, when running locally using make, exposed services redirect to URLs such as:

http://ip10_0_0_3-32768.192.168.99.103:3000/

Which doesn't work. What else needs to be set and/or run for this to work?

Limit the amount of memory to 512MB by default

According to @jpetazzo suggestions, 512MB should be enough for the default workshop expectations.

Add shortcut presets selection

Discussion started in #97 and moved here.

Add message when server is offline

Now that sessions are persisted into disk #32 it'd be nice to tell the user when the connection has been lost. I believe it should be quite simple to do with socket.io

Show people connected to the same session

And if we know in the future who they are (because they are identified somehow) then we could show it nicer and maybe even say who is typing (when someone is typing)

Delete terminal automatically if exec session exists

if you exit the shell by either exit or ctrl + d the exec session will end and all other terminal will remain unresponsive. It'd be nice to warn the user about this action and automatically delete terminals if action is intentional.

Line breaks are weird for copy/pasting

When creating a new swarm, I do:

I then select that whole text and then when I paste it in a new instance, I get:

It seems as though the line breaks or character encoding isn't entirely happy with the clipboard or something like that.

Running on macOS Sierra + Chrome latest

Save Session as Gist

Feature Request: Save session as Gist

• Each host's terminal output would go to a given gist file
• Debug Info: Each host state, memory, files open, etc could be addition debug info

I think this would help people debug a long session better and share with others.

Unable to access PWD host/cluster via Portainer

Usually, I connected Portainer to PWD either when starting Portainer via the CLI with the -H flag or via the UI by defining a new endpoint.

I used to manage the PWD endpoints using the following url pattern ip10_0_XX_XX-2375.play-with-docker.com:80.

Example:

$ docker run -d -p 9000:9000 portainer/portainer -H tcp://ip10_0_XX_XX-2375.play-with-docker.com:80

Yesterday, when I tried it again I was not able to connect to PWD either from Portainer or via the CLI using this URL pattern. After a short discussion on twitter with @marcosnils (https://twitter.com/marcosnils/status/831108149860036608), he told me to update the URL pattern to ip10_0_XX_XX.play-with-docker.com.

Now, this is working fine when I use the Docker CLI with the -H flag but it's not working anymore with Portainer (either using the -H flag or when defining the endpoint from the UI).

Portainer tries to proxy the request to the PWD host/cluster and gets a 502 Bad Gateway issue.

Bug: nano editor doesn't work

# apk --update add nano
# nano

Type in some text then do control + w control + x ... it doesn't work - just hangs. I think mandating vi will be an issue for the groups I'd be working with.

Is this to do with the remapping of disconnect keys?

Remove volumes when cleaning up containers

CPU Limit

Hello,

Is there a way to limit the CPU allocated to pwd dind instances ?

thanks

Proxy issue

From behind corporate firewall / http_proxy:

socket.io-1.3.7.js:3 WebSocket connection to 'ws://play-with-docker.com/sessions/1ee25be8-80e2-48b0-9d89-8f7478345f2e/ws/?EIO=3&transport=websocket' failed: Establishing a tunnel via proxy server failed.ws @ socket.io-1.3.7.js:3WS.doOpen @ socket.io-1.3.7.js:2Transport.open @ socket.io-1.3.7.js:1Socket.open @ socket.io-1.3.7.js:1Socket @ socket.io-1.3.7.js:1Socket @ socket.io-1.3.7.js:1Manager.open.Manager.connect @ socket.io-1.3.7.js:1(anonymous function) @ socket.io-1.3.7.js:1

wss:// does appear to work on KataCoder (Ben Hall) because it uses 443 / CONNECT.

Cut / Copy is not working

I started a new instance. Then ran 'docker swarm init'. The command worked but I could not highlight the text then right-click to copy. Nor could I use ^c.

Is there another keyboard combo I should be using?

DIND override of regular docker:dind image fails to load

This just gives a log of the Docker daemon - using the franela image does work.

Add support for terminal resize

Current terminal size is hardcoded to a default size of 80 x 24. It'd be nice to automatically handle screen resize.

I believe this should be implemented after #10 to prevent any possible re-work.

Add timer on top of the screen

So that people know how long they still have in their session!

http://play-with-docker.com/ broken?

Is http://play-with-docker.com/ broken? When I click on 'ADD NEW INSTANCE' it doesn't seem to do anything. It was working yesterday.

Client side timer somtimes is not synchronized

When sleeping and resuming the machine, the client-side timer will lose sync with the server. As stats are already being sent once every a couple of seconds, maybe we can take the opportunity to sync the timer as well.

@xetorthio thoughts?

Prevent denial of service - disk flooding

Run this...

dd if=/dev/zero of=filename bs=$((1024*1024)) count=$((10*1024))

And the whole service will grind to a halt and stop functioning.

Perhaps need to add / enforce a disk size quota on containers.