anurag / fastai-course-1

Docker environment for fast.ai Deep Learning Course 1 at http://course.fast.ai

Docker for fast.ai Course 1

A Jupyter environment for fast.ai's Deep Learning MOOC at http://course.fast.ai.

Runs a Jupyter notebook on port 8888 with the default password used in the course ('dl_course').

Uses CPUs by default and NVIDIA GPUs when run with nvidia-docker.

The container comes with:

• All notebooks from https://github.com/fastai/courses/tree/master/deeplearning1/nbs
• Python 2.7 (the default Python version used in the course)
• Conda
• Theano
• Keras
• PIL
• Jupyter
• bcolz
• kaggle-cli
• ...all other libraries needed for the course.

Usage

CPU Only

docker run -it -p 8888:8888 deeprig/fastai-course-1

With GPU

nvidia-docker run -it -p 8888:8888 deeprig/fastai-course-1

Data management

Docker containers are designed to be ephemeral, so if you need persistent data for Kaggle competitions you should download it on your local machine and mount the directory as a host volume when you run the container.

For example, if your data directory is at /Users/yourname/data, start your container with this command:

docker run -it -p 8888:8888 -v /Users/yourname/data:/home/docker/data deeprig/fastai-course-1

Your local data directory will now be visible in the container at /home/docker/data.

Don't forget to change the path to the data folder in your notebooks as well!

Installing packages

All packages should ideally be part of the Dockerfile. If something is missing, please open an issue or submit a PR to update the Dockerfile. If you need to install something as a workaround, follow the steps below:

1. Get a shell into the running container with docker exec -it <container_name> /bin/bash
2. sudo apt-get update && sudo apt-get install package_name

Running on AWS

You can also use docker-machine and preconfigured AMIs for us-west-2 using the commands below.

GPU instance

# spin up a p2.xlarge instance
docker-machine create \
  --driver amazonec2 \
  --amazonec2-region='us-west-2' \
  --amazonec2-root-size=50 \
  --amazonec2-ami='ami-e03a8480' \
  --amazonec2-instance-type='p2.xlarge' \
  fastai-p2

# open Jupyter port 8888
aws ec2 authorize-security-group-ingress --group-name docker-machine --port 8888 --protocol tcp --cidr 0.0.0.0/0

# open an SSH shell on the new machine
docker-machine ssh fastai-p2

# (on the remote machine fastai-p2) run Jupyter interactively
nvidia-docker run -it -p 8888:8888 deeprig/fastai-course-1

# (on your local machine) get the IP of the new machine:
docker-machine ip fastai-p2

Open http://[NEW_MACHINE_IP]:8888 in your browser to view notebooks.

CPU instance

# spin up a t2.xlarge instance
docker-machine create \
  --driver amazonec2 \
  --amazonec2-region='us-west-2' \
  --amazonec2-root-size=50 \
  --amazonec2-ami='ami-a073cdc0' \
  --amazonec2-instance-type='t2.xlarge' \
  fastai-t2

# open Jupyter port 8888
aws ec2 authorize-security-group-ingress --group-name docker-machine --port 8888 --protocol tcp --cidr 0.0.0.0/0

# open an SSH shell on the new machine
docker-machine ssh fastai-t2

# (on the remote machine fastai-t2) run Jupyter interactively
docker run -it -p 8888:8888 deeprig/fastai-course-1

# (on your local machine) get the IP of the new machine:
docker-machine ip fastai-t2

Open http://[NEW_MACHINE_IP]:8888 in your browser to view notebooks.

fastai-course-1's Issues

Error when running the fastai docker container

I have a CentOS 7 host running nvidia-docker (most recent v2), no issues with passthrough GPU acceleration on other containers such as the distribution PyTorch and TensorFlow containers. GTX1060 with CUDA 9.1 on the host.

Is this a v1 vs v2 issue perhaps?

/bin/docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "process_linux.go:402: container init caused "process_linux.go:385: running prestart hook 1 caused \"error running hook: exit status 1, stdout: , stderr: exec command: [/usr/bin/nvidia-container-cli --load-kmods configure --ldconfig=@/sbin/ldconfig --device=all --compute --utility --require=cuda>=8.0 --pid=25071 /var/lib/docker/overlay2/82c6669d2a69da8cf14989bc35828007b057014d489d7c1aede24380b91f1114/merged]\\nnvidia-container-cli: initialization error: cuda error: unknown error\\n\""": unknown.

Return of non-zero code:100 on build

Receiving an error code 100 when building the container. The command that is presenting the error is
/bin/sh -c apt-get update && apt-get install -y --no-install-recommends git wget ffmpeg unzip sudo && apt-get clean && rm -rf /var/lib/apt/lists/* returned a non-zero code: 100

Not the same notebook than in the video - still ok to use ?

Hello,

Jeremy Howard uses https://github.com/fastai/fastai/blob/master/courses/dl1/lesson1.ipynb
from repo fastai/fastai

You used the repo fastai/courses in your container. Is it recommended to reclone the fastai/fastai repo ?

If I should reclone another repo could you provide the commands to make jupyter server point at the new repo cloned in the container ?

Thanks for your answers.

Have a nice day !

How to get Password?

Sorry I'm new to this. I want to try the docker image but I cannot login to Jupyter Notebook. How can I get the password?

Restarting container fails

t# docker ps -a | grep d5a901fda160
d5a901fda160 deeprig/fastai-course-1 "/tini -- /bin/sh -c…" 3 days ago Exited (1) 2 hours ago jovial_johnson

docker start -i d5a901fda160

Traceback (most recent call last):
File "/opt/conda/bin/jupyter-notebook", line 6, in
sys.exit(notebook.notebookapp.main())
File "/opt/conda/lib/python2.7/site-packages/jupyter_core/application.py", line 267, in launch_instance
return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
File "/opt/conda/lib/python2.7/site-packages/traitlets/config/application.py", line 657, in launch_instance
app.initialize(argv)
File "", line 2, in initialize
File "/opt/conda/lib/python2.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/opt/conda/lib/python2.7/site-packages/notebook/notebookapp.py", line 1290, in initialize
super(NotebookApp, self).initialize(argv)
File "", line 2, in initialize
File "/opt/conda/lib/python2.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error
return method(app, *args, **kwargs)
File "/opt/conda/lib/python2.7/site-packages/jupyter_core/application.py", line 243, in initialize
self.migrate_config()
File "/opt/conda/lib/python2.7/site-packages/jupyter_core/application.py", line 169, in migrate_config
migrate()
File "/opt/conda/lib/python2.7/site-packages/jupyter_core/migrate.py", line 241, in migrate
with open(os.path.join(env['jupyter_config'], 'migrated'), 'w') as f:
IOError: [Errno 13] Permission denied: u'/home/docker/.jupyter/migrated'

error missing data

ln 9

vgg = Vgg16()
# Grab a few images at a time for training and validation.
# NB: They must be in subdirectories named based on their category
batches = vgg.get_batches(path+'train', batch_size=batch_size)
val_batches = vgg.get_batches(path+'valid', batch_size=batch_size*2)
vgg.finetune(batches)
vgg.fit(batches, val_batches, nb_epoch=1)

gives

OSError: [Errno 2] No such file or directory: 'data/dogscats/sample/train'

recommended fix?

Accessing other paths than where the fast-ai notebooks are located

Even though I can map folders to use inside the container, I can't seem to be able to create and run notebooks from that location. One solution that I think of is to leave the WORKDIR as /home/$USERNAME ?

deprecation warnings

WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10).  Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
 https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29

Using gpu device 0: TITAN X (Pascal) (CNMeM is disabled, cuDNN 5110)
Using Theano backend.

recommended fix?