udacity / dl_pytorch

Code for the Deep Learning with PyTorch lesson

Deep Learning with PyTorch

This repo contains notebooks and related code for Udacity's Deep Learning with PyTorch lesson. This lesson appears in our AI Programming with Python Nanodegree program.

• Part 1: Introduction to PyTorch and using tensors
• Part 2: Building fully-connected neural networks with PyTorch
• Part 3: How to train a fully-connected network with backpropagation on MNIST
• Part 4: Exercise - train a neural network on Fashion-MNIST
• Part 5: Using a trained network for making predictions and validating networks
• Part 6: How to save and load trained models
• Part 7: Load image data with torchvision, also data augmentation
• Part 8: Use transfer learning to train a state-of-the-art image classifier for dogs and cats

Archival Note

This repository is deprecated; therefore, we are going to archive it. However, learners will be able to fork it to their personal Github account but cannot submit PRs to this repository. If you have any issues or suggestions to make, feel free to:

• Utilize the https://knowledge.udacity.com/ forum to seek help on content-specific issues.
• Submit a support ticket along with the link to your forked repository if (learners are) blocked for other reasons. Here are the links for the retail consumers and enterprise learners.

dl_pytorch's Issues

Image missing in Part 1 - Tensors in PyTorch (Exercises).ipynb

The link in the row 56 is missing from the folder assets and the photo deoesnt show up

"<img src="assets/tensor_examples.svg" width=600px>\n"

Incorrect solution for Part 1 - Tensors in PyTorch(Solutions).ipynb (v2)

The notebook states that the answer is:

h = activation(torch.mm(features, W1)) + B1
output = activation(torch.mm(h, W2)) + B2

The output evaluates to tensor([[ 0.4813]]) which is also mentioned as the correct solution in the markdown in the below cell.

The correct answer, however, should be:

h = activation(torch.mm(features, W1) + B1)
output = activation(torch.mm(h1, W2) + B2)

which evaluates to tensor([[0.3171]]).

The video solution shows the correct code as the latter, but the cell is never run in the video segment. The expected answer as seen in the notebook from the video is tensor([[ 0.4813]]).

Typo should be fixed in Part 6

Under the subheading "Saving and loading networks" there is a sentence:

The simplest thing to do is simply save the state **** with torch.save. For example, we can save it to a file 'checkpoint.pth'.

I'm going to out on a limb and say you wanted to type "dict" with a T not a K.

Merge branch v2 to master for consistency with video lessons

I've been following the course and noticed that the videos follow the notebooks as they appear in branch v2. For example, The first exercise requires us to compute the output of a neuron, but this section is not available on the corresponding notebook on master branch.

fix HTML syntax error for cat picture to load

DL_PyTorch/Part 7 - Loading Image Data.ipynb

Line 123 in ff9bf7a

"<img src='assets/cat_cropped.png', width=244>"

Remove comma for the cat picture to load appropriately, correct formating:

"<img src='assets/cat_cropped.png' width=244>"

Parts 2-6: Fix for PyTorch 1.1

Current code in Parts 2-6 causes an error in PyTorch 1.1.

transform = transforms.Compose([transforms.ToTensor(),
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
])
RuntimeError: output with shape [1, 28, 28] doesn't match the broadcast shape [3, 28, 28]

I made the following change and it worked for PyTorch 0.4 and 1.1:

transform = transforms.Compose([transforms.ToTensor(),
transforms.Normalize([0.5], [0.5])])

confusing explanation

DL_PyTorch/Part 3 - Training Neural Networks.ipynb

Line 232 in ff9bf7a

"I'll build a network with `nn.Sequential` here. Only difference from the last part is I'm not actually using softmax on the output, but instead just using the raw output from the last layer. This is because the output from softmax is a probability distribution. Often, the output will have values really close to zero or really close to one. Due to [inaccuracies with representing numbers as floating points](https://docs.python.org/3/tutorial/floatingpoint.html), computations with a softmax output can lose accuracy and become unstable. To get around this, we'll use the raw output, called the **logits**, to calculate the loss."

"Often, the output will have values really close to zero or really close to one. Due to inaccuracies with representing numbers as floating points, computations with a softmax output can lose accuracy and become unstable. To get around this, we'll use the raw output, called the logits, to calculate the loss."

The description for the nn.CrossEntropyLoss() say: "This criterion combines :func:nn.LogSoftmax and :func:nn.NLLLoss in one single class.".

I'm a bit confused about this explanation, since the nn.CrossEntropyLoss() criterion contains a nn.LogSoftmax function.

fix typo in part 7

DL_PyTorch/Part 7 - Loading Image Data.ipynb

Line 42 in ff9bf7a

"dataset = datasets.ImageFolder('path/to/data', transform=transforms)\n",

DL_PyTorch/Part 7 - Loading Image Data.ipynb

Line 63 in ff9bf7a

"transforms = transforms.Compose([transforms.Resize(255),\n",

DL_PyTorch/Part 7 - Loading Image Data.ipynb

Line 101 in ff9bf7a

"transforms = # TODO: compose transforms here\n",

By naming the transform; transforms, we are overwriting the imported library; transforms.

DL_PyTorch/Part 7 - Loading Image Data.ipynb

Line 30 in ff9bf7a

"from torchvision import datasets, transforms\n",

I'm not sure if this was an intentional error left in as part of the lesson