sotte / pytorch_tutorial

PyTorch Tutorial (1.7)

PyTorch Tutorial

This repository contains material to get started with PyTorch v1.7. It was the base for this [pytorch tutorial] from PyData Berlin 2018.

---

Table of Contents

PART 0 - Foreword

• Foreword - Why PyTorch and why not? Why this talk?

PART 1 - Basics

• PyTorch basics - tensors, GPU, autograd - open in colab
• Debugging - open in colab
• Example: linear regression - open in colab
• Storing and loading models - open in colab
• Working with data - Dataset, DataLoader, Sampler, transforms - open in colab

PART 2 - Computer Vision

• Transfer Learning - open in colab

PART 3 - Misc, Cool Applications, Tips, Advanced

• Torch JIT - open in colab
• Hooks - register functions to be called during the forward and backward pass - open in colab
• Machine Learning 101 with numpy and PyTorch - open in colab
• PyTorch + GPU in Google's Colab
• Teacher Forcing
• RNNs from Scratch - open in colab
• Mean Shift Clustering - open in colab

PART -2 - WIP and TODO

• TODO nn and nn.Module
• TODO Deployment
• TODO Deployment with TF Serving
• TODO nn.init
• TODO PyTorch C++ frontend

PART -1 - The End

• The End

---

Setup

Requirements

• Python 3.8

Install Dependencies

python3.8 -m venv .venv
source .venv/bin/activate.fish
pip install -r requirements.txt

Optional

Run the following to enable the jupyter table of contents plugin:

jupyter labextension install @jupyterlab/toc

jupyter nbextension enable --py widgetsnbextension

Download data and models

Download data and models for the tutorial:

python download_data.py

Then you should be ready to go. Start jupyter lab:

jupyter lab

Prior Versions

• Version of this tutorial for the PyData 2018 conference: [material] [video]

pytorch_tutorial's Issues

Could not install conda environment on OS X

Hi Stefan,

first of all, thanks for providing the install instruction for the tutorial in advance :-)
However, I am struggling to install the dependencies on Mac OS X (e.g. the libgcc-ng package). Does the tutorial only work on Linux?

Why `loss.backward(loss)` in the Linear Regression example training

Thanks for putting up the notebooks and the code! I enjoyed watching your presentation and following along with the notebooks.

One question that bothered me is why you put the loss back in as the gradient for the backward method: loss.backward(loss) when training the linear regression network in the 01_pytorch_basics.ipynb notebook. Is this just an error?

Not all notebooks can run on colab without modification

rm utils.py dependency.

just want to say thanks

today, I went through this tutorial of yours and it really helped me in learning basics.
Thanks. :-)

Loss and accuracy calculation

Hallo.

Great tutorial!
In your loss and accuracy calculations you divide the loss and correct with the length of the entire data set like this:

Train Loss: running_loss / len(train_dl.dataset)
Train Acc: correct / len(train_dl.dataset)

Shouldn't it be the loss and correct divided by the batch size? Then you get the accuracy of the current batch like this:

    ite = 0
    for i, (X, y) in enumerate(train_dl):
        X, y = X.to(DEVICE), y.to(DEVICE)

        optimizer.zero_grad()
        y_ = model(X)
        loss = criterion(y_, y)
        loss.backward()
        optimizer.step()

        # Statistics
        _, y_label_ = torch.max(y_, 1)
        correct += (y_label_ == y).sum().item()
        running_loss += loss.item() * X.shape[0]
        ite = i
    print(f"  Train Loss: {running_loss / ite+1}")
    print(f"  Train Acc:  {correct /  ite+1}")

Edit: Nevermind, i am an idiot. Because you go through x images per batch you divide with number of images and what i do is dividing the loss and correct with number of batches. I will never get the correct loss

Just close this issue

2nd Edit: Okay so i found this https://discuss.pytorch.org/t/interpreting-loss-value/17665/9
They claim that the loss should be calculated in a different way. Who is right i cannot say.

One of the best PyTorch that I ever encountered!

I want to say that is a great source of learning, especially because it treats practical problems like:
Transfer Learning, PyTorch in production, along with many other interesting topics.
Most of the tutorials that I found online are just a "getting started kit" with 90% of the work focused on very basic stuff, while the other tutorials that I found were really hard to understand in the sense that the code wasn't explained and the solutions were just given at a plate along with the fact that they were not practically oriented.

A big round of applause for you, sir!
Can't wait for the final parts of the tutorial to be released!

ModuleNotFoundError: No module named 'ppt'

Sorry, but I got this error and don't know what should I do!!
I installed python-pptx, but it didn't work.
could you please tell me how to install that module?
thank you!

Link of cheetsheet doesn't work!