arthurdouillard / deepcourse

Learn the Deep Learning for Computer Vision in three steps: theory from base to SotA, code in PyTorch, and space-repetition with Anki

Bonjour ! I'm arthur, a French coder / researcher / teacher / foodie.

1. Most of the code I did during my PhD is available publicly on this github account.

2. I do research in Continual Learning at DeepMind, where I aim to train large neural networks on continually growing amount of tasks. You can see my work on my Google Scholar page. Don't hesitate to contact me to chat about that.

3. I'm the author of a free course on Deep Learning for Computer Vision --> Deepcourse You'll learn from the basics (backprop) to the latest research done in a wide variety of domains. Each topic has lessons, coding exercices (with hidden solutions), and Anki flashcards.

4. I'm french, of course I'm a foodie.

deepcourse's Issues

Error When Running Automatic Differentiation with Autograd

When running the last two code blocks in the Autograd notebook I received the following error:

---------------------------------------------------------------------------

RuntimeError                              Traceback (most recent call last)

[<ipython-input-46-50b2a169b280>](https://localhost:8080/#) in <cell line: 25>()
     23 
     24 plt.subplot(1, 2, 1)
---> 25 plt.plot(epochs, losses)
     26 plt.xlabel("Epoch")
     27 plt.ylabel("Loss")

7 frames

[/usr/local/lib/python3.10/dist-packages/torch/_tensor.py](https://localhost:8080/#) in __array__(self, dtype)
    968             return handle_torch_function(Tensor.__array__, (self,), self, dtype=dtype)
    969         if dtype is None:
--> 970             return self.tensor.detach().numpy()
    971         else:
    972             return self.tensor.detach().numpy().astype(dtype, copy=False)

RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.

I was able to fix it by changing the line:

losses.append(loss)

to

losses.append(loss.detach())

TP MLP htilde

Code solution backward pass for mlp, swtich grad_h by grad_htilde

Transformer colab

• uniformize C and D
• add patch embed after class token in answer to Vit
• typo in solution openning to multi-head
• forgot square root in self-attention

Add mention to other transformer models

• Swin
• DETR

Check that website appear well in other browsers

Apparently Chrome doesn't display the "quiz" text

Less pre-written in Transfer Learning

Let students code their own training loop

Add transparance on tsnet dot plot for domain adaptation

Add relu in solution to resblock

?

Coding auto-diff

Pro:

• really understand better backprop
• interesting parts especially with convolutions & pooling

Con:

• maybe a bit long