hulalazz / the_evolution_of_gradient_descent

This is the code for "The Evolution of Gradient Descent" by Siraj Raval on Youtube

The_evolution_of_gradient_descent

This is the code for "The Evolution of Gradient Descent" by Siraj Raval on Youtube

Coding Challenge - Due Date, Thursday June 8th at 12 PM PST

This weeks coding challenge is to write out the Adam optimization strategy from scratch. In the process you'll learn about all the other gradient descent variants and why Adam works so well. Bonus points if you add a visual element to it by plotting it in a Jupyter notebook. Good luck!

Overview

This is the code for this videon on Youtube by Siraj Raval. In the video, we go over the different optimizer options that Tensorflow gives us. Under the hood, they are all variants of gradient descent.

Dependencies

• matplotlib
• pyplot
• numpy

install missing dependencies with pip

Usage

Run jupyter notebook to see the code that compares gradient descent to stochastic gradient descent run in the browser. I've also got 2 seperate python files, one for adadelta and one for the nesterov method. Run those straight from terminal with the python command.

Credits

The credits for this code go to GRYE and dtnewman. I've merely created a wrapper to get people started.