the-full-stack / course-gitbook

Full Stack Deep Learning Online Course

description
Full Stack Deep Learning helps you bridge the gap from training machine learning models to deploying AI systems in the real world.

Full Stack Deep Learning

{% hint style="info" %} We are teaching an updated and improved FSDL as an official UC Berkeley course Spring 2021.

Sign up to receive updates on our lectures as they're released — and to optionally participate in a synchronous learning community.

Sign up for 2021**** {% endhint %}

About this course

Since 2012, deep learning has led to remarkable progress across a variety of challenging computing tasks, from image recognition to speech recognition, robotics, and audio synthesis. Deep learning has the potential to enable a new set of previously infeasible technologies like autonomous vehicles, real-time translation, and voice assistants and help reinvent existing software categories.

There are many great courses to learn how to train deep neural networks. However, training the model is just one part of shipping a deep learning project. This course teaches full-stack production deep learning:

• Formulating the problem and estimating project cost
• Finding, cleaning, labeling, and augmenting data
• Picking the right framework and compute infrastructure
• Troubleshooting training and ensuring reproducibility
• Deploying the model at scale

This course was originally taught as an in-person boot camp in Berkeley from 2018 - 2019. It was also taught as a University of Washington Computer Science PMP course in Spring 2020.

The discussion page for the course on Gitter.

The course project is on Github.

{% hint style="info" %} Please submit a pull request if any information is out of date or if you have good additional info to add! {% endhint %}

Who is this for

The course is aimed at people who already know the basics of deep learning and want to understand the rest of the process of creating production deep learning systems. You will get the most out of this course if you have:

• At least one-year experience programming in Python.
• At least one deep learning course (at a university or online).
• Experience with code versioning, Unix environments, and software engineering.

We will not review the fundamentals of deep learning (gradient descent, backpropagation, convolutional neural networks, recurrent neural networks, etc), so you should review those materials first if you are rusty.

Organizers

Guest Lectures

Newsletter

{% embed url="https://forms.gle/mDQZxsLZmep8JFgx9" caption="" %}

Course Content

{% page-ref page="course-content/setting-up-machine-learning-projects/" %}

{% page-ref page="course-content/infrastructure-and-tooling/" %}

{% page-ref page="course-content/data-management/" %}

{% page-ref page="course-content/ml-teams/" %}

{% page-ref page="course-content/training-and-debugging/" %}

{% page-ref page="course-content/testing-and-deployment/" %}

{% page-ref page="course-content/research-areas.md" %}

Guest Lectures

{% page-ref page="guest-lectures/xavier-amatriain.md" %}

{% page-ref page="guest-lectures/chip-huyen-nvidia.md" %}

{% page-ref page="guest-lectures/lukas-biewald-weights-and-biases.md" %}

{% page-ref page="guest-lectures/jeremy-howard-fast.ai.md" %}

{% page-ref page="guest-lectures/richard-socher-salesforce.md" %}

{% page-ref page="guest-lectures/raquel-urtasun-uber-atg.md" %}

{% page-ref page="guest-lectures/yangqing-jia-alibaba.md" %}

{% page-ref page="guest-lectures/andrej-karpathy-tesla.md" %}

{% page-ref page="guest-lectures/jai-ranganathan-keeptruckin.md" %}

{% page-ref page="guest-lectures/franziska-bell-toyota-research.md" %}

course-gitbook's Issues

Wishlist for FSDL v2

If we decide to do another version of the course, here are some new topics that could be exciting to add. This is off the top of my head, feel free to suggest other topics.

Bias / fairness

• Detecting and reducing bias in ML systems
• Ethics for ML practitioners

Deployment

• More complicated web serving scenarios (ensembles, graphs of models, low-latency, larger models)
• More prescriptive recommendations on deployment (how to do AB tests, shadow mode, instant rollbacks, etc)
• Model optimization (quantization, distillation, compression, etc)
• Edge / mobile deployment
• On-prem or data-sensitive deployment

Troubleshooting

• More specific pytorch recommendations

Testing

• More specific testing recommendations -- "test coverage" for ML, what to do when tests fail etc
• More on data slices, how to pick them, and how to manage them
• Testing suggestions for language data

Monitoring

• More on what to monitor
• How to set up a monitoring system

Data

• Managing data at a larger scale
• Managing user data for ML

Infrastructure / tooling

• Feature stores -- why, when, and how
• Logging infrastructure for ML
• Spark -- why, when, and how
• Tools for building reproducible data pipelines (Airflow, Kubeflow, etc)

Model lifecycle management

• How to know when to retrain models
• How to set up reproducible retraining pipelines
• How to select data for your next training run (active learning & friends)

Cleaning up READ ME

Love this course!

Just one thing, would the links in read me be cleaned up?

have a nice day!

Site Taking Too Long to Respond

After going to the site, the site is not getting open and after some seconds it shows's site taking too long to respond, and sometimes 182.79.218.34 refused to connect.

Slides for the course

Thanks a lot for creating this awesome course.

Are contributions to the pages in the course welcomed(for example: adding more text content to a page)? I ask because I see that a lot of valuable insights from the video are not captured in the corresponding summary text.