ziyuehuang / dlsys

CSE599G1: Deep Learning System

Here are my solutions for the assignments of DLSYS.

Followings are the descriptions of the course.

Course Information

Over the past few years, deep learning has become an important technique to successfully solve problems in many different fields, such as vision, NLP, robotics. An important ingredient that is driving this success is the development of deep learning systems that efficiently support the task of learning and inference of complicated models using many devices and possibly using distributed resources. The study of how to build and optimize these deep learning systems is now an active area of research and commercialization, and yet there isn’t a course that covers this topic.

This course is designed to fill this gap. We will be covering various aspects of deep learning systems, including: basics of deep learning, programming models for expressing machine learning models, automatic differentiation, memory optimization, scheduling, distributed learning, hardware acceleration, domain specific languages, and model serving. Many of these topics intersect with existing research directions in databases, systems and networking, architecture and programming languages. The goal is to offer a comprehensive picture on how deep learning systems works, discuss and execute on possible research opportunities, and build open-source software that will have broad appeal.

We will have two classes per week. Each week will have one lecture. Another class will either be lab/discuss session or guest lectures. Each lecture will study a specific aspect of deep learning systems. The lab/discussion session will contain tutorials to implement that specific aspect and will include case studies of existing systems, such as Tensorflow, Caffe, Mxnet, PyTorch, and others.

Instructors

Tianqi Chen

Haichen Shen

Arvind Krishnamurthy

Teaching Assistant

Qiao Zhang

Prerequisites

Proficiency in Python, familar in C/C++

We will mainly be using python for case study the existing systems, and C/C++ for some of the background hacking.

Basic knowledge of machine learning (e.g CSE 546)

Prior knowledge in system (operating system/database) is useful but not required.

Acknowledgement

This course is created by University of Washington.