Machine Learning for the Web

This is a repository for the "Machine Learning for the Web" class at ITP, NYU

Libraries like TensorFlow.js and ml5.js unlocked new opportunities for interactive machine learning projects in the browser. The goal of this class is to learn and understand common machine learning techniques and apply them to generate creative outputs in the browser.

This class will start with running models in the browser using high-level APIs from ml5.js, as well as explore the Layer APIs from TensorFlow.js to train models using custom data. This class will also cover preparing the dataset for training models.

At the completion of this course, students will have a better understanding of a few machine learning models, how do they work, how to train these models, and their use case to creative projects. Students will also be able to create interactive ML web applications with pre-trained models.

Info

Yining Shi, Tuesdays, 6:30pm-9:00pm, Room 20, 10/30 - 12/11/2018
Office Hours
All class dates

Get started

To run each examples, open your terminal, type in the following commands:

$ git clone https://github.com/yining1023/machine-learning-for-the-web.git
$ cd machine-learning-for-the-web
$ python -m SimpleHTTPServer     # $ python3 -m http.server (if you are using python 3)

Go to localhost:8000 in your browser, you will see a directory list like this:

week1-intro/
week2-knnClassifier/
week3-styleTransfer/
week4-pix2pix/
week5-soundClassifier/
week6-workshop/

Click into each week, you will see the example for each week.

Syllabus

Week 1 Introduction to Machine Learning, MobileNet

Notes: Introduction to Machine Learning
Coding session:
- Installing ml5.js
- Running Image Classification example with ml5.js
- Installing tf.js
- Running examples with tf.js
Homework

Week 2 Image Classification (KNN classifier, PoseNet)

Notes
Coding session:
- Make a KNN Image Classifier
- Regression with feature extractor
- Posnet
- poseNet + KNN Image Classifier
Homework

Week 3 Image Transformation (Part 1) Style transfer

Review: Showing the homework from last week, discuss any difficulties and other creative outputs
Notes
Coding session:
- Setup Spell.run training environment
- Training a new Style Transfer model
- Running Style Transfer model in ml5.js
Homework

Week 4 Image Transformation (Part 2) pix2pix

Review: Showing the homework from last week, discuss any difficulties when you are training your own style trasnfer model
Notes
Coding session:
- Running pix2pix with ml5.js
- Setup Spell.run training environment
- Prepare dataset for pix2pix
- Training a new pix2pix model
Homework

Week 5 Final assignment proposal and guest lecture

More code examples and demo here
Review: Present your final assignment proposal and the experiments you did so far
Guest speaker: Runwayml

Week 6 Speech recognition, Sketch RNN, Body pix, Build your own model

Notes
Coding session:
- Body pix from tf.js-models
- Speech recognition from p5 speech.js
- Sketch RNN from ml5.js
- Build a doodle classifier with tf.js
Homework: Work on the final assignment, prepare the presentation

Week 7

Final Presentation

Resources

Requirements

You are required to attend all class meetings and submit all weekly assignments.
Grading (pass/fail) will be based on a combination of factors:
- Attendance, participation in class discussion, and engagement in other students' projects (25%)
- Quality of weekly assignments (50%)
- Last Assignment (25%)

sombreroelgringo / machine-learning-for-the-web Goto Github PK