wvsharber / deployment-seattle-ds-102819

Deployment

SWBATs

• Explain the purpose of deploying a machine learning model
• Understand the purpose of pickling a model
• Understand the basics of some popular deployment techniques
• Start formulating a deployment plan for their business case

Flask

Flask means developing a full-stack web application. It follows a model-view-controller (MVC) pattern and requires that you sometimes have to follow a "convention over configuration" pattern and put certain files in certain folders.

There are two main ways I recommend deploying a Flask app: Heroku, or AWS EC2. The main difference is that Heroku uses "dynos", a type of container that gets fully re-created when the site is "woken up" based on certain config files it finds in a GitHub repo. EC2 is more like a "real" computer, where you can SSH in and download things, log out, log back in, and those things will still be there. EC2 gives you more configuration capabilities and more computational power, but also requires more setup than Heroku.

Pricing

Heroku free tier allows one web app for free. AWS free tier allows 750 hours of "micro" level server time per month.

Pros

• Free
• Allows you to become more familiar with web development and have a shared vocabulary with software developers
• You can generate data visualizations in Python (don't need to learn JavaScript, only HTML and CSS)
• Supports a multi-step process with more than one user interaction
• Single solution to get a live user interface you can link from your portfolio

Cons

• Learning curve for all of Flask can be steep
• More systems administration work than a cloud function or ML model

Google Cloud Functions

Google Cloud Functions allow you to make a single function that can be called from a REST API interface. The Python language versions are actually running on top of Flask, with as much boilerplate removed as possible. So you still need to learn some Flask to be able to get everything working.

Pricing

Up to 2 million free Cloud Function invocations per month. Free tier requires signing up with a credit card, but promises "No autocharge after free trial ends"

Pros

• Free
• Makes a REST API interface, so you can demo with Postman or an HTML + JS web frontend (or collaborate with an SE alum to make a React frontend for you)
• Only requires you to learn a little bit of Flask. Nothing about production WSGI servers, or nginx, or ports.

Cons

• Not making a full-stack web app means you won't have the experience of making a full-stack web app
• Sometimes there is a lot of "magic" happening with the implicit Flask app that makes it hard to debug. Probably will not be compatible with TensorFlow, for example, because you don't have fine-grained control over the threading behavior.
• If you want to deploy a web frontend, you'll need to find another tool. (GitHub Pages works fine though!)
• The platform is not designed specifically for ML models, so it might be annoyingly slow, or even so slow that it times out.
• You can't generate any data visualizations in Python, they will have to be in JavaScript

Documentation

1. Writing cloud functions
2. Deploying cloud functions
3. Calling cloud functions

Example

I have a working example of a Google Cloud function in the google_cloud_function folder. You can call it right now by pasting this into your terminal:

curl -X POST -H "Content-type: application/json" -d '{"Alcohol": 12.82, "Malic acid": 3.37, "Ash": 2.3, "Alcalinity of ash": 19.5, "Magnesium": 88.0, "Total phenols": 1.48, "Flavanoids": 0.66, "Nonflavanoid phenols": 0.4, "Proanthocyanins": 0.97, "Color intensity": 10.26, "Hue": 0.72, "OD280/OD315 of diluted wines": 1.75, "Proline": 685.0}' https://us-central1-splendid-petal-256700.cloudfunctions.net/wine-predictor

Google Cloud AI Platform

Google Cloud AI Platform allows you to upload an exported model to the cloud, then make future requests to the cloud to make new predictions. The primary use case seems to be people who are already very integrated into the Google Cloud ecosystem, since they encourage requesting predictions through their custom SDK rather than through a standard REST API interface.

Pricing

It looks like there is no free tier for this service. Pricing summary here.

Pros

• After installing their CLI, you only need to upload the model itself, no additional libraries or frameworks
• If you need better performance, it's easy to pay more money to get that
• This is a "real" tool that you might encounter on the job
• If you need to use TensorFlow, Google made both TensorFlow and this product, so they're likely to be the most compatible

Cons

• Not free
• No clear, easy way to make an "open" REST API that can communicate with Postman or a simple web frontend
• Requires versioning and other configuration work that is not necessary for our projects

Documentation

1. Exporting models for prediction
2. Deploying models
3. Getting online predictions in Python, Getting online predictions via REST