yangw1234 / bigdl-tutorials-1

A step-by-step introduction to data science based on Apache Spark and BigDL framework

Deep Leaning Tutorials on Apache Spark using BigDL

Step-by-step Deep Leaning Tutorials on Apache Spark using BigDL. The tutorials are inspired by Apache Spark examples, the Theano Tutorials and the Tensorflow tutorials.

Topics

1. RDD
2. DataFrame
3. SparkSQL
4. StructureStreaming
5. Forward and backward
6. Linear Regression
7. Introduction to MNIST
8. Logistic Regression
9. Feedforward Neural Network
10. Convolutional Neural Network
11. Recurrent Neural Network
12. LSTM
13. Bi-directional RNN
14. Auto-encoder

Environment

• Python 2.7
• JDK 8
• Apache Spark 2.2.0
• Jupyter Notebook 4.1
• BigDL 0.3.0
• Setup env on Mac OS / Setup env on Linux

Start Jupyter Server

• Download BigDL 0.3.0(linux or mac) and unzip file.
• Run export BIGDL_HOME=where is your unzipped bigdl folder
• Run export SPARK_HOME=where is your unpacked spark folder
• Run ./start_notebook.sh

Run Demo

• Open a browser - Suggest Chrome or Firefox or Safari
• Access notebook client at address http://localhost:8888, open the example ipynb files and execute.

Note

• This notebook is for BigDL 0.3.0. Please refer branch-0.2 if you need to use BigDL 0.2.0.