hpcc-systems / gpu-deep-learning

GPU-Deep-Learning

GPU Accelerated Neural Networks on HPCC Systems Platform

Bundle for building, training, and consuming neural networks on HPCC Systems with GPU acceleration.

Description

Large nueral networks typically train on very large datasets. The size of the data combined with the size and complexity of neural networks result in large computational requirements. Until now, HPCC Systems is primarily a CPU based system which can result in neural network training times that are impractcally long. With the use of modern GPUs, the training time can be drastically reduced over using CPUs alone.

Getting Started

Requirements

You must have a compatible NVIDIA GPU for the GPU acceleration to work, however this bundle will work on CPU alone, albeit significantly slower. CPUs are significanlty slower when it comes to training neural networks. If you do not have GPU/s available, see Distributed-Deep-Learning for use on CPU only instances.

There is a AWS AMI that was created for use with this bundle. It is generated using this. This produces an image with HPCC Systems Platform Community edition, version 7.2.14 pre-installed as well as all other requirements for this bundle, including CUDA version 10.0. The image is designed to run on Amazon's P2 or P3 machines.

The Bundle is built around various open source Python libraries and requires you to use Python 3. This also means any custom runtims written in Python, need to adhere to the Python 3 specifications. Additionally, and perhaps most importantly, your cluster needs to be using the Python 3 version of pyembed.

In /etc/HPCCSystems/environment.conf on your node/s on your cluster:

sudo nano /etc/HPCCSystems/environment.conf

Change the the line with the "additionalPlugins" variable to be:

additionalPlugins=python3

Then stop and start your cluster:

sudo systemct stop hpccsystems-platform.target
sudo systemct start hpccsystems-platform.target

Included Training Data

Included are some popular Datasets used in experimenting with neural networks slightly modified for easy spraying on the HPCC Systems Platform. The scripts used to generate the modified datasets is also included, which uses the original datasets as a staring point.

• MNIST, see reference
• Fashion MNIST, see reference
• IMDB
• Boston Housing, see reference
• Reuters

Spraying

Spray in the following way, with similar names, for the examples to work without modification.

• MNIST: Fixed size = 785
  • Train: mnist::train
  • Test: mnist::test
• Fashion MNIST: Fixed size = 785
  • Train: fashionmnist::train
  • Test: fashionmnist::test
• IMDB: CSV
  • Train: imdb::train
  • Test: imdb::test
• Boston Housing: CSV
  • Train: bostonhousing::train
  • Test: bostonhousing::test
• Reuters: CSV
  • Train: reuters::train
  • Test: reuters::test

Examples

Included in this bundle are some examples, found in the examples directory. It is recomended to start with an MLP trained on MNIST

Defining a Neural Network

There are three different ways you can build neural networks with this bundle.

All three methods provide an example on how to build a 10-class MLP and train on the MNIST dataset. The MLP has 2 hidden layers of size 512, each using the "relu" activation fucntion, and a 10 class output layer with "softmax" as the activation. All train for 20 epochs, using batch size of 128, and it is expected to achieve a Test accuracy of roughly 98%. The CNN is slightly different and gets 99% accuracy.

Using only ECL

The first and easiest way is to use one of the predfined architecture types and define how many layers and neurons you want. You can also hyperparameter tune using this method. For example, you can choose to build an MLP, and then define the architecture specifics, by passing in a SET of INTEGER, such as number of hidden layers, number of neurons in the each layer, the activation functions. This is done all in ECL.

• See the MLP ECL example.

The second method gives you more control over the arhictecture. Use the "model.add" part of the module to iteratively add layers until the desired architecture is realized.

• See the MLP example.
• See the CNN example.

Custom Architectures in Python

The thrid method is the most complex in that it requires you to have working knowledge of Python and the underlying library. You can define a "custom" neural network architecture using Keras or PyTorch and train via HPCC/ECL. Using this approach, any keras defined model will easily be traininable and consumable on your HPCC Systems cluster.

• See the MLP example.
• See the CNN example.

Training Data Format

Current included datasets and examples train on data in a specific HPCC format. The records in the dataset are then converted into numpy arrays, a very popular and robust way of representing scintific data in Python. For example, the MNIST data set is of the following format:

mnist_data_type := RECORD
	 INTEGER1 label;
	 DATA784 image;
END;

Here you can see that the first element is the label and the next element is a single DATA784 that includes all of the pixel data in it. The data is then converted into a numpy array with a shape of (60000, 784). Meaning an array 60,000 long (one for each of the 60k images) where each element in that array is another array with 784 pixels (28 x 28 images, where the each row is layed end to end).

This can be seen in the MNIST data loading function in here. A more generalizable method to load other types of HPCC datasets into numpy arrays is needed, and is included in the TODO section.

Using the Model (Inference)

If you persist the model and use the model.predict() fucntion, you can use a trained model to make predictions on any incoming data on your HPCC cluster, as long as it has been prepared to the same format as the training data. The prediction method outputs the result/s in one-hot-encoded form in the following format:

oneHot := RECORD
  SET of INTEGER class;
END;

A one-hot-encoded format for a 10 class output would be a set of 10 integers, all of which are 0, except for one. The index of the 1 will be the class that row of data was predicted to be. i.e. a row that is predicted to belong to class 1 would be:

[1 0 0 0 0 0 0 0 0 0 0 ]

TODO:

This is the planned future work that will expand upon this bundle:

• Continue adding supported Keras layers into layers.ecl
  • Full list starts here
• Make custom data handlers for importing different-formatted training data
  • i.e. a more generalized data loader that takes HPCC Datasets and converts them into numpy arrays
• Add support for other commonly used training data (input) formats

Author

Robert K.L. Kennedy | GitHub | LinkedIn