glavin001 / rgb-neural-net

Physical visualisation of neural network learning using RGB leds, arduino and raspberry pi.

RGB Neural Net

IOT physical neural network visualisation

What & Why?

This projects goal was to build an intuiative and visually interesting way of seeing a neural network learn. I built it becuase I had some empty wall space.

A separate computer runs the neural network training program and communicates with the RGB Neural Net over WiFi.

Using the RGB Neural Net

First I generate some data that I want the neural network to learn. In this case it's some multi label classification data, meaning that given an 'x' and a 'y' value (2 continuous features) a data point can either have label A, label B, both label A and B, or no label assigned to it. i.e Label A could be 'if someone likes apples', and label B couble be 'if someone like oranges', and therefore someone could like one, or the other, or both, or neither. There generated data looks like the following:

Then in our program (python + sklearn) when create a neural network that is the same shape as the RGB one (1 hidden layer with 3 nodes).

nn = MLPClassifier(hidden_layer_sizes=(3), 
                   activation='logistic', 
                   learning_rate_init=0.02, 
                   max_iter=1, warm_start=True)

Then import the RGB library and create the connection to the RGB neural network:

from rgb_nn import RGB_NN
rgb = RGB_NN(server_loc='http://192.168.1.153:5000')

Finally just loop over the fitting of the code neural network along with the function to update the RGB neural network:

for i in range(50):
   nn.fit(X_train, y_train)
   rgb.display_weights(nn)

Then sit back and watch the learning happen:

What the colours mean

The colours of the connections represents the value of the weights between nodes (RED - Low, BLUE - 0, GREEN - High). The colours of the nodes themselves represent the bias value.

Techincal components

Harware:

• 3D printed nodes
• Fibre optic tubing
• RGB LEDs
• Arduino Uno
• Raspberry Pi Zero

Software:

• RGB LED strip driver with serial com script on Arduino
• Flask API server running on Raspberry Pi taking requests and send intruction to Arduino via serial
• Python library that converts SciKit-Learn neural network model into the API requests for the Raspberry Pi

Its making

Installing arduino to raspberr pi

sudo apt-get install arduino

Turning on 20th LED from raspberry pi

import serial
arduinoSerialData = serial.Serial('/dev/ttyACM0',9600)
arduinoSerialData.write('10010010020\n')

Flask server API command

POST /change_leds sets the rgb values of multiple leds at once.

Json body:

{
    "leds":
    [
        { "red": 0, "green": 200, "blue": 100, "led_num": 1 },
        { "red": 0, "green": 50, "blue": 100, "led_num": 2 },
        { "red": 0, "green": 50, "blue": 100, "led_num": 3 },
        { "red": 0, "green": 50, "blue": 100, "led_num": 4 }
    ]
}

^ this probably should have been a single dict with the key being the led number... ¯\(ツ)/¯

To Do

• Design neuron to 3D print
• Print nodes
• • Input layer
• • Hidden layer
• • Output layer
• Build electrics for nodes
• • 1 node
• • 2 node
• • 3 node
• • 4 node
• • 5 node
• • 6 node
• • 7 node
• Embbed electronics nodes
• • 1 node
• • 2 node
• • 3 node
• • 4 node
• • 5 node
• • 6 node
• • 7 node
• Test wire up
• Optimise arduino code to update LEDs in batches
• Optimise flask code to use new arduino code
• Design mounting solution for nodes
• Mount nodes on board
• • 1 node
• • 2 node
• • 3 node
• • 4 node
• • 5 node
• • 6 node
• • 7 node
• Optically connect nodes
• • 1 node
• • 2 node
• • 3 node
• • 4 node
• • 5 node
• • 6 node
• • 7 node
• Create LED mapper (to map to neuron weights)
• • Refactor server sending code into one doctument that:
• • Sets an led to a value (one number converts to RGB)

Optional

• Get canvas
• Mount on canvas

Resources

http://www.meccanismocomplesso.org/en/controlling-arduino-raspberry-pi/

https://forum.arduino.cc/index.php?topic=405287.0

https://github.com/FastLED/FastLED

Notes

Serial communication was becoming incredably slow when testing sending alot of requests. This turns out to be becuase I had the arduino printing to the serial line whilst the raspberry pi was not reading it. This filled up the buffer of the arduino and puts a 1 second delay on all further communication. https://arduino.stackexchange.com/questions/22816/serial-communication-dead-slow-after-a-while

When starting the server, the first led can become frozen at its initial colour, restarting the power to the strip will fix this.

Cooling: it seems that the little pi zero can get a bit flustered running as a server and thus either helping it with some cooling or upgrading to a beafier pi board is nessasary.

Progress Images

Intital 3d design of nodes: First and second interation of input/output node: All nodes printed: Soldered node lights: Electronics fitting in node shell: All electronics fitted in nodes All nodes connected and lighting up Nodes mounted on boards Nodes connected by fiber optic All nodes connected by fiber optic High and low colour weights

Finaly!!!

RGB network connected up and displaying weights in real time from a learning sklearn NN multi-label cassifier model: