For my final project I have been thinking on how to connect what I have learned in the class and how to work with external inputs from devices like Raspberry Pi and Arduino. I wanted to somehow have my own control panel of connections I have setup, without relying on some hosting service to handle my data. For example, If I am having any one of my Microncontrollers sending any type of data from a sensor, I will be able to see it displayed in my webpage. This way, I can create a boilerplate for myself to store my data in some database online, and also have options of seeing multiple devices at the same time through some kind of personalized UI. Also, If I can input this data into my site I might be able to connect it to any other input or output, video stream, etc... I have two PIs and two arduinos laying around, so I might as well put them to use now.
I would start with documenting MongoDB's limits, to see if I need some other service that allows seamless updating. After the data can be fetched, I would grab the data stream and implement it into the UI. I would make an object for each of my sensors to send to my database, with a key/value pair which would correspond to :
{
"IP Address": "",
"Device Type": "",
"Timestamp":"",
"Sensor_1": "SensorData"
"Sensor_1_value":"",
}
So, the system setup in this project would work like this:
- RaspberryPI(any client really) -----> Server------>WebClient
- Server ------->MongoDB as a backup for data.
I think of it as a stepping stone towards my future "utility control monitor". I also want to make web interfaces to control things inside of other software with websockets functionality like Unity, Touchdesigner, and Runway.
So, everything kind of got tangled up. I set up a lot of individual parts and when joining them together they don't seem to function as well.
-
My sockets work with heroku, that's good, this means my heroku server/app are both remotely communicating with my raspberry pi and my web client. However, I am not getting through any messages from my pi to the heroku server. I suspect this is because the 'socket.io-client' package has a different method for emitting messages. I am in the process of finding out why.
-
I know the MongoDB setup works, and it's in the code lying dormant. It's just a matter of changing the data to send to the one I have received by the pi and/or any other device or socket.
-
The css and client.js are not getting served correctly, so the js code is the html
<script>tag and my css is invisible.
- This project runs on Node.js and npm. To set up the project you must have these installed and running. You can find more detailed instructions below.
- This project also has a database currently setup in MongoDB.
- Stable internet connection
- Download this repository to your computer.
- Open your terminal in the repository folder.
- Make sure you have git, npm, and node.js installed. You can check this with the following commands:
git -vnpm - vnode -v - If npm and/or node.js are not installed, you may follow this easy to follow guide from Tania Rascia. How to Install and Use Node.js and npm (Mac, Windows, Linux)
- Create a MongoDB Database and a Collection in the MongoDB Atlas Platform. More details of how to do this below.
- Once everything is installed, type in the following command while in the repository folder:
npm start - Open your preferred browser and in the address bar type in
http://localhost:3000/
To develop this document, you can follow the steps provided below:
- Create a fork of this project on Github.
- Ping the author of this repo via Github Issues to see if they are looking for contributions with the specific feature you're looking to add.
- Open the file in VS Code and make updates.
- Add and commit those changes in your forked Github repo.
- Make a pull request specifying what additions and changes were made.
- Have a nice chat and open communication with me about those changes.
- Celebrate the contribution!
- Joey Lee -- adjunct professor -- NYU ITP
- Cassie Tarakajian -- adjunct professor -- NYU ITP
- The Good Project Readme Project
I love the pixel art aesthetic, and I wanted a clean UI. I bumped into Jake Rossilli's work, specifically his 8 bit climate series. You can see some of them below. I really recently discovered I enjoy working with gradients, so it seemed like a good mix.
Jake Rossilli's 8bit climate series UI:
In different color combinations:
Still in progress. I am using a font called Pixel Operator. My logo is on the tio left corner.
-
First we have to make an account in MongoDB Atlas.
-
After that is done, we have to create a cluster. There should be something like the screen below in the "Clusters" Menu of our project. Here we press "Build a Cluster".
- Select create a cluster.
- Select a Cloud Provider and Region. And then press "Create Cluster".
- Wait for some time while it is created.
- Voila! A cluster is born. Here is where our data will live.
- On "Whitelist a connection IP address" we put 0.0.0.0 to have open access from anywhere. Also, we create a user and password.
- Finally, Connect your application will give us a link which we will add to our
.envfile in our server.
- Go into the Heroku website and make a new account. Then, in your main dashboard you should see a Createnew app option.
- Choose a name and a region for your app. This name can be changed later.
- You should see a dashboard like the one below. Since we are going to connect Heroku to our existing Git repository, choose the GitHub option in the Deployment Method.
- Copy the link to our repository into the submit form. Make sure the Github oraganization is correct.
- Enable Automatic Deployment, this way every time you update your Github Repository so will your Heroku website.
- To open your app go to the main dashboard and press 'Open app' on the upper right corner.
Now, in order to work with Heroku in our commandline we need to do two things:
-
Download the Heroku CLI. In Windows you can download it here.
-
Download or update your git. In Windows you can download it here.
While logged in into you account in the Heroku Website, use the heroku login inside of the repository folder assigned earlier. This should connect Heroku to the Github repo.
When you start looking around reliable realtime communication through the web, it is inevitable to run into Socket.io. After looking through several different resources, included this blog post, I decided to go for socket.io since 'The WebSocket API is the future of asynchronous messaging; Socket.IO is the best available resource for WebSocket in Node.js and within the browser.'. You can find the website for Socket.io here. We'll install it with npm. To use the code below I followed a page Joey sent me about setting up websockets with Heroku.
When we talk about sockets, we are using using a server which has a series of clients connected to it. Each client connects through an individual socket or TCP connection, which means that the server opens a socket, the client then connects to the server,which has a callback associated to each event that occurs. In this case, we will create en event called 'event' and send a message associated with that event every five seconds to each of our clients.
//Install dependencies
const express = require('express');
const socketIO = require('socket.io');
const INDEX = 'views/index.html'; // Define the index.html file address
const PORT = process.env.PORT || 3000;
const server = express()
.use((req, res) => res.sendFile(INDEX, {
root: __dirname
}))
.listen(PORT, () => console.log(`Listening on ${PORT}`));
/*The Socket.io server takes an HTTP server as
an argument so that it can listen for socket.io-related requests*/
const io = socketIO(server);
//Callback event for EACH client
io.on('connection', (socket) => {
console.log(`Client connected in socket ${socket.id}.`)
});
let message = "Hello";
//Send a message to every client every five seconds
setInterval(() => io.emit('event', message, 5000));
I had to establish an array with current connected users. Everytime the user disconnects the user is removed from the array and consequently, removed from the series of devices being displayed on the website.
Install socket.io-client with our terminal. The code below is enough for our client to receive and print a message form the server.
const socket = io();
socket.on('event', function (message) {
//receive the server's message
console.log(message)
});
If you have never setup a Raspberry Pi before, I would recommend following Tom Igoe's Setting up a Raspberry Pi of which I did a walkthrough blog post here.
You can find the repository for Socket.io here. We'll install it with npm with the npm install socket.io-client command in terminal.
The code below is enough for the pi to connect to our Heroku server. The idea is to send sensor data like temperature or humidity. You can see the code for working with sensors and the raspberry Pi with this Pi Environmental sensor I made.
var io = require('socket.io-client');
//Connect to the Heroku app
var socket = io.connect('https://websockets-playground-2020.herokuapp.com/', {
reconnect: true,
transports: ['websocket'],
path: '/socket.io'
});
console.log("Pi is on")
socket.on('connect', function (socket) {
console.log('Connected!');
});
socket.on('event', function (message) {
console.log(`Message from server: ${message}`);
});
Once you run the script you should see something like the image below. In the image below I decided to send the time as the message value.
In order to see the messages going towards the server in Heroku, we can use the heroku logs -a our-app-name in order the see the server console log.
Now, joining it all together. This is where I am at right now. The server is receiving messages from our web client pretty easily. Each socket connection is assigned a random value so I decided to log it's id and any message. This is made with the socket.emit() function. I made a console log to see what the message is as you can see below.
In our client I decided to add every client to the HTML file. I am in that process of making one object for each device connected to the server.
However, you can see the HTML is registering websockets connection in real time (pay attention to the timelog).
FIRST I HAVE TO SEND A LIST OF CONNECTED DEVICES, THEN I SHOULD SEND THE VALUES. For each socket in the array sent, make an object which is updated with each new message. Precisely, I will first send an array with the socket.id's which are connected to my server, then create an object inside my server which is updated with each new value.
- Don't know why I have trouble with my CSS and/or client.js getting served. This is mostly frustrating when I could not see any error message, and the error appears as if it is in
client.js, but the console shows an error in the html file.
- I did not know what should I put as the host for a websocket out of a local network. This opens up possibilities enormously. Thanks to August Luhrs and Cezar Mocan for helping me understand that. I also found some useful information in this StackOverflow post: How to connect to a remote server
- I saw this tool called Alooma Live, for working with data streams. Here is how it looks in action.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent