grey-software / toonin

Technology that allows you to tune in to your friends and family in realtime using peer-to-peer sharing.

Toonin

Toonin is technology that allows you to share or stream audio and video over the web in real time.

Overview

Toonin consists of a web app and a signaling server. WebRTC is used to stream the music from the provider to its peers.

The signaling server facilitates a WebRTC handshake between two Toonin web app instances, after which the two apps speak directly to each other over P2P WebRTC streams.

Development setup

yarn setup

Running Toonin

Toonin comes with a local development script that launches the server and app simultaneuously.

yarn dev

Running the web app independently

yarn run app

Running the signalling server independently

yarn run server

toonin's Issues

Turn signalling server infrastructure serverless

use secure web sockets wss for socket connections

Endpoints for app and extension should be defined in the root folder in .env file.

Add volume controls to vue app

Use https using letsencrypt cert for toonin.ml

Improve code for extension

Re-write Web app in Vue + Vuetify

Imrpove code for App.

Title not visible of host stream on vue app.

Add support for custom toonin room names.

Add Web App Support for MacOS/iPhone/iPad

Currently, the web app crashes on apple products or the audio doesn't work. The issue caused due to missing audio context on apple devices

Add proper state management for the vue client

Make the toonin box non-persistent

The toonin box currently persists on the DOM of the source stream tab. This is intrusive to the experience of someone using a service such as Youtube or Spotify.

Following chrome extension best practices, we should have our chrome extension's ui persist only as long as the user currently requires it to.

This would likely involve a re-design on the states that the chrome extension can be in as well so that the user sees the appropriate UI when they click the extension.

improve sound quality without the use of data channels.

Room names with slashes in them ("/") causes the app to break

Room names with "/" in them messes the call to backend server. For e.g., if room name == "event/room1", the server thinks we're making a "GET" request to the route event/room1 instead of the standard /:roomID dynamic route

Deprecate the React App

We should remove the React app from the master branch to reduce clutter.

Chrome extension is not rebuilt when files are changed

end room functionality doesn't exist.

WebRTC data channel audio streaming audio break-up

recieveChannel.onmessage = function(event) {
              
                  if(typeof(event.data) === "string") {
                      // recieve the offset from the src to compensate for extra data at the start of 
                      // every arraybuffer
                      firstBufferDur = Number(event.data);
                      console.log('offset: ' + firstBufferDur);
                  }
                  else {
                      audioContext.decodeAudioData(event.data, function(buffer) {
                          if(bufferQueue.length < 4) {
                              bufferQueue.unshift(buffer);
                          }
                          else {
                              bufferQueue.unshift(buffer);
                              nextBuffer = bufferQueue.pop();
                              soundSource = audioContext.createBufferSource();
                              soundSource.buffer = nextBuffer;


                              // here there is breakup because there is a gap between new buffer playing and old one ending


                              soundSource.start(audioContext.currentTime + prevBufferDur, firstBufferDur);
                              prevBufferDur = nextBuffer.duration;
                              soundSource.connect(soundDest); 
                              tabAudio.srcObject = soundDest.stream;
                              tabAudio.play();
                          }
                        
                        //visualizer();
                        
                    }, function(error) {
                        console.log('Audio Decoding failed. Error: ' + String(error));
                    });
                  }
              }

Extension visuals incorrect on windows/linux

No check on server if room exists or not.

Users cannot easily share their toonin room

We should allow users to copy their toonin room's link to their clipboard.

Clean up UI of toonin extension

looks messy, planning to hide some components until its needed.
Clarity for users

Add option for user to disconnect when listening to audio through the extension

Add support for tuning in via the chrome extension

How do users toonin at the moment?

Toonin currently supports tuning in via the web app by having the user enter the room name they want to enter.

How would users toonin via the extension?

I envision the user being able to enter the room name into the popup box that appears when the chrome extension is clicked.

**Would this open a new tab to the toonin web app or tune in on the current tab. **

We could make this configurable in the future, but I would imagine opening a new tab is the feature we're looking for, since you wouldn't want your toonin session to be tied to another tab's lifecycle.

Tuning in via routes doesn't work on production

The web app currently does not tune in via routes such as /roomName

This is because the app is served statically.

We may have to investigate to see if wrapping our Vue web app with a SSR framework like Nuxt would solve this for us.

Add volume controls to the toonin box

./vars issue for backend, need to figure another solution.

Show number of listeners in the extension

The web app should tune in to a room automatically if specified in the URL.

If a user enters toonin.ml/room-name, the web app should automatically toonin to the room with name 'room-name'

Add option for user to close the room in the extension.

Display connection/playback info on the webapp

Let the user know when the app is trying to connect to the room. Update the UI based on what step the app is at starting from searching for room to audio playback

Extension's CSS is messing up all the webpages even when not using it

Clean up chrome extension's build system

Audio Streaming through data channels has a clicking sound in the background

When streaming audio through data channels, there is a clicking sound that can be heard on the client along with the audio. Between multiple data packets of audio, the audio wave is cut abruptly at a point other than the natural zero crossing leading to a "click" sound.

Refer: http://alemangui.github.io/blog//2015/12/26/ramp-to-value.html

delete toonin room ids after 1-2 hours on server side.

Make toonin compatible with all platforms

Implement a distributed streaming system for scalability

Implement a distributed streaming system by converting peers of source as hosts depending on the load

Setup continuous integration

We should look into setting up continuous integration for toonin. Students get Travis CI as a part of their student developer pack.

Make Toonin extension compatible on Firefox.

https://extensionworkshop.com/documentation/develop/porting-a-google-chrome-extension/

Metrics (Google analytics)

have a general sense of how many people are using toonin and session time.

Warning on loading extension on chrome

A cookie associated with a cross-site resource at http://www.toonin.ml/ was set without the SameSite attribute. A future release of Chrome will only deliver cookies with cross-site requests if they are set with SameSite=None and Secure. You can review cookies in developer tools under Application>Storage>Cookies and see more details at https://www.chromestatus.com/feature/5088147346030592 and https://www.chromestatus.com/feature/5633521622188032.

After clicking start sharing, should also be able to stop sharing

chrome extension

Update Readme on github

The current readme file on github is outdated

Add debugging support for chrome extension

It is currently difficult to debug the extension because we only see the minified js. We could have an option to build dev and prod versions, with the former not being minified for debugging purposes.

Write a one command setup script

Toonin's DX could be improved by writing a one command setup script.

Install all reqs
Set IPs in app and ext
Build ext and output folder path
Serve app and output url
Run server

Add a config.json for global variables

let toonin users know of tab title

Create a separate file for all the global variables on the extension

Code looks messy right now. One of the main reasons is because we have global variables scattered all over the code, especially in the background.js file

Fix toonin.ml's favicon.ico

It currently shows the Vue favicon