chromato99 / harmonyxr

This is a fork repo for developing

HarmonyXR

HarmonyXR is a project to implement real-time music performance in the virtual reality environment.

HarmonyXR is a project that uses WebXR. It builds a web server with Node.js, implements 3D graphics with Three.js, and implements real-time communication between users with WebRTC using PeerJS.

Project Features

• Virtual Music Stage
• Real-time multi-user experience
• Real-time Voice Chat
• Cross platform (Any platform you can use a browser that supports WebXR.)

Run on localhost

WebXR only supports HTTPS environment, so for testing, you need to issue an SSL certificate or enable #unsafely-treat-insecure-origin-as-secure in the chrome flags page.

docker build --tag harmonyxr .
docker run -d -p 8006:8006 --name harmonyxr harmonyxr

Open http://localhost:8006 on web browser.

Implementation

Networking Structure

It builds a web server using Nodejs and communicates with users using Socket.io to implement room functions and manage the user's connection status.

Then, real-time communication between users is implemented using WebRTC technology using PeerJS.
At this time, the real-time voice chat and location information are synchronized through user-to-user communication. This P2P communication is defined in /public/networking.js.

Virtual Environment

HarmonyXR implemented 3D graphics using Three.js.
To implement various things, MusicRoom, Partner, Piano, Drum, Guitar objects, etc. were defined.

We also implemented collision handling between controllers and instruments using the Box3 object of Three.js.

The 3D models were used free resources from Sketchfab.

TODO

• Update human models
• Optimize animation and rendering, Improve FPS
• There is no wall. it would be better to add wooden or cementic wall would be better to added
• Add More Instuments

Members

Beomgi Kim | [email protected]
Doyeon Hyun | [email protected]
Siwoo Lim | [email protected]
Jaejun Park | [email protected]

Reference

3D Models from Sketchfab
Icons from Flaticon