mirkogelsomini / vr_storytellers

VR_Storytellers

The project generates the viewable Virtual-Reality content from text/story entered on a website in real time. It dynamically generates models, objects, scenes (background), coupled with a background sound based on the characters, actions and mood on the Story plot.

Any story (as plain text) is entered on a website, which is then passed on to an algorithm which analyzes and figures out the main objects, their descriptions, the background and the overall mood of the story using machine learning and natural language processing on the Cloud (Amazon Web Services). The processed data is then referenced by Unity and analyzed to pull up assets and related animations based on the objects and choose background sounds based on the mood of the plot. The objects are placed relative to each other with the appropriate background and a background music is played. This entire experience is viewable on Google Cardboard. We typically chose Cardboard as it is one of the cheapest VR device available with the best experience, hence more users would get access to the app.

The team-project was awarded the top prize in 2 categories of the MIT Media Lab's VR/AR hackathon, namely the "Most Refined Mobile VR Experience" and "Best Up & Coming Hackers".