Read this in other languages: 한국어.
In this developer journey we will create a Virtual Reality game based on Watson's Speech-to-Text and Watson's Conversation services.
In Virtual Reality, where you truly "inhabit" the space, speech can feel like a more natural interface than other methods. Providing speech controls allows developers to create more immersive experiences. Google Cardboard is by far the most popular VR headset platform, with 84 million sold in 2016 (http://www.hypergridbusiness.com/2016/11/report-98-of-vr-headsets-sold-this-year-are-for-mobile-phones) in 2016.
When the reader has completed this journey, they will understand how to:
- Add IBM Watson Speech-to-Text and Conversation to a Virtual Reality environment build in Unity.
Want to take your Watson app to the next level? Looking to leverage Watson Brand assets? Join the With Watson program which provides exclusive brand, marketing, and tech resources to amplify and accelerate your Watson embedded commercial solution.
- IBM Watson Conversation: Create a chatbot with a program that conducts a conversation via auditory or textual methods.
- IBM Watson Speech-to-Text: Converts audio voice into written text.
- Unity: A cross-platform game engine used to develop video games for PC, consoles, mobile devices and websites.
- Google Cardboard: An inexpensive viewer that mounts an Android phone to view VR apps.
On your local machine:
git clone https://github.com/IBM/vr-speech-sandbox-cardboard.git
cd vr-speech-sandbox-cardboard
In Bluemix:
- Create a Speech-To-Text service instance.
- Create a Conversation service instance.
- Once you see the services in the Dashboard, select the Conversation service you created and click the button.
- After logging into the Conversation Tool, click the button.
- Import the Conversation
workspace.json
file located in your clone of this repository.
If you followed the previous steps you should already be inside your local clone and ready to get started running the app from Unity.
git clone https://github.com/IBM/unity-sdk
- Open Unity and inside the project launcher select the button.
- Navigate to where you cloned this repository and open the "SpeechSandbox" directory.
- If prompted to upgrade the project to a newer Unity version, do so.
- Follow these instructions to add the Watson Unity SDK downloaded in step 1 to the project.
- Follow these instructions to add your Speech To Text and Conversation service credentials (located on IBM Bluemix).
- Select
Advanced Mode
in the configuration window. - Open the script vr-speech-sandbox-cardboard/SpeechSandbox/Assests/Scripts/VoiceSpawner.cs and put your workspace ID on line #34 in the Start() method.
You can find your workspace ID by selecting the expansion menu on your conversation workspace and selecting
View details
. - In the Unity editor project tab, select Assets->Scenes->MainGame->MainMenu and double click to load the scene.
- Press Play
- To Build an android .apk file and deploy to your phone, you can File -> Build Settings (Ctrl + Shift +B) and click Build.
- When prompted you can name your build and then move it to your phone.
- Alternately, connect the phone via USB and File-> Build and Run (or Ctrl+B).
Make sure you have enabled USB Debugging:
-
Open Settings-> About-> Software Information-> More
-
Then tap “Build number” seven times to enable Developer options.
-
Go back to Settings menu and now you'll be able to see “Developer options” there.
-
Tap it and turn on USB Debugging from the menu on the next screen.
Once the app is deployed to your phone it will start, but you'll need to set permissions for the app before it will work correctly:
- Open Settings-> Apps-> SpeechSandboxCardboard-> Permissions and enable Microphone and Storage.