satoshirobatofujimoto / firstrobotics

Example of a simple TCP/IP socket server for the HoloLens

First Robotics HoloLens TCP/IP server example

This is a starter Unity project for the NASA FIRST Robotics teams that are piloting the use of the HoloLens in the competition. It is a simple TCP/IP server that runs on the HoloLens and accepts connections from applications on the driver's station over a USB cable. Teams should modify and extend this project to do interesting things with the data that's received.

This project only works with a special build of the HoloLens operating system, so if you've stumbled across this project and you aren't on one of the aforementioned NASA teams, sorry - this code won't work for you. In addition, a special policy exception has been installed to the HoloLens devices that have been shipped to the NASA teams that is specific to only this application. This is why teams need to download and modify this application instead of starting from scratch - a different application will not be covered by the same exception.

If you're new to HoloLens development, then we don't recommend starting with this project. You should instead start by learning the basics of HoloLens development from Microsoft's documentation at https://developer.microsoft.com/EN-US/WINDOWS/HOLOGRAPHIC/ACADEMY. We highly recommend that you complete at least the "Holograms 100" and "Holograms 101" classes before trying to work with this project. If you're also new to Unity development, you'll want to learn more about that at https://unity3d.com/learn.

Setup

Communication from the driver station to the HoloLens via a USB cable depends on a service called IPoverUSB. We configure this service to reroute connections made to port 1000 on the driver station to port 1000 on the HoloLens. To set this up, you'll need to follow these steps on every computer that you want to be able to communicate with the HoloLens over USB:

1. Install Visual Studio 2015 Update 3 or later from this page: https://developer.microsoft.com/en-us/windows/holographic/install_the_tools. This will install the IpOverUSB service. This is the same software you have to download for all HoloLens development.
2. Double-click on the ipoverusb-first.reg file in the Setup folder of this project. This will add a registry key to the computer that configures an IPOverUSB mapping from port 1000 on the local computer to port 1000 on the HoloLens.
3. Either reboot the computer, or open an elevated command prompt by pressing Win-X and selecting "Command Prompt (Admin)" and run the following commands:

sc stop IpOverUsbSvcT
sc start IpOverUsbSvc

You can confirm that IpOverUsbSvc is functioning by running the following commands in an elevated command prompt:

sc query IpOverUsbSvc

Building and testing this application

Once you've performed the setup steps above you should be able to build this project from Unity following the same process described in the HoloLens developer documentation. You want to build the "SocketTest" scene from the "Socket" folder.

Be sure to launch the application on the HoloLens in debug mode from Visual Studio so that you can see the console messages. Then run TestConsoleApp.exe from the root folder of this project. It will connect to the HoloLens app and transmit a series of characters. You should see output from both programs indicating that the connection and transmission was successful.

Extending this application

After you've built and successfully tested this application, you can start making extensions by modifying the bytesReceived method in SocketBehavior.