rje / interaction_urp

URP Shaders for the Meta Interaction SDK

URP Shaders for the Meta Interaction SDK

What's the Interaction SDK?

Meta recently released a new experimental API for their SDK called the Interaction SDK. It provides several useful tools to enable hand tracking based interaction to your VR apps.

So What's This, Then?

Currently, the SDK only supports Unity's Built-In Render Pipeline by default. For folks who have made the jump to Unity's Universal Render Pipeline, by default most of the objects will just show up in magenta.

I wanted to be able to use a lot of the tools and samples in my prototypes, so I took on the effort of porting the custom shaders in the SDK to URP.

The package contains all of the updated shaders, copies of the materials that use the new shaders, and copies of the sample scenes that point to the updated materials. I also included a prefab variant of the sample interaction rig so you can just drop that into your own work.

How Do I Use It?

There are really two ways you could use this, importing into your existing URP project, or creating a new project from scratch and working from there.

Existing Projects

I am assuming you already have the following:

1. A project that includes Oculus SDK v37.
2. The project is configured to use URP instead of the built-in renderer.

From there, you should be able to download the latest released unitypackage, and import it into your project. It will create a new folder in the top level of your Assets hierarchy called "Interaction URP".

New Projects

1. In Unity Hub, create a new URP project
2. Install Oculus SDK v37 - you can get this on the Asset Store or directly from Meta.
3. Configure your project for VR.

Once you have a project that can build & deploy to your device, install the latest Interaction URP unitypackage.

Bonus Usage Option

Technically, this repo is a preconfigured URP project with everything set up and ready to go, so you could just clone it and open it in Unity if you want to just try it out.

Final Project Configuration

If you haven't already, update your OculusProjectConfig to enable Hand Tracking with high frequency.

Also, there are two shaders that require the addition of custom layers and custom render features, for adding the necessary stencil support for the window+skybox effect in the sample scenes.

If you haven't already modified your renderer & layers, running the included command Assets > Interaction URP > Configure Project should do everything for you. If you prefer to make the changes by hand, the required changes are as follows:

Layers

Layer 6: Stencil Mask Layer 7: Stencil Vis

Custom Render Features

Main Renderer Changes:

• Opaque Layer Mask: Remove the stencil layers (6 & 7)

Feature #1

• Name: Stencil Mask
• Event: BeforeRenderingOpaques
• Queue: Opaque
• Layer Mask: Stencil Mask (layer 6)
• Override Stencil: true
  • Value: 5
  • Compare Function: Always
  • Pass: Replace
  • Fail: Keep
  • Z Fail: Keep

Feature #2

• Name: Stencil Vis
• Event: BeforeRenderingOpaques
• Queue: Opaque
• Layer Mask: Stencil Vis (layer 7)
• Override Stencil: true
  • Value: 5
  • Compare Function: Equal
  • Pass: Keep
  • Fail: Keep
  • Z Fail: Keep

FAQ's

• Is this done?
  • Nope, currently all of the sample scenes should run as expected, but there are more shaders to rewrite.
• Why did you duplicate the scenes like this?
  • I wanted this to have minimal impact on Oculus SDK installations, so if you decide to remove it, it won't have a negative impact on your project at large
• Can you make variants of more of the prefabs in the SDK?
  • Happy to -- please file an issue and I'll try to prioritize the most common requests
• You used ShaderGraph for some shaders and raw HLSL for others -- why?
  • A mix of "I'm learning as I go" and "Some shaders would be hard to impossible in ShaderGraph". I might move more of the ShaderGraph stuff to HLSL now that I feel more comfortable with it

Questions? Issues?

Please feel free to file an issue for any changes you'd like to see - additional SDK shaders, rendering errors, &c. I'll do my best to get them updated ASAP (also, PR's welcome!).

If you just want to say hi, you can find me on twitter @rje.

interaction_urp's Issues

Right Eye Not Clearing View.

Hello!

First of all, let me thank you for this amazing work and for your efforts to translate the Oculus SDK shaders to URP. I always worked with the URP when developing for the Oculus Quest and one thing that annoyed me the most is that they don't have a way to fix their own shaders and port them out of the SRP.

I tried myself, but I am not very good at shaders and I got stuck on the Oculus Hands of the brand new Interaction SDK, I actually didn't know how to translate the Surface Shader with multiple passes to a URP shader.

Introduction

After trying your latest release package, I ran into a couple of problems and bugs that I thought you should be aware of.
The issues involve broken references, a couple of shader errors and the main problem which is described on this issue title.

I managed to fix the broken references and the shader errors (I will explain here how in case anyone may need it) but not the hand flickering and the right eye issue. I am going to explain the issues in the in the order in which I found them.

To reproduce

• Happens on every scene of the .unitypackage.
• Unity Version: 2020.3.29f1
• Oculus Integration v38.0

Broken References

I don't know if this is may be only in my particular case, which I don't think so but I didn't check before so I can't be assured.

Some of the references of the Grabbables of the scene BasicGrab URP are not set up correctly. This produces errors after entering Play Mode. In my case, each interactable object over the desk needed an extra reference to be set on the Inspector.

On each object SimpleGrabXXX of the scene, under the root Interactables every child named HandGrabInteractable needs a reference to it's parent (the actual SimpleGrabXXX) on the field Grabbable of the script Hand Grab Interactable. Let me show this:

Also, in my case and on the same objects I lost the reference to the Pointer Interactable exposed field of the script PointerGrabbableConnection. The solution is either also assign the parent (the SimpleGrabXXX) or to remove the component.

Other broken reference I noticed where on the camera rig. Under each XXXHandSynthetic prefab, on the Hand Modifier object, the DataModifier field should be assigned to either the LeftHand or the RightHand objects, located under the Hands object of the InputOVR.

Shader Errors

I got two weird shader compilation errors on two of the translated shaders:

• Oculus Hand Outline, lines 115 and 117.
• Oculus Hand Color, lines 105 and 107.

The issue is quite simple to fix, on each line, there are these two macros written:

UNITY_SETUP_INSTANCE_ID(v); // Lines 105 (Oculus Hand Color) and 115 (Oculus Hand Outline).
UNITY_TRANSFER_INSTANCE_ID(v, OUT); // Lines 107 (Oculus Hand Color) and 117 (Oculus Hand Outline).

This throws me an error saying that v is not defined. Actually, the parameter of the vertex shader is named IN so changing this to:

UNITY_SETUP_INSTANCE_ID(IN); // Lines 105 (Oculus Hand Color) and 115 (Oculus Hand Outline).
UNITY_TRANSFER_INSTANCE_ID(IN, OUT); // Lines 107 (Oculus Hand Color) and 117 (Oculus Hand Outline).

fixed the issue for me.

Right Hand Flickering and Right Eye Issue

After fixing the problems above I managed to enter the Play mode without any errors and with the interactions and shaders working and showing correctly (no pink materials). This issue is quite difficult to explain by words, so I prepared a little video to show it:

Oculus.URP.mp4

The video was recorded using the Oculus Mirror tool and the Oculus Link within the Unity Editor. The first part shows the view of the left eye, which is pretty much perfect. This part is intended to show the problem with the right hand shader flickering constantly. I actually fixed this while writing this issue, I had an object enabled which didn't have to be.

The second part is after I update the Oculus Mirror setting to display the view of the right eye, which looks so chaotic. I am not a professional of this kind of "graphics" things but seems to me that the camera isn't clearing the previous draw (I don't recall the name for that). Both cameras (on left and right eye) have the exact same configuration.

At first I thought it could be some issue with the renderer features added through the Configure Project tool but after disabling them the problem persists and I don't know what can be causing it.

Just to clarify, on other scenes with the default Oculus Hands and a simple material this problem does not happen.

Additional context

A couple days ago, the Oculus Integration Package got the v38.0 update, and I think that, at least for the broken references issue, is the reason for some of the errors that I found. Actually, the issue with the Interactables, is for a new exposed field which I think that it wasn't exposed on the v37.0 and which has no default value, also, an assertion throws if it's null.

Forgive me if my english is bad at any point, it's not my primary language.