tomorrowtodaylabs / newtonvr

A virtual reality interaction system for unity based on physics.

Summary

Now available on the Unity Asset store: https://www.assetstore.unity3d.com/en/#!/content/75712

Go to newtonvr.readme.io for documentation, guides on how to get started with NewtonVR, and basic integration instructions.

Our feature roadmap is public and can be found on Trello here: https://trello.com/b/iLvugNpK/newtonvr

Our system allows players to pick up, drop, throw, and use held objects. Items don't pass through other items (rigidbodies), or the environment (non-rigidbodies). Rather, held items interact with other rigidbodies naturally -- taking mass into account. For example, if you have two boxes of the same mass they can push each other equally, but a balloon, with considerably less mass, can't push a box.

For more information on this style of mass-based interaction, see this article by Nick Abel, one of the creators of NewtonVR.

Items can be configured to be picked up at any point, or when grabbed, can rotate and position themselves to match a predefined orientation. This lets you pick up a box from its corner as well as pick up a gun and have it orient to the grip.

We've also created a few physical UI elements to help with basic configuration and "menu type scenarios. We also give you the option to dynamically let the controllers turn into physical objects on a button press.

NewtonVR brought to you by [Tomorrow Today Labs](http://www.tomorrowtodaylabs.com):

Development: Keith Bradner, Nick Abel, Amanda End
Interaction Design: Adrienne Hunter
Modeling: Wesley Eldridge, Carli Wood
Audio: Joshua Du Chene

Windows Mixed Reality support by: [@Jesseric](https://twitter.com/jerressic) and ([@meulta](https://twitter.com/meulta))

newtonvr's Issues

Teleportation whilst holding an object

It seems that teleporting whilst holding an object is broken. If you pick up an object and then teleport the object will disappear from your hand, and be left where you teleported from.

[Suggestion] sliding door physics

Is it possible to implement a sliding door/window that can be interacted with? At the moment I'm imagining a pocket door, but this could also apply to a window.

edit: just saw the drawer in the demo scene. perfect example. thanks

Will you put this is the asset store ?

I didn't see it.. If not, Why?

Custom pickup

This might be a edge case, but if you have a small interactible on a large interactible the pickupclosest code does not work optimal. I created a virtual method on the intereractible

        public virtual NVRInteractable GetCustomPickup(NVRHand hoveringHand) {
            return this;
        }

Pickup code in PickupClosest

            if (closest != null)
            {               
                BeginInteraction(closest.GetCustomPickup(this));
            }

Usage (In this case a handgrenade with a safety pin)

        public override NVRInteractable GetCustomPickup (NVRHand hoveringHand)
        {   
            if (Pin == null) return this;
            if (!IsAttached) return this;
            if(!hoveringHand.IsHoveringOver(Pin)) return this;

            return Pin;
        }

Consider using SendMessage for NVRInteractableItem

Just thought I'd suggest that you may want to use SendMessage for stuff like UseButtonUp and UseButtonDown. It will fit the Unity-style approach a bit better, rather than requiring people to extend the NVRInteractableItem directly.

NVRHand DeregisterInteractable can hit a null error

Thanks for making this asset, here is my contribution! In our game we make an object have different intractable parts depending on game conditions. Either the lid of a box is active, or the whole object is active. To do this we register and deregister the appropriate colliders and NVRInteractables at different times. We hit an error with that that required a simple fix to NVRHand. It is just a null check on CurrentlyHoveringOver, in line 463 of NVRHand. Please include!

        public void DeregisterInteractable(NVRInteractable interactable)
        {
            if (CurrentlyInteracting == interactable)
                CurrentlyInteracting = null;
          if(CurrentlyHoveringOver !=null)
            CurrentlyHoveringOver.Remove(interactable);
        }

Adding custom models removes albedo texture and changes color to white

I've added some custom hand models to my controllers--in addition to the default vive controllers not disappearing, it also changes the albedo textures to white on the hands. Clicking the grip buttons alters their translucency, but they stay white.

How to change the grip functionality to trigger functionality?

Hi there, I've been experimenting with newton VR for the last couple weeks it's pretty sweet thanks for your work on this!!

I've been trying to figure out how to map the grip functions instead to the trigger though. It seems like it should be straight forward but so far I can't seem to figure it out.

Thanks for any assistance!

Push button

I rotated the push button to be put on a wall instead of a table. It didn't really workout since I think the speed towards the initial position is depending on world postion y or something like that?

PhysicalController does not reappear as either Physical/NonPhysical when opening and closing SteamVR Dashboard

If you activate PhysicalController on one controller, and open up the SteamVR Dashboard with the other controller, upon returning to the game, the Controller is not visible, and must be deactivated and then reactivated to force the re-appearance of the controller.

I suspect it is because PhysicalControllers are not SteamVR Tracked Devices.

Attached Objects Jitter

Hello there,

at first I would really like to thank you for your amazing work on NewtonVR. It really helped me a lot, and I had a blast with the Physics applied on my Objects in VR.

However, I am having a problem with serious Jitter when i attach an object to the controller via the grip button. Slight movements with the controller will translate to massive jumpy movements of the objects and eventually will break the attachment and the object will be launched into the open. Using the controller as an rigidbody or other physical interactions with other objects are just working as intended. Only the controller attachment is wonky.

I only have this problem with some models from me and also only with specific meshes. For Example: I have a fridge-like Object and some tubes i made myself. When i attach to the tubes I dont experience the jitter, the fridge itself however will go mad when i attach it.

I thought it would be linked to amounts of vertices or different materials on an object, but the tubes with a lot more vertices are not causing issues. Also removing the ~8 materials from the fridge did not resolve the issue.
Changing the physical values of the rigidbodies also did not help, it even seems that they do not have any effect when attached to the controller (probably intended). The object just jitters massively.

Am I overlooking something? I just have the feeling that there is something wrong with my polygons, but I simply can not imagine what it is.

Many thanks , Best Regards and I hope I got my point across!
Smack

NewtonVR does not handle Despawned/Destroyed NVRitems cleanly

In my game items you are holding, or could interact with, may explode (i.e. despawn). If I don't handle this specially, the item can stick around in 'CurrentlyHoveringOver' and you still grab onto it if you push the grab button after it is despawned. This totally screws with the controller behavior and frustrates users, as you are grabbing nothing and the controller just seems broken.

One fix i've done is to call 'ForceDetach' if something you are holding despawns. That seems to fix one case, but not the 'hovering over something' case. I'm working on finding another way to remove that item cleanly from both hands, hopefully without checking both hands 'CurrentlyHoveringOver' for every despawned item.

I can't make it work with Unity 5.4.0b25

I am using Unity 5.4.0b25 and I can't pick any object. I can see the controllers, but I can't interact with anything from the scene.

Does anyone know what should I do?

NVRInteractable

I implemented a class that enabled/disabled all sub-rigidbodies on attach/detach. I think that function should be of better use in NVRInteractable baseclass.

It's really easy and you will be able to move the control panel for example.

It will also solve the issue of the cube with the button. :)

My class:

   using System.Linq;
   using System.Collections.Generic;

public class NVRInteractableItemPanel : NVRInteractableItem
{
    private Rigidbody[] bodies;
    protected override void Awake()
    {
        base.Awake();

    }

    protected override void Start()
    {
        base.Start();
        bodies = GetComponentsInChildren<Rigidbody>().Where(x => !x.isKinematic).ToArray();
    }

    public override void BeginInteraction(NVRHand hand)
    {
        foreach (var item in bodies)
        {
            item.isKinematic = true;
        }

        base.BeginInteraction(hand);

    }

    public override void EndInteraction()
    {
        foreach (var item in bodies)
        {
            item.isKinematic = false;
        }
        base.EndInteraction();
    }
}

Change controller to custom model

How can i change vive controller model with my custom hand models ?

Adding custom hand models does not stop controllers from showing up

Seems like when you add hands, it should be easy to choose whether to show the controllers or replace that model entirely. Or automatically replace the old controller visual, although that is nice to use when
lining up the new model

When my new hands spawn, they are calling "SteamVR_Utils.Event.Send("hide_render_models", true);" after a couple frames, but it doesn't consistently turn them off. it mostly turns them off, but if you hit a button sometimes one of the physical controller models will still be there, or will pop back into existence.

Teleporting off objects

Since Teleporting is possible as long as the laser hits an object, using two controllers and attempting to teleport to my controller with the grip buttons pressed, will teleport me higher or lower depending where I hold the other controller when teleporting

Laser of ghost controller does not match laser of visible controller.

Indication of misalignment of translation/rotation of laser end points.

Motion

Great updates lately!

However, try to put the interactable objects (switch, button etc) on a moving gameobject. There may be some undesirable results to all rigidbodies. Some works better if you create fixed joints, some not.

NVRInteractibleItem follow latency

Hi guys, excellent work on the library, We are using it for our Multiplayer shooter.
I have tried to modify the FixedUpdate to get it to follow more quickly but no luck, have you have any progress in this matter?

SteamVR laser pointer

I'm experiencing an issue when I put a SteamVR_tracked controller and steamVR_laser pointer on an NVR controller, the laser pointer has two lasers which point in apparently random directions. I'm wondering if you have any idea what might be causing that behavior?

Thanks!

NVRInteractables.Deregister() implementation and/or usage example needed.

Destroying NVRInteractable-derived objects (or changing their colliders) can result in a bugged ghost entry in NVRHand.CurrentlyHoveringOver, which produces a number of buggy behaviors such as being able to grab the object from anywhere, or making the controllers vibrate perpetually (and probably more). It seems like NVRInteractables.Deregister (and the associated downstream functions in NVRPlayer & NVRHand) are made for this purpose, but it's unclear how they're intended to be used.

Using multiple scripts together?

Hi there,

I'm trying to create a handle that can pull out, and have rotational control. I can use the scripts to make either one work alone, but I can' figure out how I can get both functions out of a handle.

Any ideas? Thank you!

Event driven design

Event driven design should be implemented to all controls.

For example in a slider:

    public delegate void ResultAction(string tag, float value);
    public event ResultAction OnResultChanged;
    float previousValue;

 if (previousValue != CurrentValue)
        {
            previousValue = CurrentValue;
            if (OnResultChanged != null)
            {
                OnResultChanged(tag, CurrentValue);
            }
          }

Recommend TouchPad press as hold button.

Hold, throw, and release feels not unlike casting with a fishing rod. Just a suggestion.

NVRInteractables sometimes lose all momentum when thrown

About 50% of the time my interactables don't throw correctly. They either stop forward progress and drop by gravity, or they sometimes wildly bounce backwards. They may even get a momentum boost forwards sometimes, but i can't tell if that is just my throw.

It seems to be enhanced by swinging my arm fast, sometimes... but it happens at slow and fast speeds. I can't reliably reproduce it and it has been very hard to track down. It can happen when i 'let go' in the middle of an arm swing or at the end. When i flick my wrist, or keep it locked. Has anyone else seen this issue?

Troubleshooting:
I tried (briefly) changing the layer of the thrown item to something that doesn't interact with the layer the controller is on.

I can't figure out the root cause of this one, please post if you have an idea! I can post a video if that helps.

Add option to make controller invisible when IsInteracting

For my purposes, I want the controller to completely disappear (not just Ghost) when holding another object. This creates a better illusion of the controller becoming something else rather than just grabbing it.

Should allow non-ghost renderers to be attached to NVRHand

NVRHand assumes any component of type Renderer attached to the hand gameobject is for use as a ghost renderer. It will then drive the enabling/disabling of those renderers which the ghost rendering logic which may not be desirable.

I'm submitting a pull request that deals with this by introducing a GhostRenderer component for ghost renderers to inherit from, creating a clear distinction for these.

Doesn't compile: NVRCollisionSoundProvider not defined

public class NVRCollisionSoundProviderUnity : NVRCollisionSoundProvider

No where in the project is NVRCollisionSoundProvider defined. You can do a search on GitHub repository and it won't be found. Someone probably forgot to commit some files...

SteamVR plugin versioning

Thank you for this repository; I don't know how far along I'd be in my project without your help.

I had already added the SteamVR Plugin to my project before I imported the NewtonVR assets and it threw an error with functions relating to the SteamVR Plugin. Making a new project and importing all the assets including the NewtonVR version of SteamVR Plugin works as expected.

It seems the SteamVR plugin has updated since the latest NewtonVR release and now Newton VR's version of the SteamVR plugin keeps popping up asking me to download the latest version of SteamVR. I know from before that the latest version of SteamVR throws an error with NewtonVR, so should I hide that window forever?

I would recommend adding a section to the readme which explains this nuance; that a) you must not have added SteamVR Plugin to your project before importing NewtonVR and b) you must not update the SteamVR plugin that comes with NewtonVR.

So I don't really have an issue I'm just trying to get a sense of what the procedure is for using the library as it relates to the SteamVR plugin. Moving forward, how and what should we (users of NewtonVR) update? I'll just keep pulling the latest repo changes from here and that'll take care of the SteamVR stuff entirely so I can hide the window and forget about it?

Thanks for any insights, and thanks again for such a useful toolset!

Hand doesn't reappear if holding something that is force detached

When a held object is disabled and force detached, the hand stays inactive if Make Controller Invisible on Interaction is set. It should instantly reappear I think.

it reappears if you let go of the grip buttons and then functions normally.

Networked shared VR with NewtonVR

Is it possibile to use NewtonVR in networked shared VR, for example using DarkRift:

https://www.assetstore.unity3d.com/en/#!/content/16711
http://darkrift.ninjapokestudios.co.uk/

Is it complex? How to integrate this features?

Leap Motion Orion with NewtonVR

Hi people, I love your work!

I have a small issue I hope you might be able to help me with, even though it doesn't directly relate to HTC Vive :)

Issue Description

I have been trying to implement NewtonVR for Leap Motion, and had the same #2 by robnewman, but solved it using his solution (i.e. initialized). I no longer have any errors, but even though pinch is registered, it doesn't pick up any objects with the NVRInteractableItem-script.

Project Screenshots

You can see screenshots of the project structure and warnings https://www.dropbox.com/sh/79vmwcocb5lfe01/AADxc4IRFSAkBbBhl_aTWWlna?dl=0 *

* Don't know why the link doesn't work, but try copy-pasting it into the browser instead.

Quick Overview

1. The project uses the PinchDetect-package/script.
2. The HandFollow-script is designed to follow the position and orientation of the palm of the hand.
3. The LeapTrigger_XXXX has a collider that should detect object in question

Do you have any idea what I'm doing wrong?

Ghost Controllers seem to cause flickering of "held" objects.

FYI, I disabled all the ghost controller code and just went with the standard models. After doing so, the held objects quit their flickering effect.

Add toggle option for grip buttons

I don't mind using the grip buttons to grab things but it would be nice to have the option of using them to toggle grabbing instead of keeping them held. I.e. clicking once grabs, and clicking again drops.

UI Interaction Strange Behavior

I have implemented NewtonVR's NVR Hand script in order to leverage the UI pointer functionality. In my scene, there is a primary canvas and a secondary canvas that becomes activated in place of the original canvas. So, two canvases, one initially active, and the second becoming active later. When the second canvas becomes active, the first becomes deactivated.

Both are world-space canvases. The first is free standing in the world and pointing at it with either controller works perfectly (thank you).

The second canvas is a child of the left hand controller, so it is always attached to the left hand. The problem is that I can not point at this second, hand-attached canvas with the other free hand. Instead, for some reason, I must 'point' at the UI elements on this second canvas by gazing at them with the VIVE headset. If I gaze at the canvas, the cursor appears and I can then use the trigger to select whichever button/slider I am gazing at.

I can not figure out why this is happening.

Can you please let me know if this is a feature or a bug? I'd like to simply treat this canvas as any other, and use the free controller to interact.

I hope this is clear and I look forward to your reply.

Throwable object does not seem to use mass? and Different control options for objects.

Hey guys great project first off all, i'm working on a game and would love to use NewtonVR

I have two small problems though, when i throw a object it does not seem to use the mass setting, so i can throw a 1000kg object a 100 meters. i expect a 5kg object to fly no more than 30 meters when i release it. Bug?

Also is it possible to use the grip toggle function for some objects and the grip hold down for others?
I don't want to use toggle on doors, but would love to use it on weapons for example.

It would be great to have this so the user can tailor NewtonVr more to their needs.

Letter Spinner Momentum

Hi, I was wondering if there is a way to prevent the letter spinner from doing it's little "wobble" from the momentum it has? So following the user's grab point and instantly stopping where they hold their hand.

Thanks

Steam VR v1.0.8 has model as child of controller.

When I upgraded to Steam VR v1.0.8 I had to change a line in NVRHand.cs:

return this.GetComponent<SteamVR_RenderModel>().renderModelName;

to

return this.GetComponentInChildren<SteamVR_RenderModel>().renderModelName;

because the model gameobject is a child of the controller gameobject in the [CameraRig] prefab that comes with Steam VR v1.0.8

Loosing grabbed objects when shaking

First I'd like to say thanks a lot for releasing this excellent kit! This really makes developing for VR much easier!

That being said, I encountered the following issue:
When grabbing objects and shaking the Motion Controllers really quick the grabbed object drop out of the hand at some point.

Do you have any idea how I can fix this in my project? Thanks!

Link controller trigger to start animation

Hi, I'm new to unity VR and am trying to have an animation start playing when I pick up an object. I'm using the HTC VIVE. So far, using your intractable item and attach point scripts I can pick the object up but i cant figure out how to get the animation to play. Basically, its a greetings card that i want to open when its picked up. So far my script (that doesn't work) looks like this:

using UnityEngine;
using System.Collections;

public class PlayAnimation : MonoBehaviour
{
// Update is called once per frame
void Update()
{
if (Controller_Hand.GetPress(triggerButton))
{
GetComponent().Play("Take 001");
}
}
}

Can you please help me? I'm a bit over my head here. Thank you,

UnityEvents implemented for Interactables: code attached.

I added UnityEvents to NVRInteractable and would like to share the code, but i don't know a good way to post it!

I've attached the files to this issue, This adds a UnityEvent for OnAttach, OnDetach, OnHoverEnter(for closest interactable), and OnHoverExit. The attached code is based off commit from Sep 16, 2016.

UnityEventsForInteractables.zip

NVRHand fails to initialize when unloading and reloading NVRHand component

When unloading one scene and loading a new scene each with their own player rigs, The NVRHand components being unloaded receive a render_model_loaded message after being destroyed. This seems to cause an exception that keeps the new NVRHand component from properly initializing.

Solution is to deregister from the event in the OnDestroy callback of NVRHand. See attached pull request for the fix.

IsAttached or IsChildAttached when child is being interacted with?

It would be nice if "IsAttached" were set to true when interacting with a child object. Better yet, introduce a new "IsChildAttached" property. In my situation, I'm simulating more complex physics interactions by adding children with hinge joints (or other types of joints). But for all intents, the user "feels" like they're interacting with the parent -- by way of the children -- and it would be handy to know when they're grabbing and holding on.

BTW, thank you SOOO much for NewtonVR. Really incredible work.

Tracking becomes laggy when NVRInteractableItem when gameObject.activeSelf is toggled.

We have a paddle object(NVRInteractableItem) that is always attached to your hand (NVRHand). Depending on the state of the hand, we want to show/hide the attached paddle.

The issue is, if we set paddle.IsActive = false in one frame, and then paddle.IsActive=true in a subsequent frame, tracking becomes very slow, lagging far behind the real world position. Primarily on rotation it seems.

Something like this should replicate the issue:

void Start(){
paddle,transform.position = transform.position;
paddle.transform.rotation = transform.rotation;
nvrHand.BeginInteraction(paddle);
paddle.gameObject.SetActive(false);//Omit this line, and tracking is great.
}

void Update(){
paddle.gameObject.SetActive(true);
}

Grabbing the same object with both hands causes NaN velocity and angular velocity

Hey guys, first of all, thank you for making this! The interaction system is so kickass

I ran into an issue when attempting to grab an object with both hands at the same time. Unity gives this error:

rigidbody.velocity assign attempt for 'NVRExampleDrawer' is not valid. Input velocity is { NaN, NaN, NaN }.
UnityEngine.Rigidbody:set_velocity(Vector3)

rigidbody.angularVelocity assign attempt for 'NVRExampleDrawer' is not valid. Input angularVelocity is { NaN, NaN, NaN }.

Interacting with objects with two hands.

Hi, we've been demoing a VR game we made over a course of 4 weeks and we have a steering wheel that is an NVRInteractableRotater (I think) and I got a lot of requests (from our team as well) to have the ability to grab the steering wheel with two hands and drive. Is this possible with NewtonVR as it currently stands? Would this be something the devs could implement?

Leap Motion Orion Support

I tried to use the physics interactions with DK2 and Leap Motion but got errors related to the Valve Hand references. Can you offer any advice on how I might be able to use this in conjunction with Leap Motion hands?

Using NewtonVR with Arc Teleporter

Hey I'm trying to use an asset called VR Arc Teleporter http://u3d.as/sty with NewtonVR and running into issues since the instance of a the tracked controller object is no longer a SteamVR_TrackedController type, but an NVRHand and the latter does not inherent from the former. I've tried to remap the different button triggers but I couldn't reference the exact instance of the NVRHand objects in the scene. Do you guys have any suggestions on how I can make the two assets work together? Obviously there is some clashing with the specific buttons but I can deal with that later. I can also share the VR Arc Teleporter script with you privately to inspect it since it is not free.

Thanks for all your great work
Ryan

Can I import this as a package?

Can I import newtonVR into another project as a package or asset?