adarosecannon / handy-work Goto Github PK

Currently there is a hack for supporting the correct grip space in Oculus devices

Once that bug is fixed then all the weird grip space hacks can be removed and everything made a lot neater.

Hi @AdaRoseCannon,

Is it possible to integrate only the pose recognition feature of the handy-work module with the hand-tracking-controls component to emit the pose of the hands?

Ensure that transient inputs are able to be tracked with data elements, perhaps data-transient="0", data-transient="1", data-transient="2" etc to track multiple inputs.

Hi @AdaRoseCannon,

I was wondering if the magnetic helper module from the handy-work component can be used as a standalone module to grab objects in the scene by pinching.

This is to make it easier to identify when an object goes to grab what it is grabbing.

• Dumping hand pose does not work because it only updates one hand at a time
• Can't attach objects to hands to pick them up

Suggestion: Don't provide your own hand component make a system/scene component for detecting poses that just fires events.

See explanation in aframevr/aframe#5298
If someone is up to doing a PR with the needed changes, please go ahead.

Weight each position according to weight/totalWeight

This will enable letting the thumb move freely without breaking the pose.

Should probably also make an editor.

If a data-[left|right|none] element has a raycaster then perhaps events should be fired to it's interesected els too.

The AFrame handy-controls only look at the original targetRayModes screen and tracked-pointer, it would be handy to add support for transient-pointer which don't need a model to be drawn and only have a targetRaySpace and a gripSpace.

Some hardware has a single controller I think this should work but won't be cleaned up correctly.

Work out the size of the hand scale accordingly to accommodate small hands better.

Firstly, check whether finger length can be measured consistently between poses.

In https://www.cam.ac.uk/research/news/machine-learning-gives-users-superhuman-ability-to-open-and-control-tools-in-virtual-reality and the HotGestures: Complementing Command Selection and Use with Delimiter-Free Gesture-Based Shortcuts in Virtual Reality from IEEE Transactions on Visualization and Computer Graphics (2023) by Zhaomou Song, John J Dudley and Per Ola Kristensson they describe series of "shortcuts" as hand poses. Could be interesting to try to reproduce it.

In the A-Frame handy-controls component, the left hand's poses don't correspond to those detected by the right hand.

Added below:

• Demo video, showing all the detected poses and events.
• Poses comparison, highlighting the inconsistent poses from the left hand.
• Link to Glitch project, the basic scene I used in the demo video to log in VR mode all the poses and events.

@AdaRoseCannon, I haven't found out what is causing this problem, but I'll be more than happy to help QA test any changes you might make if/when you work on this issue. So just ping me 🙂

PS: apparently, both hands cannot detect other standard WebXR API events such as squeeze, squeezestart, and squeezeend.
(IDK if you are going to fix this part in another dedicated issue, or still here).

Demo Video

Poses Comparison

Glitch Link

https://glitch.com/edit/#!/handy-controls-poses-events?path=index.html%3A1%3A0

How can i build this project to use standalone/locally/offline? Which assets are required and where can i get them?

Hello!

Newb question. Calling loadPose results in me getting a TypeError: Failed to construct 'URL': Invalid URL error. I've tried passing in absolute paths, relative paths and so forth - none of them seem to be working. I use webpack and have confirmed that the pose files have been copied over to the bundle.

What am I doing wrong? Thanks!

Don't render controllers or hands in AR since the user can see the physical hardware/wetware anyway