Endless Loading State about mediapipe-touchdesigner

[domisjustanumber]

Hi @EZUR11 this is an odd one for sure! It sounds like it's failing to start your webcam for some reason. The component currently only works with a 720p webcam. Can you confirm if your webcam supports that resolution and/or if the component
works with any other webcam sources if you have them?

A quick test is in a new project to drop a video device in top down, select your webcam, and see what resolution options you get.

Let us know if those things don't work, and we can think some more

[torinmb]

Another thing to try. Could you open up the TextPort in TouchDesigner and attach a screenshot of the logs?

[EZUR11]

Ok it is all figured out! Just had to refollow the exact steps given in the install tutorial, and realized I had forgotten a crucial piece: connecting the video device input as SpoutCam. With SpoutCam downloaded and running with the right settings (aka the suiting FrameRate for my TouchDesigner version (mine was 30 fps), set the starting resolution (1280x720), and add the right "Starting Sender" input ("TDSyphonSpoutOut"), then registering these parameters). Also, made sure to have the latest release of both your MediaPipe plugin (20.2) and Touchdesigner (2022.33910). It works! Great work!
Now that I ghot to play around with the hand tracking module, I have a few other questions haha!

First, is SpoutCam the only video input that can be selected to work with mediapipe? In other words, could I use the video feed from a NDI TOP node connected to a phone camera for example across the network?

Secondly, any way to have any additional data about the hands positions relative to itself? for example, the distance between the tip of the ring finger and the thumb, or the distance between the ring finger of each hands - the output CHOP data seems to be limited to one hand and one finger at a time. Would like to create an interactive shape building from hands tracking (e.i. a rectangular shapes adjusting its dimensions in real time to the distance between fingers).

Finally, could be very interesting to combine these trained models tools with depth sensors of modern video input gadgets, such as the most modern SmartPhones.
In short, any way to use your wonderful plugin with other video inputs than SpoutCam?
Many thanks!

[domisjustanumber]

Glad you got it working @EZUR11!

You can absolutely use it with other sources - the only caveat is that source has to appear as a webcam. If you install NDI Tools, there are 4 virtual NDI webcams you can use. You configure which NDI source each one will have in NDI Tools, then select the relevant one in the Media Pipe component.

For your hands questions, yes as well. You get an xyz coordinate for each joint in your hand, so you can use a select chop to get the points you want to measure the distance between, then feed them into an Object CHOP and select distance. It will output the distance (or bearing or other things) between the 2 points.

If you just want the horizontal distance, you can also select just the x coordinates for the 2 points and use a math chop to figure out the distance between them.

And if you want to figure out the distance between hands, there's already a CHOP outlut for that, and you can also open up the hand tracking component and take a look at how that is calculated and modify it for your own use. Just be aware if you modify anything within the tox, you’ll have to save it individually as well as the main toe file.