Crop/modify webrender's input about mediapipe-touchdesigner

[domisjustanumber]

Hi @kenyeresrudolf that sounds super cool!

Right now the best (if not the most elegant) way to do this would be as follows:

Video Device In > Syphon Spout Out > OBS Studio > VirtualCam plugin

The VirtualCam plugin gives you multiple virtual webcams (rather than just the 1 by default), so you can set it up so it sends the same Spout video source out to 2x Virtual Webcams.

You can then run 2x instances of the MediaPipe plugin, with each one using a different virtual webcam as it's media source. You could either crop the video in TD before sending it out via Spout, or in OBS before sending it to a virtual webcam.

It means you have to run OBS in the background the whole time, but until we can figure out a more convenient way to get TOPs into the web browser that MediaPipe runs in, this will work for now

[kenyeresrudolf]

hi @domisjustanumber ,
Thank you for the quick reply.

It seems that the OBS double (virtual) output method could work, it does not require sending the webcam picture from TD to Spout either, as it is possible to cut/mask the camera image internally in OBS.

You are correct, it means that the OBS will eat some computational power. It would be a way more convenient way to modify the TOPs that feed the web browser internally in TD. If you can add this feature to the update, that would be greatly appreciated.

This plugin is already a gamechanger, thank you a lot for your hard work!

[kenyeresrudolf]

hi @domisjustanumber ,

Sorry, I tried to build this up in my project, but the method seems not working, as I could not create two half webscam screen input to feed the mediapipe instances:

• following your guidance: from TD i can send two separate outputs with syphonspoutout, but in OBS I will see only one
• while in OBS if I create two instances for a videocapture device with copypasting, then i cannot apply different crop settings to them
• if I add a new videocapture device, then I could add a new filter with different crop settings. BUT if I am using the the same webcam as a source, then only one of those will run. Maybe it is a bug in OBS itself.
  Please let me know if you succeed, maybe i forgot something or in case if you have any other idea :)

thanks a lot

[domisjustanumber]

Ah you're so close! Instead of using the VirtualCam under Tools, you create a scene for each crop you want, then apply a VirtualCam filter to each scene that sends each scene out to a different VirtualCam

Source Filter section of https://obsproject.com/forum/resources/obs-virtualcam.1744/

[domisjustanumber]

I had another thought about this - I also have a webcam I can only run once, but I'm able to get this solution to work and I think the answer is not using source filters.

If you create 2 scenes in OBS and add the same webcam source to each, you can then crop the video n each scene using Alt + drag. That lets you have the same source in as many places as you want without issue.

Once you've set your scenes up, you then add a virtualcam filter to each scene to send it to a different virtual camera.

[kenyeresrudolf]

hi @domisjustanumber,
Yes, it is working now. My only issue here, is that it looks like mediapipe stretch the analysed image to the 16:9 ratio, despite it is cropped to 8:9. It recognizes the hand gestures, but the overlay is shifted and stretched therefore I assume this workaround result in some kind of performance drop. Did you experience the same?
Again, it is working, so I am really appreciating your help! But in the long run, I believe this could be improved!
Thanks again!

[domisjustanumber]

I made a quick tutorial video on how to set it all up so you get a non-stretched version for each side of the screen - hopefully it's what you're trying to achieve

https://youtu.be/9OFfWGsBIyM

[kenyeresrudolf]

Hey Dom,

Thank you a lot, almost there! I still have some issues on maintaining the performance but on the way!
Thank you a lot for your kind help!