Reuse video texture for mask mixing about mediapipe

[kuaashish]

Hi @danrossi,

Could you kindly provide the detailed steps you are following from our documentation, or alternatively, share the standalone code with us? This will help us to reproduce the issue and gain a better understanding of it.

Thank you!!

[danrossi]

I'll find a vanilla code. Its actually the webgl post render on the same context. Hence binding the mask texture. Would be good somehow to apply the video texture the same to not have to reupload a video frame. It's already efficient enough but trying to improve it further.

[kuaashish]

Hi @danrossi,

We apologize for any confusion. Could you please confirm whether you are utilizing our new Task API as outlined in the documentation, or if you are still using our legacy Selfi Segmentation from here? Please note that support for legacy solutions has been discontinued, and they are no longer maintained. Our current focus is on enhancing our new Task APIs, so new features will only be added to those.

Thank you!!

[danrossi]

I was tied up with massive VR upgrades to fix issues with my Safari hacks and apple vision I am sorry. I have put up a working test file. requesting media device isnt possible in jsfiddle.

You can see this bit I have to reupload the video frame even from within the same context. Hopefully rebinding the internal video texture on the offscreen canvas context from mediapipe is enough to render video to be mixed ? Im assuming its scaled down so can be rescaled ?

rendering in webgl is performing ok enough Ive done performance tests. Just that one last bit I am hoping for. Like applying the mask texture from mediapipe.

gl.activeTexture(gl.TEXTURE1)
    gl.bindTexture(gl.TEXTURE_2D, videoTexture)
    gl.uniform1i(frameTextureLocation, 1)

    gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, video);

https://electroteque.org/dev/mediapipe/

Example it working although I bundle it in, so haven't made an update for a while. I'll double check.

https://electroteque.org/plugins/videojs/rtcstreaming/demos/virtual-background/