A Mouth tracking system for VR, 100% Opensource!
Currently the architecture is a modified efficientnetv2-b0 training on blenshapes from ARKit! The model outputs 31 blenshapes of which most cross over with SRanipals blendshapes.
- This model is not a 1 to 1 translation to SRanipal blenshapes. While most blendshapes do cross over with SRanipal cheek and tounge movements are limited.
Go to the releases tab, download the latest exe. Clone this repo, and then move the calibration file (calib.json) to the same location as the exe. Next get the .onnx model in the repo and move it to the same location as the exe. Run the exe and enjoy!
It ruins the model's quality
Lol
Thank you to dfgHiatus#7426 for providing the unity demo as well as the intergrations with other apps!
Thank you to Rames The Generic#3540 who made the dataset, tested and evaluated the performence for the models, and for being awesome!
Copyright (c) 2022 Sameer Suri