catherineytw / 3d-posenet

Control 3D Virtual Character through Tensroflow.js Posenet

3D-Posenet

Controlling a 3D Virtual Character Through WebCam

Personal Project

Check out the Live Demo: https://mishig25.github.io/3d-posenet/

Master Branch

Experimental Branch

Note: Master Branch contains functionalities only for head movements. Experimental Branch contains functionalities for arms movements as well; however, the functionalities for arms movements do not work well enough to be added to Master Branch yet.

Description:

This web app is an experiment to combine Machine Learning and Computer Graphics through TensorFlow.js and BabylonJS. Tensorflow.js is an official Javascript API of Google's popular machine learning framework Tensorflow, and since Tensorflow.js is a Javscript library, it provides a way to run machine learning models in browser environments. On the other hand, BabylonJS is a 3D engine that lets you create and run 3D graphics in web apps.

Contents of the repository:

• app.js - main React app
• posenet.js - class for running Tensorflow.js and Posenet
• graphics.js - class for running BabylonJS and creating the 3D scene
• joints.js, transform.js - miscellaneous classes

Development:

yarn
yarn watch

License

MIT