This web app is an experiment to combine Machine Learning and Computer Graphics through TensorFlow.js and BabylonJS. Tensorflow.js is an official Javascript API of Google's popular machine learning framework Tensorflow, and since Tensorflow.js is a Javscript library, it provides a way to run machine learning models in browser environments. On the other hand, BabylonJS is a 3D engine that lets you create and run 3D graphics in web apps.
Flow of the app:
- Get webcam input.
- Estimate human pose through TensorFlow.js. Click here to read more about it.
- Apply human pose predictions on a rigged 3D character through BabylonJS.
- Render updates.
Source code is available at: https://github.com/mishig25/3d-posenet