vpuppr 0.10.0-alpha.1

vpuppr updated to Godot 4.1.2. Systems are still being re-implemented and only the bare minimum required for loading a model and tracking are implemented.

If you would like to get involved, there is a Discord server.

New features

• Support for VRM 0 and VRM 1 models via godot-vrm (technically only VRM 1 support is new)
• mediapipe support as a face tracker via GDMP which allows for face tracking using a webcam
• Model manipulation using inverse kinematics instead of direct bone manipulation via the renik library
• New UI
• Various optimizations using Rust in libvpuppr

Re-implemented features

• iFacialMocap tracking
• MeowFace tracking
• VTubeStudio tracking
• Data save/loading

Missing/in-progress features

• OpenSeeFace tracking. This actually somewhat difficult to re-implement, since OpenSeeFace is the only tracker that doesn't provide blend shapes out-of-the-box and the previous implementation was full of workarounds
• Mouse tracking (where the model looks at the user's mouse position even without the window focused)
• Manual blend shape manipulation
• Props
• Environment (lighting/camera/model positioning) settings
• Lip sync
• Limited mod support
• Remote control (aka expose an http endpoint to control vpuppr via your own app)
• GLB model support (binary GLTF)

Planned new features

• Live2D support via gd_cubism, distributed as a mod since the core Cubism license is proprietary. This was attempted before, but made very little progress since custom shaders were required to emulate a stencil buffer as Godot does not expose a stencil buffer. Luckily, gd_cubism appears to provide their own stencil buffer shaders
• Inochi2D support. This is technically a better option in terms of open source tools compared to Live2D, but stencil buffer support is also required here and additional help will be required to make this happen
• Pngtuber support

Q/A

Where is the Linux version of the alpha?

I am lazy and don't feel like exporting a Linux version at this time.

The longer explanation is that it's a huge pain to build mediapipe in general, and I don't feel like trying configure my Linux environment to build mediapipe. I was working on a GitHub action to automatically build mediapipe for Windows/Linux, but that effort has stalled since builds take forever. Help would be appreciated with making that work and upstreaming it.

Where is the Mac version of the alpha?

vpuppr has always worked on Mac along with all of the trackers (with the exception of mediapipe which is untested). However, distributing a binary for Mac is an extremely involved process, and I haven't gotten around to creating a pipeline/script for automating this.

Please reach out on Discord if you would like to help with this.

Can feature A, B, and C be implemented?

Many features can technically be implemented! Please open an issue in the GitHub repo and the feature will be implemented when someone eventually contributes an implementation.

Keep in mind, this is an open source project that is distributed for free, so features are only implemented when someone has time, knowledge, and willpower to implement them.

How can I contribute?

Please see the CONTRIBUTING.md document present in this repo!

Since this seems to be missed fairly often, there is a Discord server where communication can happen in real-time.

Why try upgrading vpuppr to Godot 4 instead of enhancing the Godot 3 version?

One reason is that VRM support via godot-vrm was only being supported for Godot 4 going forward. This is understandable, since the maintainers are actively using the library for their own project.

Another reason is that it's nice to use the latest and greatest toys instead of last generation stuff.

Why rewrite the entire program instead of just porting from Godot 3 to Godot 4?

After the experiment in making vpuppr extremely moddable in 0.9.0, the decision was made to go back to go back to a more hard-coded architecture. As it turns out, implementing all core systems as mods was more trouble than it was worth, especially since the contributors/maintainers working on the project remained at just me.

Mod support is still planned, but it is not a primary concern.

Why not ask the community for help?

Open source projects are like group projects in school. People say that they will help, but it usually comes down to one person doing all the work and everyone else wanting to claim some of the credit.

Saltiness aside, I do acknowledge that it can be tough to work with me since I do review every line of a PR and will often request changes. This is not necessarily nit-picking by me (although sometimes it is), but rather me trying to maintain code quality in a codebase that is already struggling with code quality. If we are making spaghetti, let's at least try to keep the spaghetti consistent :)

Virtual Puppet Project 0.9.0 stable

After a long delay, vpuppr version 0.9.0 is now available!

New features:

• Added support for the following trackers
  • iFacialMocap
  • VTubeStudio
  • MeowFace
  • Mouse tracking
• New and improved GUI
• Better performance on all platforms (hopefully)
• Support for user extensions

UPDATE: 23/01/2023@01:32 ET
The Windows resources were accidentally bundled with the Linux files. This problem should be resolved.

Virtual Puppet Project 0.9.0 alpha 2

More new features and returning features from openseeface-gd!

• iFacialMocap support
• Updated OpenSeeFace to the latest version
• Better splash screen
• Translation support
• Auto-center model when starting a tracker
• Bug fixes
• ...and other stuff!

Ninja edit - set executable bit for linux binaries, fix outdated OpenSeeFace files on linux

SHA256 hashes for the zip files

• Windows - b88931226936c5164754b03c091139e6d3980099f49dd458c15481bfd866b5f5
• Linux - acfb0737929c7e7ec32c3559edd42337ebbe95f7856d5cee1ba25fff79f0eee4
• Linux flatpak - 15ab443fe42d75004d3754a5f89f1243eac5dac7f41519073989df8c67df7c0f

Virtual Puppet Project 0.9.0 alpha

Virtual Puppet Project 0.9.0 alpha

Linux version will be released soon™

ninja upload: OpenSeeFace packed with pyinstaller on Linux included as osf.zip

Update July 31, 2022

Added Linux release. This version is slightly ahead of the Windows version.

ninja edit: updated the camera select logic for Linux

ninja edit: when updating the camera select logic for Linux, I accidentally included the wrong OpenSeeFace binary. This has been rectified.

ninja edit: Added a flatpak-compatible release (i.e. the .pck file is not bundled into the binary)

openseeface-gd 0.8.1

openseeface-gd 0.8.1

• Add built-in websocket server for controlling the application
• Mouth shapes should be automatically applied to VRM models
• Bug fixes!

Linux-specific

• Starting the tracker should be more reliable
• Ability to specify a specific python interpreter
• Video devices should show up properly

HOTFIX February 17, 2022

Recompile linux build on Debian 9 for better compatibility with older distros

Even hotter hotfix February 21, 2022

Recompile Linux build of lip sync for better compatibility on older distros

July 21, 2022

Add RPI release (32-bit)

openseeface-gd 0.8.0

openseeface-gd 0.8.0

• Rudimentary lip sync (non-AI driven but should be fast/good enough)
• Better notifications
• Add A-Pose button
• Better logging in case of crashes

January 7, 2021 updates

• took out debug information
• added linux build

openseeface-gd 0.7.3

openseeface-gd 0.7.3

Many QoL improvements made. These include:

• Better performance due to tracking data receiver optimization
• Ability to start the tracker from within the application
• Less delay between starting the tracker and the model responding to tracking data
• Loading in models/props store the last searched directory (so you can resume where you left off)
• Toggle for raw eye input and fix gaze strength (model eye rotation can match your IRL eyes exactly)
• Support for vrm models where blinking uses more than 1 blendshape

Many thanks to @Zutatensuppe for all the patches and for implementing automatic tests!

openseeface-gd 0.7.2

openseeface-gd 0.7.2

• hotfix for application crashes on Linux when starting the face tracker
• thanks to @Zutatensuppe for adding several enhancements to starting the tracker!
• added app message console for debug purposes

EDIT: Re-export the linux build
EDIT JR: Re-exported the linux build once again, this time from a Fedora computer

openseeface-gd 0.7.1

openseeface-gd 0.7.1

EDIT: Withdrawing the Linux release since some compatibility changes I made actually made the app unusable on Linux. If you need Linux support, please use the 0.7.0 release for now.

• basic expression support using dropdowns
• tracker fps is broken and doesn't fill from config
• add blink sensitivity
• store pose values in config
• lighting doesn't load from config
• make reading config files less prone to hard crashes
• cannot stop and restart the face tracker from within the app
• if a model has its name changed, the corresponding config will break
• the previous linux release had the windows executable bundled with it by accident

openseeface-gd 0.7.0

openseeface-gd 0.7.0

• the UI has been completely rewritten to be more responsive
• various tracking bugs have been squashed
• interpolation added for vrm facial movements
• running OpenSeeFace (the tracker) should work out of the box on Linux/Mac
• added FXAA
• add ability to toggle the transparent background off