Blendshape and kinematics calculator for Mediapipe/Tensorflow.js Face, Eyes, Pose, and Finger tracking models.
-
Updated
Feb 20, 2022 - TypeScript
{{ message }}
Blendshape and kinematics calculator for Mediapipe/Tensorflow.js Face, Eyes, Pose, and Finger tracking models.
Robust realtime face and facial landmark tracking on CPU with Unity integration
Use Unity 3D character and Python deep learning algorithms to stream as a VTuber!
虚拟爱抖露(アイドル)共享计划, 是基于单目RGB摄像头的人眼与人脸特征点检测算法, 在实时3D面部捕捉以及模型驱动领域的应用.
Get live translations for YouTube streams, crowdsourced from multilingual viewers!
A free, cross platform SDK for real time facial motion capture using blendshapes, and rigid head pose in 3D space from any RGB camera, photo, or video.
We'll have to support multiple languages. A good start is looking at Crowdin as a source. We'll support languages by demand but for starters I think we'll support English, Japanese, and Chinese (Simplified and Traditional) given we have people proficient in those languages.
As for implementation, That would be the second part of investigation.
A 3D virtual head control system for VTuber in Unity with smooth motion and robust facial expressions
A real-time motion capture system for 3D virtual character animating.
Inochi2D reference implementation aimed at rendering 2D puppets that can be animated in real-time (using eg. facial capture).
Monitor and record live streams from YouTube, OPENREC, TwitCasting, etc. Made for VTuber fans. (VTuber 直播自动录像脚本)
VTuber application made with Godot 3.4
Monitor and download live streams from YouTube
Thank you for your interest about this project. I did not expect that much attention on YouTube in last year when I uploaded.
###Background
I changed the game running aspect ratio from a landscape 4:3 to a portrait 9:16 in the view of showing the full body of the model. This is common in vtuber capturing software running on mobile, and for streamer to slowly revealing their new outfits from
Realtime hand tracking and finger tracking in Unity using Mediapipe
A SteamVR powered mocap solution for Unreal Engine
FaceLandmark Detector using Dlib (Unity Asset Plugin)
CVVTuberExample(Computer Vision Virtual YouTuber Example) is an example project of controlling 3D humanoid model ("Unity-chan!" Model) using WebCamTexture.
This is a template for most Cubism Live2D models. If you just want to customize your own Live2D models, probably you can read the following instructions.
Add a description, image, and links to the vtuber topic page so that developers can more easily learn about it.
To associate your repository with the vtuber topic, visit your repo's landing page and select "manage topics."
On Multiview, maybe it's possible to merge together subtitles from all the different streams into a single TL dex box?
... A lot more convenient than having four chat thingies open only for the mod messages
Lack of backend dependencies means this is very easy to try and implement for new devs!