opensourceprojects.dev

A broadsheet for software that doesn't ask for your email

Video to editable human animation, with AI processing running locally
GitHub RepoViews2

Project Description

View on GitHub

Turn Video Into Editable Animation With Local AI Motion Capture

You've probably seen those motion capture setups where an actor in a suit covered in reflective markers drives a digital character. It looks great, and it's completely out of reach for most people working on their own. But what if you could skip the suit, the studio, and the cloud subscription, and just point a camera at someone? RGB MoCap is a native Windows app that turns plain video into editable human animation, and it runs the AI processing locally on your machine.

What It Does

RGB MoCap is a Windows application that extracts human motion from ordinary video and gives you a workspace to review and export that animation. The AI processing happens on your own hardware—no uploading footage to someone else's server. It supports several motion estimation runtimes (ExMR, FlowHMR, and GVHMR), along with multi-camera capture and character retargeting.

The app isn't a single black box. It's a workspace where you track video, preview 3D motion, and then send the results where you need them. The README's sample videos show motion capture, character retargeting, multi-cam capture, streaming into Blender, and live streaming to an Unreal Engine MetaHuman. That last one is worth noting—this isn't just about baking out a file, it's built to drive characters in real time. There's also PEAR live-preview support and optional tracking via SAM 3.1 for text-prompt and click-guided "Find person" selection.

Why It's Cool

It keeps the AI on your machine. The headline feature is local processing, and the README is upfront about the tradeoff: you prepare the model files separately through a Model Manager, and some require account registration and manual downloads. That's friction, but it's also the reason your footage never leaves your computer.

The install process is honest about complexity. This is a stack of runtimes and dependencies, and the project doesn't pretend otherwise. install.cmd sets up the base runtimes, update.cmd handles updates, and optional models like MAMMA, PromptHMR, and GEM-X each get their own separate setup commands. The README even tells you that if PEAR or camera-alignment support fails after the base install, the base app still works and you can rerun the installer to retry. That kind of detail suggests someone who's actually lived through the failure modes.

It plays well with the tools you already use. Blender and Unreal Engine MetaHuman streaming are first-class use cases, not afterthoughts. If you're already working in those ecosystems, the output lands where you need it.

Optional really means optional. The separate runtimes don't depend on each other, and the main install and update scripts won't pull them in. You only pay the setup cost (including Visual Studio C++ Build Tools and CUDA Toolkit 12.8 for the native-build models) if you actually need those models. GEM-X, notably, skips the compiler requirement entirely.

Multi-camera support. The sample list includes multi-cam motion capture, which is a meaningfully different capability from single-view tracking and opens up more reliable capture setups.

How to Try It

You'll need Windows and Git for Windows. Open a Command Prompt in a writable folder and clone the repo:

git clone https://github.com/toyxyz/toyxyz_RGBMocap.git
cd toyxyz_RGBMocap

Then prepare the runtimes:

install.cmd

Start the app:

RGBMoCap.cmd

Keep that console window open while the app runs. After that, open Model Manager to prepare the AI weights and body files—remember that some of these require registration and manual downloads, so budget some time for that step. To update later, close the app and run update.cmd from the same folder.

If you want the optional models, run their setup commands (Setup_MAMMA.cmd, Setup_PromptHMR.cmd, Setup_GEMX.cmd) only after the base installation succeeds. For the text-prompt and click-guided tracking, run Setup_SAM31.cmd and pull the 1.75 GB checkpoint through Auto Download in Model Manager—and note that SAM 3.1 uses more VRAM than the default tracking.

The full details, including disk space requirements and recovery steps, live in the installation guide. Start at the repository.

Final Thoughts

RGB MoCap isn't a one-click tool, and it doesn't pretend to be. Between the runtime setup, the separate model downloads, and the occasional compiler requirement, it asks for some patience up front. But for developers and animators who want motion capture without a cloud dependency or a marker suit, the tradeoff looks reasonable—especially if you're already comfortable in Blender or Unreal. If you've got a Windows machine with a decent GPU and some video of a person moving, it's worth the setup.


Follow @githubprojects for more developer tools and open source projects.

Back to Projects
Project ID: toyxyz-rgbmocapLast updated: November 19, 2026 at 07:15 PM