Reverse-Engineering Video, Shot by Shot

You've watched a video and thought, "how did they make that?" Maybe it's a slick AI-generated product ad, or a 3D explainer that looks too clean to be real footage, or a TikTok clip that somehow feels effortless. Figuring out what's actually happening in a video—frame by frame, cut by cut—is usually a manual, tedious process. rea-video is an agent skill that automates it: point it at a video, and it tells you how each shot was made.

What It Does

rea-video is a tool for AI coding assistants like Claude Code and Codex. You tell the agent to install it and then hand it a video file, and the agent runs through the whole breakdown process itself—installing the command-line tool, reading the analysis workflow, and producing a report.

The output is a shot-by-shot breakdown. It identifies what kind of video it's looking at (live-action, AI-generated, digital human, 3D/animation, code-driven motion graphics, screen recording, or some mix), then explains how each piece was constructed. For live-action, that means shooting setups, camera positions, shot sizes, lighting (with twelve equipment-backed schemes), and color grading. For AI video, it gives you the reasoning behind the call and reverse-engineered prompts per shot, plus how many generation passes you'd need and which model to use. For code or template-based work, it describes the text, layout, and motion timing, and notes that you could rebuild it as a Remotion project.

Beyond the visuals, each video also gets broken down for subtitles and on-screen text (with start and end times exportable as SRT or ASS), stickers and picture-in-picture, editing rhythm, and audio—whether the voice is real or AI, whether the music is AI-made or library-sourced, the BPM, and where sound effects land. The tool runs on Python 3.10 or higher and needs ffmpeg installed.

Why It's Cool

• It separates judgment from generation. The core value here isn't making videos—it's telling you how one was made, in enough detail that you could reproduce it yourself. That's a different and arguably harder problem, and the reports show the reasoning rather than just a label.

• It handles mixed videos honestly. Real videos often blend AI characters, code interfaces, and genuine screen recordings. rea-video judges each shot separately and then reassembles them along the original timeline, which is closer to how these things actually get made.

• The cases show the tool catching its own mistakes. One case is telling: after transcoding, the tool classified 35% of a live-action unboxing video as AI, then the agent re-checked the frames and landed on 97% live-action. Another case had a vision model calling a shot "live-action drama" until the agent spotted fake characters on a document and recognized it as AI. That kind of self-correction is what makes the results worth trusting.

• You don't strictly need a vision model. It works better with one (Gemini or Doubao, for instance), and with one configured it nails down live-action versus AI versus 3D more accurately and reverse-engineers prompts per shot. But if your agent can see images on its own, it'll fill in the shots it's unsure about.

• The report is thorough. You get a one-line verdict (something like "41% AI video + 59% code interface, 8 seconds of real screen recording, library music at 122 BPM"), a reproduction plan split by live-action, AI generation, and code, and a full report.html with per-shot screenshots, reasoning, prompts, shooting notes, and audio breakdown.

How to Try It

You'll need ffmpeg and Python 3.10+ on your machine, plus an agent like Claude Code or Codex that can run commands.

1. In your agent, tell it:

安装 https://github.com/JuneYaooo/rea-video ，然后拆解这条视频：/path/to/video.mp4

1. The agent installs the CLI, reads the breakdown workflow, and runs the whole thing end to end.
2. Optionally, give it a vision model's API endpoint and key to improve accuracy and get per-shot prompts.

When it's done, you'll have the one-line conclusion, the reproduction plan, and the full report. The repo is at github.com/JuneYaooo/rea-video, and the README links to several case studies—a 19-second mixed clip, a 67-second 3D explainer, a 70-second product promo, an AI music video, and more—each with the source video and the generated report so you can see the output before committing.

Final Thoughts

rea-video is a niche tool for a specific workflow, and that's fine—if you're trying to understand how a video was put together (or you're reverse-engineering a competitor's ad to make your own), this saves you a lot of frame-scrubbing. It's built for people already working inside an agent environment, so it won't be for everyone. But the case studies are honest about edge cases and misclassifications, which is a good sign. Worth a look if video breakdown is something you do even occasionally.

Follow @githubprojects for more developer tools and open source projects.