Commavq: Compressing 5,000 Minutes of Driving Video Into $500 of Tokens

Most video compression tools are designed for entertainment: think Netflix, YouTube, or surveillance footage. But what about video for machine learning? Specifically, what about compressing thousands of hours of driving data into a format that’s cheap to store, fast to process, and still useful for training self-driving models?

That’s exactly what comma.ai’s new commavq repository does. It takes massive amounts of driving video (like 5,000 minutes of it) and losslessly compresses it into tokens. The result? You can store that entire dataset for about $500 in cloud storage. No, that’s not a typo.

What It Does

commavq is a video compression tool built specifically for driving data. It takes raw dashcam footage (or any driving video) and converts it into a compact tokenized format. The key word here is lossless — meaning after decompression, the video is pixel-for-pixel identical to the original.

Unlike video codecs like H.264 or H.265, which trade quality for size, commavq uses a data-aware approach. It analyzes the driving video frames and encodes them into discrete tokens that can be stored efficiently. The result is a file that’s dramatically smaller than the original, while preserving every single detail.

Why It’s Cool

A few things make this project stand out:

• Insane compression ratio. 5,000 minutes of driving video for $500. Compare that to raw storage, which for that length of high-res footage would cost thousands — or even tens of thousands — in cloud fees.

• Built for developers, not just researchers. The repo is straightforward. You give it a directory of video files or a comma 3 device’s data, and it spits out compressed tokens. No exotic hardware required.

• Lossless is rare in this space. Most “compression” for ML video is actually lossy. commavq retains every pixel, which matters when you’re training models to detect pedestrians, lane lines, or stop signs. A single blurred frame could mean a missed edge case.

• Ideal for open datasets. If you’re working with public driving datasets (like comma’s own openpilot data), you can compress it, share it, or archive it without worrying about storage costs ballooning.

How to Try It

You can dive in right now. The GitHub repo has clear instructions.

git clone https://github.com/commaai/commavq
cd commavq
pip install -r requirements.txt

Then point it at your video directory:

python compress.py --input /path/to/videos --output /path/to/compressed

The output will be a .commavq file (or a directory of them). To decompress later:

python decompress.py --input /path/to/compressed --output /path/to/reconstructed

If you don’t have driving video handy, the repo includes a small sample dataset to test with. You can also use it on any driving footage from comma devices or your own dashcam.

The repo includes a --help flag, but honestly, it’s clean enough that you’ll be compressing video in under 10 minutes.

Final Thoughts

commavq is a practical tool for a very specific pain point: storing massive amounts of driving data without going broke. If you’re training self-driving models, collecting data on a fleet, or just archiving dashcam footage for future ML experiments, this is worth a few minutes to set up.

It’s not flashy. There’s no fancy UI or viral demo. But for the developers working in autonomous driving, robotics, or computer vision — this kind of compression is the difference between a $5,000 monthly AWS bill and a $500 one.

Go compress some video.

Brought to you by @githubprojects