Rendering Real Scenes at 1080p and 30 FPS: 3D Gaussian Splatting

You've probably seen those impressive neural rendering demos that reconstruct a 3D scene from a handful of photos. They look great, but they're often painfully slow—training takes hours, and rendering a single frame can crawl along at a few frames per second. So what if you could get that same visual quality in real time? That's exactly what this project from the GRAPHDECO team at Inria and MPI sets out to do.

What It Does

This repository is the official implementation of the paper "3D Gaussian Splatting for Real-Time Radiance Field Rendering." It's a method for novel-view synthesis—taking a set of photos or a video of a scene and reconstructing it so you can render new viewpoints. The twist is that instead of relying on a neural network to represent the scene, it uses 3D Gaussians.

Here's the core idea. Starting from sparse points produced during camera calibration (the kind COLMAP gives you), the scene gets represented as a collection of 3D Gaussians. These preserve the useful properties of continuous volumetric radiance fields during optimization, but they avoid wasting computation on empty space. The optimization interleaves density control with the usual fitting, and it specifically optimizes anisotropic covariance so the Gaussians can accurately represent the scene's geometry. On top of that, there's a fast, visibility-aware rendering algorithm that supports anisotropic splatting—which speeds up training and enables real-time rendering.

The result, per the abstract, is state-of-the-art visual quality with competitive training times and high-quality novel-view synthesis at 1080p resolution running at 30 frames per second or better.

Why It's Cool

• Real-time at 1080p is the headline. The paper's own framing is blunt: for unbounded, complete scenes (not just isolated objects) at 1080p, no prior method could hit real-time display rates. This one claims it can. That's a meaningful line to cross, because it changes what's practical—interactive viewers, live demos, that sort of thing.

• It dodges the usual speed-versus-quality tradeoff. Most faster radiance field methods get their speed by giving up visual quality. The authors explicitly position this work as avoiding that compromise, keeping state-of-the-art quality while staying fast.

• The representation is the interesting part. Using 3D Gaussians instead of a neural network means you're not paying for a network forward pass every frame. The Gaussians live in space, so empty regions don't cost you anything. Optimizing anisotropic covariance lets each Gaussian stretch and orient to fit the actual scene rather than being stuck as a uniform blob.

• The rendering algorithm is visibility-aware. This matters because naive splatting wastes effort on primitives that don't contribute to the final image. The paper calls this out as one of the three key elements, and it's what lets the same representation serve both fast training and real-time rendering.

• The authors shipped the goods, not just the paper. Alongside the code, the repository provides the reference images used for the paper's error metrics and recently created pre-trained models. That makes it much easier to verify results or build on top of the work without reproducing everything from scratch.

How to Try It

The repository is at github.com/graphdeco-inria/gaussian-splatting. The README itself doesn't include install commands in the portion available, so you'll want to check the full repository for setup instructions and dependencies.

What the README does point you to directly:

• Datasets: The T&T+DB COLMAP dataset (650MB) is available for input.
• Pre-trained models: A 14GB archive of pre-trained models, so you can evaluate without training your own.
• Viewers for Windows: A 60MB download, which is the most direct way to see the real-time rendering in action.
• Evaluation images: A 7GB set of reference images used for the paper's error metrics.

If you want to go deeper, the full paper and a video are linked from the README, along with the project webpage. The BibTeX entry is included if you're citing this in your own work:

@Article{kerbl3Dgaussians,
  author  = {Kerbl, Bernhard and Kopanas, Georgios and Leimk{"u}hler, Thomas and Drettakis, George},
  title   = {3D Gaussian Splatting for Real-Time Radiance Field Rendering},
  journal = {ACM Transactions on Graphics},
  number  = {4},
  volume  = {42},
  month   = {July},
  year    = {2023}
}

Final Thoughts

This is a research implementation, and the README reads like one—it's focused on the paper, the datasets, and the pre-trained artifacts rather than hand-holding you through installation. That's worth knowing before you dive in. But if you're working on novel-view synthesis, radiance fields, or anything that needs real-time rendering of reconstructed scenes, this is one of the more notable pieces of work in the space, and the availability of pre-trained models and viewers means you can see the results without a full training run. The bar for real-time 1080p rendering of unbounded scenes has been set, and it'll be interesting to see what gets built on top of it.

Follow @githubprojects for more developer tools and open source projects.