What is a Gaussian splat?

A 3D Gaussian splat scene represents the world as millions of anisotropic 3D Gaussians, each carrying a position, a covariance (an oriented ellipsoid shape), an opacity and a view-dependent colour encoded as spherical harmonicsBasis functions on the sphere used to encode view-dependent colour per Gaussian; band 0 is diffuse colour, higher bands add specular-like variation.. It’s a photoreal, real-time alternative to a mesh or a point cloud, built directly from ordinary photos or video rather than hand-modelled. If you capture a site, run it through COLMAP and PostShot, and view the result in PlayCanvas, a splat is what comes out the other end.

Points, but with volume and direction

A point cloud gives you a position and a colour per sample and nothing else. A Gaussian splat replaces the point with a small blob of probability: an ellipsoid with its own scale and rotation, so it can be a thin sliver along an edge or a broad soft patch across a wall, whichever fits the surface it’s approximating. Opacity lets Gaussians overlap and blend rather than compete for a single pixel, which is how a splat scene renders soft edges, thin structures like railings and foliage, and reflective or glancing surfaces that a rigid mesh triangle struggles with. The colour isn’t a single fixed value either: spherical harmonics store how a Gaussian’s apparent colour changes with viewing angle, so a glossy surface can look convincingly different from two sides without any separate reflection pass.

A radiance field, made explicit

Formally, a splat scene is one way of representing a radiance fieldA function that maps a 3D position and viewing direction to a colour and density or opacity, describing how a scene looks from any viewpoint. Both NeRFs and Gaussian splats are representations of a radiance field, one implicit, one explicit.: a function that says what colour and opacity you’d see from any position looking in any direction. NeRFNeural Radiance Field: a scene representation that encodes colour and density as the weights of a neural network, rendered by marching rays through the volume and querying the network at each sample point. represents that same function implicitly, as weights inside a neural network, and recovers an image by marching rays through the volume and querying the network at each sample point along the way. A Gaussian splat represents the function explicitly instead, as a list of primitives you can point a GPU rasteriser at directly. Explicit and queryable is what makes splats fast to render and comparatively easy to edit, prune or merge; it’s also why they’re memory-hungry compared with a small MLP, which is the whole reason compressed formats like SOG exist further down the pipeline.

No network at render time

This is the detail that matters most in practice: at render time, nothing gets queried. The Gaussians already know their own position, shape, opacity and colour, so the renderer only needs to sort them and composite them onto the screen, which is exactly the kind of embarrassingly parallel job a GPU rasteriser is built for. That’s the core claim of the original 2023 3D Gaussian Splatting paper, and it’s the reason splat viewers hit real-time frame rates on hardware that would choke rendering even a single frame of a comparable NeRF. Training is still expensive, often minutes to tens of minutes depending on scene complexity and target quality, but that cost is paid once, offline, rather than on every frame a user looks at.

In practice

For a working pipeline this distinction is what decides where the effort goes. Capture and Structure-from-Motion (COLMAP) exist to recover accurate camera poses and a sparse point cloud that seeds the Gaussians. Training (PostShot or the reference implementation) is where those Gaussians are actually optimised against the training images, densified where detail is missing and pruned where they add nothing. Compression (SOG) exists purely to make the resulting file small enough to stream. Web delivery (PlayCanvas) is the one stage that only has to sort and rasterise, which is why it’s the cheapest part of the whole chain, and why a splat that looks wrong in the browser is almost always a capture, training or compression problem, not a rendering one. Keeping that mental model, representation is separate from rendering, saves a lot of time chasing bugs in the wrong stage.