How splats render

Once a scene is trained, a Gaussian splatA single anisotropic 3D Gaussian, an ellipsoid with position, covariance, opacity and view-dependent colour, used as the basic rendering primitive of a splat scene. Millions of these together, optimised against training images, make up the scene. renderer has one job every frame: work out which Gaussians are visible, put them in the right order, and blend them together into pixels. There’s no neural network in that loop and no per-pixel ray marching either. It’s a GPU rasterisationConverting geometric primitives directly into pixels on screen, rather than tracing or marching rays through a scene. 3D Gaussian Splatting adapts a tile-based GPU rasterisation pipeline to render Gaussians instead of triangles. pipeline adapted to a primitive that isn’t a triangle, and understanding its three stages explains most of the artefacts you’ll see in a browser viewer.

Project: turning ellipsoids into ellipses

The first stage takes each 3D Gaussian, its centre, its covariance (the ellipsoid’s size and orientation), its opacity and its spherical-harmonics colour, and projects it into the current camera’s 2D image plane. A 3D covariance matrix projects to a 2D covariance matrix under the camera’s view and projection transforms, which turns each ellipsoid into a 2D ellipse: a splat, in the literal sense. This stage also computes each Gaussian’s depth from the camera and culls anything outside the view frustum, so the later stages only ever touch what could plausibly land on screen.

Sort: per-tile, by depth

The screen is divided into small non-overlapping tiles, 16x16 pixels in the original formulation. Every projected Gaussian is duplicated into every tile its 2D footprint overlaps, and each of those tile instances is given a sort key that packs the tile ID and the Gaussian’s depth together. Sorting that whole key list once puts every tile’s Gaussians in depth order in a single pass, which is what lets the next stage run independently and in parallel per tile rather than resolving overlap pixel by pixel across the whole image. This per-tile sort is also the single most expensive part of the pipeline computationally, which is why recent accelerator work targets exactly this step: reducing redundant sorting between tiles without changing what gets drawn.

Blend: alpha compositing, front to back

Within a tile, the renderer walks its sorted Gaussians front to back and accumulates colour with alpha blendingA compositing technique that accumulates colour from ordered, semi-transparent primitives, each weighted by its own opacity and by how much light nearer primitives have already absorbed. Splat rasterisers use it to blend overlapping Gaussians into a final pixel colour.: each Gaussian contributes its colour weighted by its own opacity and by how much of the light behind it has already been absorbed by nearer Gaussians. A pixel under a dense stack of overlapping Gaussians converges quickly because the accumulated opacity saturates; a pixel that only grazes the edge of a few thin Gaussians stays translucent and keeps compositing further back. This is also why splat scenes handle soft edges, hair, foliage and glass so much more naturally than a triangle mesh: there’s no single depth test winner per pixel, only a weighted stack.

Why rasterise instead of ray-march

A NeRF produces an image by marching a ray through the volume for every pixel and querying its network at each sample point along that ray, which means the cost of a frame scales with how many samples each ray needs and how expensive the network query is. A splat renderer instead has an explicit, finite list of primitives it can project once, sort once and blend directly, with no network evaluation anywhere in the loop. That difference is what the original 2023 paper’s real-time claim rests on: rendering becomes a fixed, GPU-friendly workload instead of one that scales with sampling density, which is exactly the property that made splats viable for real-time delivery in the first place rather than just another offline-rendered radiance field.

In practice

Most of what looks broken in a splat viewer traces back to one of these three stages, not the scene itself. Popping or flickering as the camera moves is usually a sort-order artefact at tile boundaries. A hazy or “foggy” look over part of a scene is accumulated alpha from Gaussians that should have been pruned in training but weren’t. And frame-rate drops in dense areas are a tile-overlap and sort-cost problem, which is why splat counts and Gaussian scale (how large a footprint each one casts across tiles) matter as much for browser performance as raw file size does. When a PlayCanvas viewer stutters on a scene that compressed fine, this pipeline, not the network tab, is where to look first.