Training fundamentals

Training turns a COLMAP sparse point cloud and a folder of photos into a scene made of millions of optimised Gaussians, and it does this the same way most modern vision models learn: repeatedly rendering, comparing to a real photo, and nudging parameters to close the gap. Understanding what each part of that loop actually does explains two things practitioners hit constantly: why longer training helps some scenes and barely touches others, and why the loss function is built the way it is rather than scored on raw pixel error alone.

What updates on every step

Each Gaussian carries a handful of parameters: position, a covariance describing its size and orientation, opacity, and spherical-harmonics colour coefficients. A single training iteration picks a training camera, renders the current Gaussian set from that viewpoint through the differentiable rasteriser, compares the render against the real training photo taken from that pose, and computes a loss. That loss is then backpropagated through the rasteriser via gradient descentAn optimisation method that repeatedly nudges parameters in the direction that most reduces a loss function, following that loss's gradient with respect to each parameter. Splat training uses it, via backpropagation through the differentiable rasteriser, to adjust every Gaussian's position, shape, opacity and colour., nudging every Gaussian that contributed to the rendered pixels a small step in whatever direction reduces the error for that particular view.

Camera poses and the initial point positions come from COLMAP and are not themselves optimised in the reference pipeline: training only ever adjusts the Gaussians, never the cameras that observed them. densificationTraining-time process that clones and splits Gaussians in under-reconstructed regions and prunes low-opacity ones. runs interleaved with this loop, cloning, splitting and pruning Gaussians in response to gradient pressure; the mechanics of that process are covered in depth in densification and pruning. What matters here is the shape of a run: for a large stretch of training, gradient descent and densification happen together, and only in the back half does the Gaussian population settle down into pure refinement.

The loss: L1 plus a structural term, not raw pixel error

The original 3D Gaussian Splatting paper (Kerbl et al., 2023) combines two terms: L = (1 - lambda) times L1 plus lambda times L_D-SSIM, with lambda fixed at 0.2 across every experiment in the paper, an 80/20 split weighted toward plain per-pixel L1 error. The D-SSIM term contributes structural similarity, penalising loss of local contrast and structure rather than just per-pixel colour difference. That distinction matters in practice: a render can be off by a small, evenly distributed amount everywhere, something L1 punishes heavily, while still looking structurally correct, or it can hit the right average pixel value while quietly blurring away real detail, something L1 barely notices but D-SSIM does. Training on L1 alone tends to produce results that measure close but look visibly softer; the 20% D-SSIM weighting is a deliberate nudge toward preserving edges and texture, and it’s a fixed hyperparameter in the reference implementation rather than something that adapts itself per scene.

Iterations: what 7,000 and 30,000 actually buy

The reference implementation trains to 30,000 iterations by default and checkpoints and evaluates at both 7,000 and 30,000. The paper’s own comparison is direct: the fully converged, 30k model trains in roughly 35 to 45 minutes on their hardware and reaches quality on par with, and sometimes slightly ahead of, Mip-NeRF360, whose own reference training took around 48 hours on theirs. At 7,000 iterations, the paper notes quality is “already quite good” on many scenes, though not universally; some scenes still need the full run to resolve fine detail and view-dependent effects properly.

That split maps onto a genuinely useful production distinction. A 7k checkpoint is enough to sanity-check a capture, confirm exposure and coverage look right, spot an obviously failed region, before committing to a full run. It’s not, in general, a delivery-quality result. And because densification only runs through iteration 15,000, the difference between a 15k and a 30k checkpoint isn’t about growing more Gaussians: it’s the optimiser refining position, shape, opacity and colour on a population that’s already settled, which is exactly the kind of quiet convergence that’s easy to undervalue if progress is judged by watching Gaussian count rather than the render itself.

Iterations cannot fix a structural gap left by the capture. If a region comes out patchy at 30k, running to 60k or 100k rarely rescues it, because the underlying signal, matched features and reliable poses from COLMAP, was never there for gradient descent to work with in that area. That’s a coverage problem to solve by recapturing, not a training-length problem.

In practice

Most day-to-day training doesn’t touch the reference implementation’s raw flags directly; commercial tools expose iteration count as a preset or a quality slider rather than a command-line argument, but the same trade-off sits underneath every one of them. A fast preview pass is the right tool for judging capture quality early; a full pass is what compute time gets committed to once the capture itself is trusted. When a training run is dragging and quality genuinely isn’t improving, the reference implementation’s own checkpoint discipline, evaluate at a fast point, evaluate again once fully converged, is worth borrowing even inside a GUI trainer: if the fast checkpoint and the finished one look nearly identical in a walkthrough, the extra time bought very little, and it’s worth checking Gaussian count and quality settings before assuming a longer run will help.