Spherical harmonics and view-dependent colour

A single flat colour per Gaussian would make every splat scene look matte and slightly wrong: real surfaces shift in brightness and tint as you move around them, from a dull tile floor to a wet street to actual specular highlights on glass or metal, which is the problem spherical harmonicsBasis functions on the sphere used to encode view-dependent colour per Gaussian; band 0 is diffuse colour, higher bands add specular-like variation. solve: they give each Gaussian a compact function that answers “what colour, from this direction” instead of a single fixed answer, and understanding what you’re buying with each extra band explains a real, visible trade-off in your training and file size.

A function on a sphere, not a single value

Spherical harmonics are a family of basis functions defined over the surface of a sphere, the same role trigonometric functions play over a circle. Any reasonably smooth function of direction, here, “colour as seen from this viewing angle”, can be approximated by adding up a weighted sum of these basis functions. Each Gaussian stores the weights, called coefficients, rather than a lookup table of colours for every possible direction, which is what makes the representation compact enough to store per Gaussian at all.

SH bands: what each one buys you

The basis functions are organised into SH bandsThe successive groups of spherical-harmonics coefficients, band 0, band 1, band 2 and so on, each adding 2l+1 coefficients per colour channel; band 0 gives flat diffuse colour and higher bands add increasingly directional, specular-like variation at increasing storage cost., indexed from 0 upward. Band 0 is a single constant term: it has no directional variation at all, so it’s exactly the flat, view-independent diffuse colour you’d expect from a matte Lambertian surface. Band 1 adds 3 more coefficients per colour channel, band 2 adds 5, band 3 adds 7, each band contributing 2l + 1 coefficients for band index l. Every band beyond zero lets the colour function bend more sharply as the viewing angle changes, which is what produces glancing highlights, sheen and other angle-dependent effects. The 3D Gaussian Splatting paper uses four bands total (band 0 through band 3), giving 16 coefficients per colour channel, 48 numbers per Gaussian in total for red, green and blue combined, on top of position, covariance, and opacity.

Why training introduces bands gradually

If you let the optimiser fit all four bands from iteration one, it tends to overfit high-frequency directional noise before the low-order, low-frequency colour has even converged, especially in regions the training cameras only covered from a narrow range of angles. The paper’s optimiser instead starts with band 0 only and introduces one further band every 1000 iterations until all four are active. That ordering matches how the function itself behaves: band 0 is the cheapest, most stable thing to fit, and each subsequent band refines a colour that’s already roughly right rather than fighting for control of it from a blank start.

The storage cost is real, and it’s the biggest attribute per Gaussian

Position is 3 numbers, opacity is 1, the scale-and-rotation covariance parametrisation from covariance matrixA matrix describing a Gaussian's spread and orientation in space: its terms define how far the ellipsoid extends along each axis and how those axes are tilted, together encoding size, shape and orientation as a single object. is 7 (3 scale, 4 quaternion). Full spherical-harmonics colour is 48. That’s the majority of every Gaussian’s footprint in an uncompressed PLY, and it’s exactly why splat compression formats spend so much of their effort on colour specifically. SOG, for instance, keeps band 0 at higher precision and quantises the higher bands more aggressively, on the reasoning that a small error in a specular-highlight coefficient is far less visible than the same error in the base diffuse colour.

In practice

If a delivered splat looks flat and slightly plasticky compared to how the subject looked on site, especially reflective or wet surfaces, it may genuinely be a low-band export rather than a training failure: some pipelines and viewers cap SH degree to save bandwidth, which trades away exactly the directional shading that made the surface look real in person. Conversely, if a scene trained from a narrow, mostly frontal camera path shows odd colour shifts or blotching when you orbit around it in PlayCanvas, that’s usually higher-band SH coefficients doing their best to fit view directions the capture never actually covered, extrapolating confidently into directions it has no evidence for. Both point back to the same lever: SH degree is a genuine quality-versus-size dial, not just a compression detail, and it’s worth checking what degree PostShot exported and what your viewer is actually rendering before assuming a scene needs retraining.