Spectral duality structures and the Fisher--Rao geometry of reset distributions
arXiv:2608.15805
2026
Geometry
2 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper identifies a rigid information-geometric structure for probability simplices: under the square-root map, the Fisher–Rao metric becomes the Euclidean metric on a sphere, and the reset-neutral separatrix becomes a totally geodesic subsphere. This gives a principled way to represent and optimize mixture weights when a model should remain on a lower-dimensional calibrated family, avoiding distortions from Euclidean interpolation in probability coordinates. Its resolvent lemma also provides a constructive low-rank basis for parameter-dependent response functions, suggesting compression of scalar-conditioned modules and routers. The most practical experiment is a Fisher–Rao constrained routing layer, while the spectral response basis is a more speculative but potentially valuable architecture-compression technique.
Ideas from this paper
Unverified
2026
Replace Euclidean updates and interpolation of probability vectors in a mixture-of-experts router or attention simplex with updates in square-root coordinates, where the Fisher–Rao geometry is spherical. If the task has a desired neutral or calibrated family of distributions, represent that family as a linear subsphere in square-root space and project router outputs onto it after every update.
Useful6/10
Difficulty4/10
Novelty4/10
Unverified
2026
Compress a module whose output changes with a scalar condition such as diffusion time, temperature, or compute budget by representing its response in a low-rank basis generated by resolvent-like functions. Distinct spectral modes produce rational factors \((1-\tau\lambda_k)^{-1}\), allowing a small number of learned components to approximate a large hypernetwork or condition-dependent parameter table.
Useful5/10
Difficulty6/10
Novelty7/10