Theoretical Analysis of Thermodynamic Matrix Inversion: First-order Equivalence to Preconditioned Gradient Descent and Implications for Analog Computing

What the math gives to ML

The paper exposes a deterministic covariance iteration hidden inside thermodynamic matrix inversion: an Ornstein–Uhlenbeck discretization updates a matrix by two-sided contraction plus isotropic injection, and its fixed point is the inverse of an SPD matrix. This is a potentially useful optimizer primitive because it constructs an inverse-curvature or inverse-gradient-covariance preconditioner using only matrix multiplications, without stochastic sampling or explicit matrix inversion. The most direct neural-network transfer is a small-block adaptive preconditioner for second-order or K-FAC-like training, with damping and spectral step-size control; feasibility is limited by quadratic memory and compute cost, so experiments should begin with low-dimensional parameter blocks.