Curvature-Density Monitor for Optimization Transitions
Implementation & benchmark of arXiv:2607.29170 — Scalar curvature density as a new invariant in thermodynamic geometry: metric dependence and critical exponents
Source paper: Scalar curvature density as a new invariant in thermodynamic geometry: metric dependence and critical exponents arXiv:2607.29170 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Build a two-dimensional local metric from the neural-network loss along a pair of controlled parameter directions, such as the optimizer velocity and a stochastic-gradient fluctuation direction. Compute both scalar curvature R and curvature density mathcal R = sqrt(|g|) R, then use their different peaks or scaling laws to detect sharp optimization transitions and trigger learning-rate or regularization changes.
Formulas
Mathematical statement
For a two-dimensional metric g_ab(z), with coordinates z = (z^1,z^2), determinant g = det(g_ab), and scalar curvature R[g] computed from the metric and its derivatives, define the curvature density mathcal R(z) = sqrt(|g(z)|) R[g](z). The paper reports critical scalings R proportional to t^(-d nu) and mathcal R proportional to t^(-(1+beta)), where t is reduced distance from a critical point, d is spatial dimension, nu is the correlation-length exponent, and beta is the order-parameter exponent. For neural training, t can be defined as normalized distance from a detected optimization transition along update count or cumulative parameter displacement. The transferable prediction is that R and mathcal R have different peak locations or fitted exponents, so they can provide complementary early-warning signals.
Implementation notes
1. Integration point: add a monitoring callback to SGD or Adam every K updates. Define a local chart theta(z1,z2) = theta_k + z1 u_k + z2 v_k, where u_k is the normalized recent parameter-update direction and v_k is a normalized mini-batch-gradient-noise direction orthogonalized against u_k. Evaluate the loss on a small 5 by 5 grid around z = 0. 2. Pseudocode: estimate per-example or mini-batch gradients at each grid point; form g_ab as the average of partial_a loss times partial_b loss plus lambda times the identity; fit a smooth bicubic interpolant to the three independent metric components; use automatic differentiation or finite differences to obtain metric derivatives; compute Christoffel symbols and the two-dimensional scalar curvature R; output mathcal R = sqrt(abs(det(g))) times R. If either observable crosses a threshold, multiply the next learning rate by gamma_down, and restore it after M stable checks. 3. Taken from the paper: the distinction between R and curvature density and their different critical scaling. Estimated empirically: the local metric, derivatives, transition location, and exponents. Use lambda equal to 10^-6 times the median diagonal metric value to avoid singular charts. 4. First cheap experiment: train two-layer MLPs of widths 128 and 256 on MNIST with SGD over a learning-rate sweep. Compare standard SGD, curvature-triggered SGD, and a Hessian-trace monitor. Record loss, gradient norm, sharpness, R, and mathcal R. The predicted signature is a reproducible curvature peak or kink before divergence, with peak locations for R and mathcal R differing by several monitoring intervals. Fit each observable to Q proportional to abs(s - s_c)^(-p); the exponents should differ rather than collapse to one value. The method is promising only if the divergence boundary shifts upward or loss variance decreases by at least 10 percent at equal gradient budget.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.