Counterdiabatic spectral transport
Implementation & benchmark of arXiv:2608.04850 — Two-dimensional Toda--Arnoldi correspondence: Holomorphic Krylov geometry and counterdiabatic transport
Source paper: Two-dimensional Toda--Arnoldi correspondence: Holomorphic Krylov geometry and counterdiabatic transport arXiv:2608.04850 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
When a learned operator changes during training, add a frame-connection correction that transports its current Arnoldi representation instead of allowing hidden states to jump between evolving spectral directions. This is a geometry-aware residual or optimizer correction intended to reduce representation drift during aggressive learning-rate schedules, fine-tuning, and continual learning.
Formulas
Mathematical statement
Let s be a protocol parameter and U(s) = [u_0(s), ..., u_(D-1)(s)] be the unitary Arnoldi-frame matrix. Let A(s) = U(s)^* H(s) U(s) be the upper-Hessenberg representation of the operator H(s). Differentiating U^* U = I gives Omega(s) = U(s)^* dot(U)(s), with Omega^* = -Omega. Therefore G(s) = i Omega(s) is Hermitian. The paper's counterdiabatic construction uses the Arnoldi-frame connection as a Hermitian tridiagonal generator of exact isospectral transport along a smooth path. The ML adaptation keeps only diagonal and first off-diagonal entries, defining G_tri. In Arnoldi coordinates c = U^* h, add the transport term -i alpha G_tri c to the hidden-state dynamics. A finite-difference estimate uses dot(U)_t approximately (U_t - U_(t-1)) / Delta t after phase alignment of corresponding columns. The correction is not an assertion that arbitrary neural dynamics are isospectral; it is a falsifiable approximation whose purpose is to reduce basis-induced transition noise.
Implementation notes
Use this as a correction inside a residual SSM or as a representation-transport term coupled to an optimizer. Choose a small learned matrix H_t acting on hidden width D. Every 10 optimizer steps, compute an Arnoldi frame U_t from a fixed probe batch of hidden states and store U_(t-1). Align each new column by multiplying it by the phase or sign of u_(t-1,j)^* u_(t,j), preventing arbitrary basis flips. Estimate dot(U)_t = (U_t - U_(t-1)) / Delta t. Form Omega_t = one-half times [U_t^* dot(U)_t - (U_t^* dot(U)_t)^*], which explicitly enforces anti-Hermiticity, then set G_t = i Omega_t. Retain only entries with absolute index difference at most one to obtain G_tri,t. In Arnoldi coordinates c_t = U_t^* h_t, add -i alpha G_tri,t c_t to the residual update and transform back using h_t = U_t c_t. Initially stop gradients through U_t and tune alpha in {0.01, 0.1, 1}; later test full differentiation. The connection and tridiagonal locality come from the paper, while finite differences and alpha are engineering choices. Start with a width-64 residual SSM on sequential MNIST or a small language-model fine-tuning task. Compare AdamW with and without the correction under a ten-times larger learning rate or abrupt interpolation between two operators. Measure loss spikes, hidden-state cosine drift, gradient norms, and final accuracy. Success means fewer transition-induced spikes and better accuracy at the same update count.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.