Unverified 2026

Counterdiabatic spectral transport

Implementation & benchmark of arXiv:2608.04850 — Two-dimensional Toda--Arnoldi correspondence: Holomorphic Krylov geometry and counterdiabatic transport

Usefulness6/10
Difficulty7/10
Novelty8/10

Source paper: Two-dimensional Toda--Arnoldi correspondence: Holomorphic Krylov geometry and counterdiabatic transport arXiv:2608.04850 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

When a learned operator changes during training, add a frame-connection correction that transports its current Arnoldi representation instead of allowing hidden states to jump between evolving spectral directions. This is a geometry-aware residual or optimizer correction intended to reduce representation drift during aggressive learning-rate schedules, fine-tuning, and continual learning.

Formulas

$$\gamma:s\longmapsto(z(s),\bar{z}(s)),\qquad \bar{z}(s)=z(s)^*.$$
$$\mathsf{H}\mathsf{U}=\mathsf{U}\mathsf{A},\qquad \mathsf{A}=\begin{pmatrix}a_0&h_{0,1}&\cdots&h_{0,D-1}\\ b_1&a_1&\ddots&\vdots\\ \vdots&\ddots&\ddots&h_{D-2,D-1}\\ 0&\cdots&b_{D-1}&a_{D-1}\end{pmatrix}.$$
$$\Omega=\mathsf{U}^*\dot{\mathsf{U}},\qquad \Omega^*=-\Omega,\qquad \mathsf{G}=i\Omega=i\mathsf{U}^*\dot{\mathsf{U}}=\mathsf{G}^*.$$
$$c_{t+1}=c_t+\eta F_\theta(c_t)-\eta i\alpha\mathsf{G}_{\mathrm{tri},t}c_t,\qquad (\mathsf{G}_{\mathrm{tri}})_{ij}=\begin{cases}\mathsf{G}_{ij},&|i-j|\le 1,\\0,&|i-j|>1.\end{cases}$$

Mathematical statement

Let s be a protocol parameter and U(s) = [u_0(s), ..., u_(D-1)(s)] be the unitary Arnoldi-frame matrix. Let A(s) = U(s)^* H(s) U(s) be the upper-Hessenberg representation of the operator H(s). Differentiating U^* U = I gives Omega(s) = U(s)^* dot(U)(s), with Omega^* = -Omega. Therefore G(s) = i Omega(s) is Hermitian. The paper's counterdiabatic construction uses the Arnoldi-frame connection as a Hermitian tridiagonal generator of exact isospectral transport along a smooth path. The ML adaptation keeps only diagonal and first off-diagonal entries, defining G_tri. In Arnoldi coordinates c = U^* h, add the transport term -i alpha G_tri c to the hidden-state dynamics. A finite-difference estimate uses dot(U)_t approximately (U_t - U_(t-1)) / Delta t after phase alignment of corresponding columns. The correction is not an assertion that arbitrary neural dynamics are isospectral; it is a falsifiable approximation whose purpose is to reduce basis-induced transition noise.

Implementation notes

Use this as a correction inside a residual SSM or as a representation-transport term coupled to an optimizer. Choose a small learned matrix H_t acting on hidden width D. Every 10 optimizer steps, compute an Arnoldi frame U_t from a fixed probe batch of hidden states and store U_(t-1). Align each new column by multiplying it by the phase or sign of u_(t-1,j)^* u_(t,j), preventing arbitrary basis flips. Estimate dot(U)_t = (U_t - U_(t-1)) / Delta t. Form Omega_t = one-half times [U_t^* dot(U)_t - (U_t^* dot(U)_t)^*], which explicitly enforces anti-Hermiticity, then set G_t = i Omega_t. Retain only entries with absolute index difference at most one to obtain G_tri,t. In Arnoldi coordinates c_t = U_t^* h_t, add -i alpha G_tri,t c_t to the residual update and transform back using h_t = U_t c_t. Initially stop gradients through U_t and tune alpha in {0.01, 0.1, 1}; later test full differentiation. The connection and tridiagonal locality come from the paper, while finite differences and alpha are engineering choices. Start with a width-64 residual SSM on sequential MNIST or a small language-model fine-tuning task. Compare AdamW with and without the correction under a ten-times larger learning rate or abrupt interpolation between two operators. Measure loss spikes, hidden-state cosine drift, gradient norms, and final accuracy. Success means fewer transition-induced spikes and better accuracy at the same update count.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.