Unverified 2026

Microcanonical Krylov Stability Monitor

Implementation & benchmark of arXiv:2607.12585 — From phase space to Krylov space, one shell at a time

Usefulness6/10
Difficulty6/10
Novelty7/10

Source paper: From phase space to Krylov space, one shell at a time arXiv:2607.12585 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Construct a Lanczos chain for the neural-network vector field or hidden-state evolution, separately within bins of approximately constant loss, energy, or activation norm. Use the resulting Krylov complexity and Lanczos-coefficient growth as an early-warning signal for unstable training or long-horizon hidden-state amplification, then reduce the learning rate or recurrent integration step only in the unstable shells.

Formulas

$$\mathcal{L}f(z)=\nabla_z f(z)\cdot F(z),\qquad \langle f,g\rangle_e=\frac{1}{|\mathcal{S}_e|}\sum_{z_i\in\mathcal{S}_e}f(z_i)g(z_i),\qquad \mathcal{S}_e=\{z_i:|E(z_i)-e|<\Delta e\}.$$
$$\mathcal{L}O_n=b_{n+1}O_{n+1}+a_nO_n+b_nO_{n-1},\quad a_n=\langle O_n,\mathcal{L}O_n\rangle_e,\quad b_{n+1}=\left\|\mathcal{L}O_n-a_nO_n-b_nO_{n-1}\right\|_e.$$
$$K_e(t)=\sum_{n=0}^{m-1}n|\phi_n(t)|^2,\qquad \dot{\boldsymbol\phi}(t)=-T_e\boldsymbol\phi(t),\qquad (T_e)_{nm}=a_n\delta_{nm}+b_{n+1}\delta_{n+1,m}+b_n\delta_{n-1,m}.$$
$$\lambda_{K,e}=\frac{1}{2}\frac{d}{dt}\log\bigl(K_e(t)+\epsilon\bigr),\qquad \eta_e^{\mathrm{new}}=\min\!\left(\eta_e,\frac{c}{\lambda_{K,e}+\epsilon}\right).$$

Mathematical statement

Let $z$ denote parameters or a recurrent hidden state, let $F(z)$ be its continuous-time vector field, and let $\mathcal{L}f=\nabla_z f\cdot F(z)$ be the Koopman/Liouville generator acting on observables $f(z)$. For a shell $\mathcal{S}_e=\{z:|E(z)-e|<\Delta e\}$, define the empirical inner product $\langle f,g\rangle_e=|\mathcal{S}_e|^{-1}\sum_{z_i\in\mathcal{S}_e}f(z_i)g(z_i)$. Starting from normalized $O_0$, Lanczos orthogonalization gives $\mathcal{L}O_n=b_{n+1}O_{n+1}+a_nO_n+b_nO_{n-1}$, where $a_n=\langle O_n,\mathcal{L}O_n\rangle_e$ and $b_{n+1}=\|\mathcal{L}O_n-a_nO_n-b_nO_{n-1}\|_e$. The evolved observable has coefficients $\phi_n(t)=\langle O_n,O(t)\rangle_e$ and Krylov complexity $K_e(t)=\sum_n n|\phi_n(t)|^2$. The proposed neural transfer uses $\lambda_{K,e}=\tfrac12 d\log(K_e+\epsilon)/dt$ as an empirical instability rate and chooses a safe step size scaling as $\eta_e\lesssim c/\lambda_{K,e}$, where $c$ is calibrated.

Implementation notes

Integrate the mechanism at the optimizer or recurrent-cell level. For optimizer monitoring, take $z=\theta$ and $F(\theta)=-\widehat{\nabla}_\theta L$; for an RNN or SSM, take $z=h$ and $F$ equal to the continuous-time hidden-state drift. Choose an observable $O_0(z)$ such as a random projection of the gradient, hidden activation, or perturbation energy $\|\delta h\|^2$. Every $q$ updates, collect a replay window of states and assign samples to shells using loss, gradient norm, or activation norm. Estimate $\mathcal{L}O_n$ by finite differences $(O_n(z_{t+1})-O_n(z_t))/\Delta t$, or by Jacobian-vector products when available. Run $m=8$--$32$ Lanczos steps per shell with reorthogonalization, storing $a_n$ and $b_n$, then solve the resulting small tridiagonal ODE to obtain $K_e(t)$. Do not backpropagate through the monitor initially. Estimate $\lambda_{K,e}$ from a linear fit of $\log(K_e+\epsilon)$ over the first few time points, smooth it with an exponential moving average, and set the shell-specific learning-rate or recurrent-step multiplier to $\min(1,c/(\lambda_{K,e}\eta+\epsilon))$. The paper supplies the phase-space generator, Lanczos recursion, and microcanonical decomposition; the neural implementation must estimate the generator, shell measure, and calibration constant $c$. First test a 2-layer MLP on MNIST and a small chaotic-copy RNN task, comparing SGD or Adam with and without monitoring across a learning-rate sweep. The falsifiable prediction is that shells with larger measured $\lambda_{K,e}$ reach loss or hidden-state divergence first, and that the instability boundary follows $\eta_c\approx c/\lambda_{K,e}$ within 20 percent. Shell-resolved control should reduce failures more effectively than a global average Krylov complexity.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.