Backward-error penalty for learned latent dynamics
Implementation & benchmark of arXiv:2607.25086 — On residual bounds of the EDMD solution to the eigenvalue problem for the Koopman operator and backward shadowing stability of the EDMD/KMD
Source paper: On residual bounds of the EDMD solution to the eigenvalue problem for the Koopman operator and backward shadowing stability of the EDMD/KMD arXiv:2607.25086 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Train a learned latent transition not merely to fit one-step data, but to require only a small operator correction before its selected spectral modes become exact eigenmodes. The correction is a measurable backward error, so the regularizer penalizes models whose apparent eigenstructure is highly sensitive to noise or finite-sample error. At inference time, the correction norm can trigger conservative rollout or mode suppression.
Formulas
Mathematical statement
The paper fuses eigenpair residuals r_i=K phi_i-lambda_i phi_i into a perturbation Delta K satisfying (K-Delta K)phi_i=lambda_i phi_i, equivalently Delta K phi_i=r_i. For a matrix of latent eigenvectors Phi=[u_1,...,u_s] and residual matrix R=[r_1,...,r_s], the minimum-Frobenius-norm correction satisfying Delta K Phi=R is Delta K_*=R Phi^dagger, where Phi^dagger is the Moore-Penrose pseudoinverse. This is the finite-dimensional implementation of the paper's backward perturbation idea. Its computable stability penalty is ||Delta K_*||_2 or ||Delta K_*||_F. For orthonormal Phi, Delta K_*=R Phi^* and ||Delta K_*||_F=||R||_F. Here K_theta is the learned latent transition, lambda_i and u_i are its eigenvalues and eigenvectors, and r_i is evaluated on observed transitions as u_i^*z_{t+1}-lambda_i u_i^*z_t. Penalizing backward error differs from ordinary prediction error because Phi^dagger accounts for eigenvector conditioning: nearly dependent or poorly conditioned modes require a larger operator perturbation.
Implementation notes
Integrate the penalty after the latent transition matrix in a Koopman-inspired state-space model, recurrent predictor, or world model. For each minibatch of sequences, encode z_t=E_theta(x_t), estimate K_theta by ridge regression K_theta=Z_+Z_-^T(Z_-Z_-^T+eta I)^(-1), and compute its s eigenpairs (lambda_i,u_i), selecting modes with largest latent energy or decoder sensitivity. Form each residual r_i=K_theta u_i-lambda_i u_i and stack them into R; stack eigenvectors into Phi. Compute the pseudoinverse with an SVD Phi=U Sigma V^T, replacing singular values below 1e-5 times the largest singular value by zero, then calculate DeltaK=R Phi^dagger. Add beta times the squared spectral norm, approximated by 3-5 power iterations, to the task or reconstruction loss. Detach eigenvectors and K for the first prototype to avoid unstable eigenvalue gradients; refresh the penalty every 10-50 optimizer steps, then test a fully differentiable version. The exact backward-error relation is computed from the paper's residual construction, while ridge strength eta, mode selection, and power-iteration tolerance are empirical. Compare against ordinary one-step latent MSE, spectral-radius clipping alone, and Jacobian regularization on a noisy Lorenz-63 or Kuramoto benchmark. Evaluate validation residual, ||DeltaK||_2, rollout error at 10, 50, and 100 steps, and degradation under injected observation noise. The key falsifiable signal is lower long-horizon error and smaller noise-induced degradation at matched one-step loss, without simply shrinking all eigenvalues toward zero.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.