Unverified 2026

Backward-error penalty for learned latent dynamics

Implementation & benchmark of arXiv:2607.25086 — On residual bounds of the EDMD solution to the eigenvalue problem for the Koopman operator and backward shadowing stability of the EDMD/KMD

Usefulness6/10
Difficulty6/10
Novelty7/10

Source paper: On residual bounds of the EDMD solution to the eigenvalue problem for the Koopman operator and backward shadowing stability of the EDMD/KMD arXiv:2607.25086 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Train a learned latent transition not merely to fit one-step data, but to require only a small operator correction before its selected spectral modes become exact eigenmodes. The correction is a measurable backward error, so the regularizer penalizes models whose apparent eigenstructure is highly sensitive to noise or finite-sample error. At inference time, the correction norm can trigger conservative rollout or mode suppression.

Formulas

$$(\mathcal{K}-\Delta\mathcal{K})\phi_i=\lambda_i\phi_i,\qquad \Delta\mathcal{K}\phi_i=\mathsf{r}_i=\mathcal{K}\phi_i-\lambda_i\phi_i$$
$$\Delta K_* = R\Phi^\dagger,\qquad \Phi=[u_1,\ldots,u_s],\quad R=[r_1,\ldots,r_s],\quad r_i=K_\theta u_i-\lambda_i u_i$$
$$\mathcal{L}_{\mathrm{total}}=\mathcal{L}_{\mathrm{task}}+\alpha\|Z_+-K_\theta Z_-\|_F^2+\beta\|R\Phi^\dagger\|_2^2+\gamma\max(0,\rho(K_\theta)-1)^2$$
$$\widehat{\|r_i\|^2}=\sum_{t=1}^{N-1}w_t\left|u_i^*z_{t+1}-\lambda_i u_i^*z_t\right|^2$$

Mathematical statement

The paper fuses eigenpair residuals r_i=K phi_i-lambda_i phi_i into a perturbation Delta K satisfying (K-Delta K)phi_i=lambda_i phi_i, equivalently Delta K phi_i=r_i. For a matrix of latent eigenvectors Phi=[u_1,...,u_s] and residual matrix R=[r_1,...,r_s], the minimum-Frobenius-norm correction satisfying Delta K Phi=R is Delta K_*=R Phi^dagger, where Phi^dagger is the Moore-Penrose pseudoinverse. This is the finite-dimensional implementation of the paper's backward perturbation idea. Its computable stability penalty is ||Delta K_*||_2 or ||Delta K_*||_F. For orthonormal Phi, Delta K_*=R Phi^* and ||Delta K_*||_F=||R||_F. Here K_theta is the learned latent transition, lambda_i and u_i are its eigenvalues and eigenvectors, and r_i is evaluated on observed transitions as u_i^*z_{t+1}-lambda_i u_i^*z_t. Penalizing backward error differs from ordinary prediction error because Phi^dagger accounts for eigenvector conditioning: nearly dependent or poorly conditioned modes require a larger operator perturbation.

Implementation notes

Integrate the penalty after the latent transition matrix in a Koopman-inspired state-space model, recurrent predictor, or world model. For each minibatch of sequences, encode z_t=E_theta(x_t), estimate K_theta by ridge regression K_theta=Z_+Z_-^T(Z_-Z_-^T+eta I)^(-1), and compute its s eigenpairs (lambda_i,u_i), selecting modes with largest latent energy or decoder sensitivity. Form each residual r_i=K_theta u_i-lambda_i u_i and stack them into R; stack eigenvectors into Phi. Compute the pseudoinverse with an SVD Phi=U Sigma V^T, replacing singular values below 1e-5 times the largest singular value by zero, then calculate DeltaK=R Phi^dagger. Add beta times the squared spectral norm, approximated by 3-5 power iterations, to the task or reconstruction loss. Detach eigenvectors and K for the first prototype to avoid unstable eigenvalue gradients; refresh the penalty every 10-50 optimizer steps, then test a fully differentiable version. The exact backward-error relation is computed from the paper's residual construction, while ridge strength eta, mode selection, and power-iteration tolerance are empirical. Compare against ordinary one-step latent MSE, spectral-radius clipping alone, and Jacobian regularization on a noisy Lorenz-63 or Kuramoto benchmark. Evaluate validation residual, ||DeltaK||_2, rollout error at 10, 50, and 100 steps, and degradation under injected observation noise. The key falsifiable signal is lower long-horizon error and smaller noise-induced degradation at matched one-step loss, without simply shrinking all eigenvalues toward zero.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.