Unverified 2026

Degenerate Invariant-Manifold RNN

Implementation & benchmark of arXiv:2607.25577 — Degenerate fixed points of maps in Banach spaces and lattices with decay and their invariant manifolds

Usefulness6/10

Difficulty6/10

Novelty7/10

Dynamics Stability Accuracy Memory Rnn Ssm World model Training dynamics Dynamical systems Functional analysis Approximation theory Linear algebra

Source paper: Degenerate fixed points of maps in Banach spaces and lattices with decay and their invariant manifolds arXiv:2607.25577 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an unconstrained recurrent state update by a locally parameterized invariant manifold h equals K of z, where the latent dynamics z at the next step equal R of z and preserve slow modes near a degenerate fixed point. Train the embedding and reduced map jointly with an invariance residual, while a weighted lattice norm discourages perturbations in distant channels or spatial sites from growing.

Formulas

$$F\circ K=K\circ R$$

$$\|h\|_w=\sum_{i\in\mathbb{Z}^q}w_i\|h_i\|_2,\qquad \mathcal{L}_{\mathrm{inv}}=\mathbb{E}_{z\sim\mu}\left[\|F(K(z))-K(R(z))\|_w^2\right]$$

$$K(z)=Az+\sum_{2\leq |\alpha|\leq p}K_\alpha z^\alpha,\qquad R(z)=Jz+\sum_{2\leq |\alpha|\leq p}R_\alpha z^\alpha$$

$$\|h_t-K(z_t)\|_w\approx C\rho_\perp^t,\qquad z_{t+1}=R(z_t),\quad \rho_\perp=\rho\left(DF(K(0))\big|_{\mathrm{normal}}\right)$$

Mathematical statement

Let F map a Banach hidden-state space E to itself, with fixed point F(0)=0. Let K map latent coordinates z in a subset V of R to the full hidden state, and let R map V to itself. The parametrization method seeks F composed with K equal to K composed with R. In a neural model, enforce this approximately with L invariant equal to the expected weighted squared norm of F(K(z)) minus K(R(z)). For lattice states h indexed by i in Z to the q power, use the weighted norm equal to the sum over i of w sub i times the Euclidean norm of h sub i, where w sub i are positive decay weights. Parameterize K by a linear term A z plus polynomial terms K sub alpha z to the alpha, and parameterize R by J z plus polynomial terms R sub alpha z to the alpha. The invariance equation preserves the manifold, while the normal spectral radius controls transverse stability.

Implementation notes

1. Exact integration point: use this construction as a recurrent or state-space cell in a GRU, SSM, or latent world model. Organize the hidden state into lattice sites h sub i, such as ordered memory slots, channels, or spatial patches. Define a decoder K with parameters theta from a d-dimensional latent state to the full hidden state and a small polynomial or MLP map R with parameters phi. The transition F with parameters psi remains the task model. For the first experiment, use zero input after an initial sequence so a local fixed-point neighborhood is meaningful. 2. Pseudocode: sample latent z near zero; compute h = K theta of z; compute h next = F psi of h and zero input; compute h target = K theta of R phi of z; add the weighted squared residual between h next and h target to the task loss. Use weights w sub i equal to one plus the absolute site index raised to s. Also penalize the norms of K of zero and R of zero. Initialize A from PCA of hidden trajectories and initialize J with eigenvalues near one for slow modes. 3. Computed from the paper: the invariance equation, the reduced-map representation, and the decay-weighted norm. Estimated empirically: K, R, the invariance residual, and the normal Jacobian spectrum using finite-difference Jacobian-vector products or automatic differentiation. 4. First cheap experiment: compare a vanilla GRU and this manifold-GRU on sequential MNIST or a synthetic delayed-copy task, with 32 hidden channels, a one-dimensional lattice, latent dimension d equal to 4, and polynomial degree p equal to 3. After the input ends, run both models for 500 free-recurrence steps. The quantitative prediction is that when the estimated normal spectral radius rho sub perpendicular is below one, the distance to the learned manifold decays with log slope approximately log of rho sub perpendicular. Check agreement within 20 percent. Sweep the invariance-loss coefficient and identify whether long-horizon stability degrades near the measured boundary rho sub perpendicular equal to one.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.