Peel-and-pass polynomial latent dynamics
Implementation & benchmark of arXiv:2607.06449 — Sparse space-time spectral methods can time-step by peel and pass
Source paper: Sparse space-time spectral methods can time-step by peel and pass arXiv:2607.06449 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace step-by-step hidden-state storage in a latent ODE, state-space model, or world model with a polynomial trajectory represented independently on short time blocks. At the end of each block, pass the next hidden state by summing temporal coefficients, allowing training and inference to discard the completed block while retaining a mathematically exact block interface.
Formulas
Mathematical statement
The paper uses a tensor-product spectral expansion u(t,\mathbf{x})=\sum_{j,\mathbf{k}}u_{j,\mathbf{k}}p_j(t)q_{\mathbf{k}}(\mathbf{x}), where p_j is a time polynomial, q_{\mathbf{k}} is a spatial basis function, and u_{j,\mathbf{k}} are coefficients. For Legendre polynomials P_j and Chebyshev-T polynomials T_j, P_j(1)=T_j(1)=1. Therefore, after mapping a block [t_n,t_{n+1}] to s\in[-1,1], a latent trajectory h(s)=\sum_{j=0}^{K}a_{n,j}p_j(s) satisfies h(t_{n+1})=h(1)=\sum_{j=0}^{K}a_{n,j}; the vector coefficients a_{n,j}\in\mathbb{R}^d are summed elementwise. The neural adaptation learns or computes coefficients for a latent state trajectory h_n(s) and passes h_{n+1}=\sum_j a_{n,j} to the next block. For a learned first-order dynamics model, coefficients can be trained with the collocation residual r(s)=\frac{2}{\Delta t}\sum_{j=0}^{K}a_{n,j}p'_j(s)-f_ heta(\sum_j a_{n,j}p_j(s),x_n(s),t_n(s)), where \Delta t=t_{n+1}-t_n, f_ heta is the neural vector field, and x_n(s) is the input interpolation. The endpoint sum is exact for the truncated polynomial representation; approximation error is controlled by increasing K or reducing \Delta t.
Implementation notes
Integrate this at the recurrent transition of a latent ODE, world model, or continuous-time SSM, specifically replacing the usual sequence of K ordinary Euler or RK hidden updates inside a chunk with a coefficient-prediction module. Use normalized local time s in [-1,1], choose shifted Legendre polynomials p_j(s)=P_j(s), and choose K between 3 and 8. For each chunk, inputs are the incoming state h_n, the chunk input tokens or observations x_{n,0:L-1}, and the chunk length Δt. A small transformer or MLP coefficient head predicts A_n=[a_{n,0},...,a_{n,K}], with each a_{n,j} in R^d. Evaluate the trajectory at C fixed Gauss-Legendre nodes s_l using H_l=Σ_j a_{n,j}p_j(s_l). If a neural vector field f_theta is available, add λ_dyn/C Σ_l ||r_n(s_l)||² to the task loss, where r_n is the displayed collocation residual and p'_j is precomputed once. The recurrent update is exactly h_{n+1}=Σ_j a_{n,j}; discard all H_l after the chunk, so activation memory is O(dK) rather than O(dL). A simpler first MVP can omit the residual and train the coefficient head end-to-end against next-chunk prediction, while retaining an endpoint consistency penalty between the coefficient sum and a teacher-forced final hidden state. Compute polynomial values and derivatives analytically or precompute their C×(K+1) matrices; no numerical estimation of mathematical constants is needed. Start with a 2-layer GRU or small MLP world model on sine-wave system identification and Penn Treebank character prediction, comparing against an equally sized GRU unrolled for L steps and a standard neural ODE solver. Measure peak activation memory, wall-clock time, long-horizon rollout error, and loss at equal parameter count. The expected signal is at least 2x lower activation memory for L much larger than K, with no degradation in one-chunk prediction and slower growth of rollout error when K is increased from 3 to 6. A falsification criterion is failure to match the baseline at equal FLOPs or instability when Δt is doubled.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.