✗ Failed on benchmark 2026

Quadratic center-manifold bottleneck

Implementation & benchmark of arXiv:2608.19486 — Beyond linear subspaces: Nonlinear moment matching meets quadratic manifolds

Usefulness7/10

Difficulty5/10

Novelty6/10

Architecture Accuracy Memory Sample efficiency Autoencoder World model Ssm Compression Dynamical systems Linear algebra Tensor decomposition Approximation theory Control theory

Source paper: Beyond linear subspaces: Nonlinear moment matching meets quadratic manifolds arXiv:2608.19486 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace a conventional linear decoder in an autoencoder or latent state-space model with an explicit quadratic manifold decoder, allowing a small latent vector to represent curved and transport-like state trajectories. Add a dynamics-aware invariance loss that penalizes the discrepancy between the time derivative of the quadratic manifold and the neural dynamics evaluated on that manifold.

Formulas

$$\Phi(z)=V_{1}z+V_{2}(z\otimes z),\qquad z\in\mathbb{R}^{r},\;V_{1}\in\mathbb{R}^{n\times r},\;V_{2}\in\mathbb{R}^{n\times r^{2}}.$$

$$W^{\top}V_{1}=I_{r},\qquad \hat z=W^{\top}x,$$

$$D\Phi(z)=V_{1}+V_{2}\bigl(I_{r}\otimes z+z\otimes I_{r}\bigr),$$

$$\mathcal{L}_{\mathrm{inv}}=\mathbb{E}_{(z,u)}\left\|D\Phi(z)h_{\theta}(z,u)-f_{\theta}(\Phi(z),u)\right\|_{2}^{2},\qquad \mathcal{L}=\mathcal{L}_{\mathrm{recon}}+\lambda_{\mathrm{inv}}\mathcal{L}_{\mathrm{inv}}+\lambda_{\mathrm{orth}}\|W^{\top}V_{1}-I_{r}\|_{F}^{2}.$$

Mathematical statement

The paper's construction uses a quadratic manifold of the form $\Phi(z)=V_{1}z+V_{2}(z\otimes z)$, where $z\in\mathbb{R}^{r}$ is the reduced coordinate, $V_{1}\in\mathbb{R}^{n\times r}$ is the linear projection matrix, $V_{2}\in\mathbb{R}^{n\times r^{2}}$ is a quadratic projection matrix, and $z\otimes z$ is the Kronecker product containing all ordered degree-two monomials. The reduced coordinate can be extracted with a test matrix $W\in\mathbb{R}^{n\times r}$ satisfying $W^{\top}V_{1}$ invertible; after normalization one may impose $W^{\top}V_{1}=I_{r}$. For a full dynamics $\dot{x}=f(x,u)$, the manifold dynamics are constrained by the invariance equation $D\Phi(z)\dot z=f(\Phi(z),u)$, where $D\Phi(z)=V_{1}+V_{2}(I_{r}\otimes z+z\otimes I_{r})$ under the ordered-Kronecker convention. The neural adaptation minimizes the practical residual $\|D\Phi(z)h_{\theta}(z,u)-f_{\theta}(\Phi(z),u)\|^{2}$, with $h_{\theta}$ the learned latent dynamics. The useful mathematical property is that the explicit quadratic manifold supplies both curvature and an analytic tangent map, rather than asking an unconstrained MLP decoder to discover them.

Implementation notes

1. Integration point: use this as the decoder and latent transition in a sequential autoencoder, neural ODE, or state-space model. The encoder maps an observed state or feature tensor $x_t\in\mathbb{R}^{n}$ to $z_t\in\mathbb{R}^{r}$; the decoder is the explicit quadratic map $\hat x_t=\Phi(z_t)$. For controlled data, use a latent transition $\dot z=h_\theta(z,u)$, or a discrete update $z_{t+1}=z_t+\Delta t\,h_\theta(z_t,u_t)$. 2. Pseudocode: initialize $V_1$ with the top PCA directions and $V_2$ near zero. For each minibatch, compute $z=E_\theta(x)$, $q=\mathrm{kron}(z,z)$, $\hat x=V_1z+V_2q$, and latent velocity $v=h_\theta(z,u)$. Compute the Jacobian analytically as $J=V_1+V_2(\mathrm{kron}(I,z)+\mathrm{kron}(z,I))$, then add $\|Jv-f_\theta(\hat x,u)\|^2$ to the reconstruction loss. Apply AdamW and optionally re-orthogonalize $V_1$ after each epoch so that $W^\top V_1\approx I$. 3. Computed from the mathematics: the Kronecker feature, analytic Jacobian, tangent-space invariance residual, and biorthogonality penalty. Estimated empirically: the latent dimension $r$, penalty weight $\lambda_{\mathrm{inv}}$, and whether all $r^2$ quadratic terms are needed; a symmetric Kronecker basis can reduce parameters from $r^2$ to $r(r+1)/2$. 4. First experiment: train a linear autoencoder, an MLP decoder with the same parameter budget, and this quadratic decoder on a one-dimensional advection equation or small forced wave-equation dataset, using $r=4,8,16$. Compare one-step reconstruction, long-horizon rollout error, and parameter count against PCA and a standard latent ODE. The expected signal is substantially lower reconstruction and rollout error at fixed $r$, especially for translated wave profiles, or the same error with a smaller latent dimension. Also test whether the invariance loss improves steady-state tracking under held-out forcing frequencies.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built an explicit ordered-Kronecker quadratic manifold decoder with analytic tangent evaluation and reproducible numerical checks. The mechanism manifested: central differences of the quadratic map were exact to floating-point error, the quadratic tangent contribution scaled with coefficient slope 1.00, invariance residual scaled as epsilon^2 with observed slope 2.00, and linear-decoder curvature error scaled as curvature^2 with observed slope 2.00. On translated Gaussian-profile data, the quadratic decoder reduced test MSE from 0.0443181 to 0.0229951, although it used more parameters.

Agent confidence: 8/10

Baseline: Linear decoder: test MSE 0.0443181, train MSE 0.0434609, 96 parameters
Idea: Quadratic decoder: test MSE 0.0229951, train MSE 0.0229754, 224 parameters; exact analytic invariance residual 0.0

Limitations: The experiment used fixed known latent coordinates and closed-form regression rather than training an encoder, latent dynamics, or full invariance-regularized autoencoder. It did not test long-horizon rollouts, controls, held-out forcing frequencies, equal-parameter comparisons, or a learned full-state vector field.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.19486) Sep 1, 2026

No significant difference vs. the tuned baseline (+21.7%, p=0.6974, wins 5 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0003
Idea mean: 0.0004
Effect (Δ): 0.0001 (+21.7%; negative = idea better)
Wins: 5 / 8 paired seeds
p-value: 0.6974 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±82.3%
Mechanism: Not confirmed ✗
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

Only the dynamics track was tested; long-horizon rollouts, explicit invariance-loss training, held-out forcing-frequency tracking, and larger latent-dimension sweeps were not tested. The implementation tests an explicit quadratic readout bottleneck end-to-end but does not add the full neural invariance residual loss because the built-in benchmark exposes a supervised one-step prediction task rather than a vector-field training interface.

How to run: python3 quadratic_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented and evaluated an end-to-end quadratic bottleneck on the structurally matched controlled-dynamics track, using paired GRU systems with identical data, training budget, and learning-rate search space. The analytic tangent sanity check passed, and the trained quadratic contribution scaled with the predicted slope of 2.0, confirming the mechanism quantitatively. However, the quadratic system had higher mean test MSE than the baseline (0.000373633 vs 0.000306926), with paired delta +0.0000667074 and permutation p=0.6974, so there was no significant improvement.

Baseline: Best lr=0.01; full 8-seed test MSE mean 0.000306925562, std 0.000114631221
Idea: Best lr=0.01; full 8-seed test MSE mean 0.000373633002, std 0.000292045128; paired delta +0.0000667074401; p=0.6974; 5/8 paired wins
Mechanism signature: Trained quadratic decoder output contribution norm versus latent scaling had observed log-log slope 1.999999995 versus predicted 2.0; confirmed=true; parameters baseline=13777, idea=13840

Benchmark result