Simplex-Stable Companion Memory
Implementation & benchmark of arXiv:2607.16529 — Positive-Allocation Companion Predictors for Nonlinear Dynamics and Their Finite-Difference Diagnostics
Source paper: Positive-Allocation Companion Predictors for Nonlinear Dynamics and Their Finite-Difference Diagnostics arXiv:2607.16529 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace an unconstrained linear recurrent or state-space memory with a finite-history recurrence whose coefficients are nonnegative and sum to one. The resulting companion transition is nonnegative and row-stochastic, guaranteeing spectral radius at most one while retaining a neutral constant-history mode at eigenvalue 1.
Formulas
Mathematical statement
Let h_t be a latent state in R^d and let alpha equal (alpha_0,...,alpha_{m-1}) satisfy alpha_j greater than or equal to zero and sum_j alpha_j equal to one. Define h_{t+1} as the sum over j of alpha_j h_{t-j}, and stack z_t as (h_t,h_{t-1},...,h_{t-m+1}). The scalar companion matrix C has first row alpha transposed, ones on its first subdiagonal, and zeros elsewhere; the vector-state transition is C Kronecker I_d. Every row of C sums to one and C is nonnegative, so its infinity norm is one, its spectral radius is at most one, and C times the all-ones vector equals the all-ones vector, implying that 1 is an eigenvalue. If C is diagonalizable and z_t is expanded in eigenmodes with eigenvalues lambda_l, then the first difference multiplies mode l by lambda_l minus 1 and the second difference multiplies it by lambda_l minus 1 squared. Thus modes near lambda equal to 1 are selectively suppressed by finite differences.
Implementation notes
1. Exact integration point: use this module as the linear memory core of an RNN, SSM, or latent world model. An encoder maps observation x_t to a latent vector u_t. Maintain a history of latent states h_t and predict the next state with the simplex-constrained companion recurrence. Inputs can be incorporated through a bounded additive term B u_t, or through a residual nonlinear branch whose output is clipped or norm-normalized: h_{t+1} equals the weighted history plus g_theta(x_t). For the cleanest stability test, first use the autonomous recurrence without the residual branch. 2. Pseudocode: initialize logits a in R^m and set alpha to softmax(a). Maintain a buffer containing h_t through h_{t-m+1}. Compute h_next as the sum over j of alpha_j times buffer[j], shift the buffer, decode x_hat_next from h_next, and optimize prediction or reconstruction loss. If coefficients adapt to context, compute logits a_t from a small network and apply softmax at every step; each instantaneous transition remains row-stochastic, although a common fixed spectral decomposition no longer applies. 3. Computed versus estimated: the simplex constraint and infinity norm bound are exact. Estimate spectral radius, Euclidean transient amplification, and modal decay numerically from learned coefficients. 4. First cheap experiment: train one-step and 50-step predictors on FitzHugh-Nagumo or SIR trajectories, comparing a GRU, an unconstrained companion recurrence, and this simplex companion with memory lengths 8, 16, and 32. The quantitative prediction is that the constrained model has spectral radius no greater than 1 plus 10^-6 and contains an eigenvalue within 10^-6 of 1. Its infinity norm power bound should satisfy ||C^k||_infinity <= 1 for every k. For each eigenmode, first-difference amplitude should scale as the absolute value of lambda minus 1, and second-difference amplitude as its square. Regress log difference amplitude against log absolute lambda minus 1 and expect slopes near 1 and 2.
Verification
Stage 1 · Toy mechanism gate: Passed ✓
Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗
Stage 2 · Practical benchmark result: Significantly worse ✗
Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built and ran a fixed-seed simplex companion-memory verification and forecasting experiment. Across 180 random simplex companions, spectral-radius and infinity-power excesses were below 5.8e-15, the distance to eigenvalue 1 was below 5.8e-15, and first/second finite-difference multiplier relative errors were below 6.0e-15/2.6e-14. The m=2 parameter sweep matched the predicted nonconstant eigenmode magnitude |q-1| at every q; the forecasting comparison showed a modest 40-step MSE improvement for simplex versus unconstrained (0.0003438 vs 0.0003532, 2.65%), at a tiny one-step cost (0.00006321 vs 0.00006320), so the stability mechanism clearly manifested while the predictive win is only preliminary.
- Agent confidence
- 9/10
- Baseline
- Unconstrained least-squares companion: one-step MSE 6.319867e-05; 40-step rollout MSE 3.531688e-04; no simplex stability guarantee.
- Idea
- Softmax simplex companion: coefficient sum 1.0, minimum coefficient 0.01360; one-step MSE 6.320586e-05; 40-step rollout MSE 3.437913e-04; max observed rho/power excess 5.8e-15; max distance to eigenvalue 1 5.8e-15; max first/second difference relative errors 6.0e-15/2.6e-14.
Limitations: Only a small synthetic scalar autoregressive forecasting task was tested; no GRU, FitzHugh-Nagumo/SIR trajectories, vector latent states, adaptive context-dependent coefficients, learned nonlinear residual branch, FLOP/speed comparison, or statistical repetition across independent experiment seeds was evaluated.
How to run: python3 verify_experiment.py
Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.16529)
Significantly WORSE than the tuned baseline (+40.7%, p=0.0081, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- companion_memory
- Paired seeds
- 8
- Baseline mean
- 0.003
- Idea mean
- 0.0042
- Effect (Δ)
- 0.0012 (+40.7%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0081 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±30.6%
- Mechanism
- Not confirmed ✗
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
Only the registered built-in dynamics track was tested. The implementation uses a fixed companion coefficient vector and does not test context-adaptive coefficients, longer rollout loss, alternative memory lengths, GRU baseline from the stock model, FLOP/speed, or larger datasets.
How to run: python3 bench_experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Stage 2 — Benchmark latest Failed ✗
Agent confidence: 9/10
The registered dynamics benchmark used matched unconstrained and simplex companion-memory systems with the same task, architecture components, optimizer, epochs, batch size, and paired seeds. The simplex idea was significantly worse: mean MSE 0.00419236 versus 0.00298017, paired delta +0.00121220, permutation p=0.0081, with zero wins in eight pairs. Its trained coefficients had the expected simplex stability properties, but the neural-scale finite-difference slopes did not quantitatively confirm the claimed 1 and 2 scaling.
- Baseline
- mean test MSE 0.002980166274937801 at lr=0.01
- Delta mean
- 0.0012121985055273399
- Idea
- mean test MSE 0.004192364780465141 at lr=0.01
- P value
- 0.0081
Benchmark result
Significantly WORSE than the tuned baseline (+40.7%, p=0.0081, wins 0 / 8 paired seeds).
- Benchmark
- Dynamics forecasting
- Model
- companion_memory
- Paired seeds
- 8
- Baseline mean
- 0.003
- Idea mean
- 0.0042
- Effect (Δ)
- 0.0012 (+40.7%; negative = idea better)
- Wins
- 0 / 8 paired seeds
- p-value
- 0.0081 (permutation test, 20 000 shuffles)
- Smallest detectable effect
- ±30.6%
- Mechanism
- Not confirmed ✗
- Practical effect
- Hurts
- Baseline tuning
- swept over 3 configs
Only the registered built-in dynamics track was tested. The implementation uses a fixed companion coefficient vector and does not test context-adaptive coefficients, longer rollout loss, alternative memory lengths, GRU baseline from the stock model, FLOP/speed, or larger datasets.
How to run: python3 bench_experiment.py
Artifacts
- bench_experiment.py 6.1 KB View Raw
- bench_report.json 4.5 KB View Raw
- report.md 1.7 KB View
- report_bench_2026-08-30T193049.md 6.4 KB View
- results.json 3.7 KB View Raw
- verify_experiment.py 5.8 KB View Raw