# Cross-prediction determinism gate

- ID: 1927
- Canonical URL: https://synthcore.org/idea/1927/cross-prediction-determinism-gate
- API JSON: https://synthcore.org/api/idea/1927.json
- API Markdown: https://synthcore.org/api/idea/1927.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.07109](https://arxiv.org/abs/2608.07109)
- Category: dynamics
- Solves: stability, generalization, accuracy
- ML areas: rnn, ssm, training-dynamics, regularization
- Math tags: dynamical-systems, statistics, stochastic-processes, control-theory
- Ratings: usefulness 7/10; difficulty 4/10; novelty 8/10

## Idea description

Use a frozen echo-state reservoir and a linear readout to measure whether a time series contains reproducible dynamical structure rather than memorisable temporal correlations. Apply the held-out cross-prediction score as an early-stopping signal, data-quality gate, or regularizer for an RNN or neural state-space forecaster. The mechanism should reduce overfitting to stochastic fluctuations while preserving genuinely predictable chaotic structure.

## Mathematical statement

For observations u_t in R^d, maintain a fixed reservoir state r_t in R^n with leak rate a in (0,1], recurrent matrix W_r, input matrix W_in, bias b, and tanh activation: r_t=(1-a)r_{t-1}+a tanh(W_r r_{t-1}+W_in u_t+b). Define z_t=[r_t;u_t;1] and the future increment Delta_tau u_t=u_{t+tau}-u_t. Fit only a linear readout W_o by ridge regression on training trajectories. Evaluate on an independent trajectory or temporal block using the squared Pearson correlation Q_tau=rho(Delta_tau u, predicted Delta_tau u)^2. Cross-prediction prevents a readout trained on one realization from receiving credit for correlations that do not transfer to another realization. Deterministic or chaotic dynamics should produce a larger Q_tau than matched-correlated-noise and shuffled surrogate data.

## Key formulas

- $$\mathbf{r}_t=(1-a)\mathbf{r}_{t-1}+a\tanh(W_r\mathbf{r}_{t-1}+W_{in}u_t+b),\qquad z_t=[\mathbf{r}_t;u_t;1]$$
- $$\Delta_\tau u_t=u_{t+\tau}-u_t,\qquad \widehat{\Delta_\tau u}_t=W_o z_t$$
- $$W_o=\arg\min_W\sum_{t\in\mathcal{T}_{train}}\|Wz_t-\Delta_\tau u_t\|_2^2+\lambda\|W\|_F^2$$
- $$Q_\tau=\rho(\Delta_\tau u,\widehat{\Delta_\tau u})^2,\qquad G_\tau=Q_\tau^{real}-\operatorname{median}_{k=1}^{K}Q_{\tau,k}^{surrogate}$$

## Implementation notes

1. Integration point: add this diagnostic beside any sequence forecaster, especially a GRU, RNN, or neural state-space model. Keep the reservoir fixed and do not train it jointly with the main model. Draw W_r and W_in once, rescale W_r to a selected spectral radius, discard an initial washout interval, and fit only the ridge readout on one subset of trajectories. Evaluate on independent trajectories or non-overlapping temporal blocks. Use G_tau as an early-stopping monitor or as a gate that downweights examples whose apparent predictability is not cross-reproducible. 2. Pseudocode: initialize r=0; at each time step update r=(1-a)r+a*tanh(W_r*r+W_in*u+b); store z=[r,u,1] and target u[t+tau]-u[t]; solve W_o=Y X^T (X X^T+lambda I)^(-1) on split A; compute Q_tau on split B; repeat for K phase-randomized, block-shuffled, or matched-autocorrelation surrogate datasets. Reduce the main optimizer learning rate or stop training when forecasting loss improves while G_tau decreases. 3. Taken directly from the paper are the frozen-reservoir architecture, future-increment target, squared Pearson criterion, and cross-prediction protocol. Estimate empirically the reservoir spectral radius, leak rate, ridge coefficient, surrogate null distribution, and threshold for G_tau. 4. First cheap experiment: compare a GRU and neural SSM on Lorenz-63 observations, an AR(1) process with matched autocorrelation, and noisy Lorenz data. Use 5 to 10 reservoir seeds and horizons tau=1,2,4,8,16. The predicted signature is Q_tau(Lorenz)>Q_tau(AR1), while G_tau for shuffled controls is near zero. The real-data Q_tau should decline approximately monotonically with tau, and the real-versus-surrogate gap should remain positive across seeds and exceed the surrogate 95 percent interval.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built a frozen leaky-tanh reservoir, ridge readout, squared-Pearson cross-prediction score, AR control, and phase-surrogate evaluation in experiment.py. Three quantitative math predictions were confirmed: reservoir states stayed below 1.0 for spectral radii 0–3, Pearson Q was invariant to prediction scaling with numerical spread below 1e-15, and ridge regularization shrank readout norm from 0.208 to 1.0e-6 while leaving Q unchanged as expected for correlation. The mechanism produced a strong cross-trajectory determinism gap for logistic dynamics versus AR(1) noise at N=3200, but phase-randomized surrogates were not consistently separated at longer horizons, so the result is promising rather than universally validated.

### Mechanism check

- Verdict: Built a frozen leaky-tanh reservoir, ridge readout, squared-Pearson cross-prediction score, AR control, and phase-surrogate evaluation in experiment.py. Three quantitative math predictions were confirmed: reservoir states stayed below 1.0 for spectral radii 0–3, Pearson Q was invariant to prediction scaling with numerical spread below 1e-15, and ridge regularization shrank readout norm from 0.208 to 1.0e-6 while leaving Q unchanged as expected for correlation. The mechanism produced a strong cross-trajectory determinism gap for logistic dynamics versus AR(1) noise at N=3200, but phase-randomized surrogates were not consistently separated at longer horizons, so the result is promising rather than universally validated.
- Confidence: 8/10
- Limitations: No GRU or neural state-space early-stopping integration was tested; only scalar logistic dynamics and AR/phase controls were evaluated. The phase-surrogate null was not reliably below the chaotic score at long horizons, and no confidence intervals beyond three reservoir seeds were computed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/731/bench_report.json)
- [bench_stage2.py](https://synthcore.org/code/731/bench_stage2.py)
- [experiment.py](https://synthcore.org/code/731/experiment.py)
- [report.md](https://synthcore.org/code/731/report.md)
- [report_bench_2026-09-01T013726.md](https://synthcore.org/code/731/report_bench_2026-09-01T013726.md)
- [results.json](https://synthcore.org/code/731/results.json)
- [Download all files as ZIP](https://synthcore.org/download/731)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.