△ Mechanism confirmed, baseline not beaten 2026

Persistent Workspace for Online Adaptation

Implementation & benchmark of arXiv:2608.14019 — Emergent Models: Intelligence from Tiny Substrates

Usefulness7/10

Difficulty6/10

Novelty7/10

Architecture Sample efficiency Memory Accuracy Rnn Ssm World model Training dynamics Dynamical systems Computation Optimization

Source paper: Emergent Models: Intelligence from Tiny Substrates arXiv:2608.14019 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Turn the latent substrate into a persistent computational workspace for sequential inputs: each new observation is written into a designated subspace, processed by the same local rule, decoded, and then selectively retained or reset. This creates a compact recurrent model whose state can accumulate algorithmic information across a stream without expanding the parameter count.

Formulas

$$\begin{array}{l}\textbf{Input stream }(x_i)_{i\in\mathrm{macrosteps}}\\ \textbf{Program }p\\ \texttt{s <- p}\\ \textbf{for }i\textbf{ in macrosteps do}\\ \qquad\texttt{I <- InputSubspace(s, x_i)}\\ \qquad\texttt{s[I] <- E(x_i)}\\ \qquad\textbf{for }t\textbf{ in microsteps do}\\ \qquad\qquad\texttt{s <- f(s)}\\ \qquad\texttt{y_i <- D(s)}\\ \qquad\texttt{s <- R(s)}\end{array}$$

$$s_{i,0}[I_i]\leftarrow E(x_i),\qquad s_{i,t+1}=f_\theta(s_{i,t}),\qquad y_i=D_\theta(s_{i,T}),\qquad s_{i+1,0}=R_\theta(s_{i,T})$$

$$g_i=\sigma\!\left(W_g[\operatorname{pool}(s_{i,T});E(x_i)]+b_g\right),\qquad R_\theta(s)=g_i\odot s+(1-g_i)\odot s_{\mathrm{reset}}$$

$$\mathcal L=\mathbb E\left[\sum_i\ell(y_i,\hat y_i)\right]+\beta\,\mathbb E\left[\|s_{i+1,0}-R_\theta(s_{i,T})\|_2^2\right]$$

Mathematical statement

For a sequential stream $(x_i)_{i\in\mathrm{macrosteps}}$, the paper writes each input into an input subspace of a persistent state, iterates the same rule for microsteps, decodes $y_i$, and applies a retention map $R$: I <- InputSubspace(s,x_i), s[I] <- E(x_i), s <- f(s) repeatedly, y_i <- D(s), s <- R(s). Here $s$ is the recurrent workspace, $I$ is the set of writable coordinates, $E$ is the input encoder, $f$ is the shared transition, $D$ is the readout, and $R$ controls memory retention or reset. Adapt $R$ to a learned elementwise gate $g_i=\sigma(W_g[s_i;E(x_i)]+b_g)$ and update $s_i^+=g_i\odot s_i^-+(1-g_i)\odot \tilde s_i$, while protecting separate read and write regions. The resulting recurrence is a structured state-space model with nonlinear local computation and explicit write/retain semantics. The paper's objective $\theta^\star\in\arg\min_\theta\mathbb E[\ell(\hat y,y)]$ can train the interface and rule; auxiliary state-consistency or reset penalties can prevent uncontrolled accumulation.

Implementation notes

Integrate this module between an input projection and a prediction head in a streaming classifier, language-modeling toy task, or partially observed control environment. Partition the latent vector or grid into three fixed regions: write ports containing the current observation, workspace cells that persist across steps, and read ports used by the decoder. At macrostep $i$, compute encoded=E(x_i), overwrite only the write ports, run $T$ applications of the shared local transition s=f(s), read y_i=D(s), and apply the retention gate. Pseudocode: s=init(); for x_i,target_i in stream: z=E(x_i); s[write_idx]=z; for t in range(T): s=f(s); y_i=D(s); g=sigmoid(Wg*concat(pool(s),z)+bg); s=g*s+(1-g)*s_reset; loss += CE(y_i,target_i). Detach the state only at truncated-BPTT boundaries, not after every macrostep. The paper supplies the explicit write–iterate–read–retain protocol; the gated $R$ is the proposed adaptation. Estimate memory retention by measuring predictive information after distractor inputs, and monitor state norm and gradient stability. First experiment on permuted sequential MNIST with distractor symbols between relevant digits, using equal-parameter GRU and S4/SSM baselines. Success is higher accuracy at the same hidden size, or comparable accuracy with 2x fewer state dimensions, especially when relevant evidence is separated by long distractor intervals. Also test online adaptation after a distribution switch without resetting the state.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a gated persistent workspace with explicit write-port/workspace partitioning, shared nonlinear transition, readout, and learned retention gate. The three mechanism predictions were numerically confirmed: retention matched g^n, overwrite matched 1-g, and the stability boundary matched lambda*g <= 1 across the tested sweep. The delayed-bit task showed no practical accuracy win: the ungated baseline reached 1.00 at all gaps, while the gated workspace reached 1.00, 0.495, and 0.529 for gaps 2, 8, and 16; therefore the mechanism manifested, but this setup harmed long-range task performance.

Agent confidence: 8/10

Baseline: Delayed-bit accuracy: gap 2=1.000, gap 8=1.000, gap 16=1.000; analytic retention and stability predictions matched the tested cases.
Idea: Gated workspace delayed-bit accuracy: gap 2=1.000, gap 8=0.495, gap 16=0.529; retention at g=0.8,n=20 was observed/predicted 0.011529/0.011529 and at g=0.99,n=20 was 0.817907/0.817907.

Limitations: Only a tiny synthetic delayed-bit task was tested, not sequential MNIST, language modeling, S4, or distribution-switch adaptation. The baseline was an ungated version of this toy workspace rather than a separately tuned GRU, and only one random seed and training configuration were used.

How to run: python3 persistent_workspace_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.14019) Sep 2, 2026

Significantly WORSE than the tuned baseline (+13.9%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Sequence modeling
Model: persistent_workspace_shared_transition
Paired seeds: 8
Baseline mean: 0.1449
Idea mean: 0.1651
Effect (Δ): 0.0202 (+13.9%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±5.6%
Mechanism: Confirmed ✓
Practical effect: Hurts
Baseline tuning: swept over 9 configs

Limitations:

Only the registered sequence track was tested; dynamics, longer horizons, larger datasets, GRU/SSM baselines, and distribution-switch adaptation were not tested.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

The persistent workspace was evaluated on the registered sequence forecasting track with 8 paired seeds and a tuned baseline sweep. The idea's best MSE was 0.165073 versus baseline 0.144882; paired delta was +0.020191 with permutation p=0.0081, so the idea was significantly worse. The trained-model mechanism signature was confirmed, but the task metric did not improve.

Baseline: mean test MSE 0.1448821258, std 0.0113162536, best lr=0.006 retention=1.0
Idea: mean test MSE 0.1650733370, std 0.0140552122, best lr=0.006 gate_bias=1.0

Benchmark result