△ Mechanism confirmed, baseline not beaten 2026

Doob barrier consolidation

Implementation & benchmark of arXiv:2607.06924 — Intrinsic-Noise Consolidation: A Doob-Barrier-Conditioned Diffusion Turns Analog Device Noise into a Continual-Learning Resource

Usefulness8/10
Difficulty4/10
Novelty8/10

Source paper: Intrinsic-Noise Consolidation: A Doob-Barrier-Conditioned Diffusion Turns Analog Device Noise into a Continual-Learning Resource arXiv:2607.06924 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Add a Doob-transformed barrier drift to parameters during sequential-task training, conditioning each noisy parameter trajectory to remain within an interval around its previous-task anchor. The correction is weak at the anchor, grows toward the barriers, and increases with the injected noise variance, providing state-dependent protection that quadratic anchoring does not provide.

Formulas

$$dw_i=\Big[-\partial_{w_i}\mathcal{L}_{\mathrm{task}}-s_i(w_i-\mu_i)+\sigma_i^2\partial_{w_i}\log h_i(w_i)\Big]dt+\sigma_i\,dW_i.$$
$$I_i=(\mu_i-b_i,\mu_i+b_i),\qquad h_i(w)=\cos\!\left(\frac{\pi(w-\mu_i)}{2b_i}\right),\qquad \partial_w\log h_i(w)=-\frac{\pi}{2b_i}\tan\!\left(\frac{\pi(w-\mu_i)}{2b_i}\right).$$
$$w_i^{k+1}=w_i^k+\eta\left[-g_i^k-s_i(w_i^k-\mu_i)-\sigma_i^2\frac{\pi}{2b_i}\tan\!\left(\frac{\pi(w_i^k-\mu_i)}{2b_i}\right)\right]+\sqrt{\eta}\,\sigma_i\varepsilon_i^k,\quad \varepsilon_i^k\sim\mathcal{N}(0,1).$$

Mathematical statement

The paper uses the stochastic differential equation $dw_i=[-\partial_{w_i}\mathcal{L}_{\mathrm{task}}-s_i(w_i-\mu_i)+\sigma_i^2\partial_{w_i}\log h_i(w_i)]dt+\sigma_i dW_i$. Here $w_i$ is parameter $i$, $\mathcal{L}_{\mathrm{task}}$ is the current-task loss, $\mu_i$ is the parameter value stored after the previous task, $s_i\geq 0$ is an importance coefficient such as an online diagonal Fisher estimate, $\sigma_i\geq 0$ is the noise scale, and $W_i$ is standard Brownian motion. The parameter is constrained to the interval $I_i=(\mu_i-b_i,\mu_i+b_i)$, where $b_i>0$ is the barrier half-width. For Brownian motion killed at the endpoints, the ground-state h-function is $h_i(w)=\cos(\pi(w-\mu_i)/(2b_i))$. Its logarithmic derivative is $\partial_w\log h_i(w)=-(\pi/(2b_i))\tan(\pi(w-\mu_i)/(2b_i))$. Consequently, the additional drift is $-\sigma_i^2[\pi/(2b_i)]\tan(\pi(w_i-\mu_i)/(2b_i))$. It is zero at $w_i=\mu_i$, points toward the anchor on both sides, and diverges as $w_i$ approaches either barrier. The ordinary $-s_i(w_i-\mu_i)$ term is standard EWC-like anchoring; the transferable mechanism is the variance-scaled logarithmic derivative of the h-function.

Implementation notes

Integrate the rule into the optimizer update for parameters that should retain previous-task knowledge. At the end of task t, copy each protected parameter into an anchor tensor mu, estimate an importance tensor s using the diagonal Fisher or an online-EWC running sum, and set a positive barrier-width tensor b. A practical initialization is b_i=clip(c/(sqrt(s_i)+epsilon), b_min, b_max), with c selected on a validation stream. For each minibatch, compute the current gradient g, choose step size eta, and sample epsilon from a standard normal distribution. For every protected parameter, calculate z=clip((w-mu)/b, -1+delta, 1-delta), then calculate the Doob correction r=-(sigma^2*pi/(2*b))*tan(pi*z/2). Apply either the direct Euler-Maruyama update w=w+eta*(-g-s*(w-mu)+r)+sqrt(eta)*sigma*epsilon, or add eta*r and the stochastic term to an AdamW update while keeping the baseline optimizer identical across comparisons. The clipping delta, for example 1e-4, avoids floating-point overflow near the barriers. Track the maximum absolute normalized displacement |z|, the percentage of coordinates with |z| above 0.95, current-task accuracy, and prior-task accuracy. Reduce eta or sigma if discrete updates frequently jump across the interval. The paper-derived quantities are the cosine ground state, its logarithmic derivative, and the sigma-squared scaling; s, b, sigma, and the discrete-step stabilization policy are empirical choices. First run a small MLP on rehearsal-free Split-MNIST, comparing SGD, noisy SGD, EWC, OU-style quadratic anchoring, and the Doob rule at equal injected noise. Sweep sigma over {0, 0.005, 0.01, 0.02, 0.04, 0.08} and sweep barrier widths. The key success signal is an inverted-U prior-task retention curve with a significant interior optimum, improved retention over matched EWC at equal current-task accuracy, and fewer large parameter excursions. A failure is monotone degradation with sigma or no advantage over the matched unconditioned noisy anchor.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a NumPy Doob-barrier diffusion module and matched noisy OU toy comparison. The core mechanism manifested quantitatively: sigma-squared stiffness scaling relative error was 2.85e-11, inverse-barrier-width-squared scaling error was 0, tangent boundary ratios matched to 1.84e-16, and the numerical log-h derivative error was 1.72e-10. In the toy task, Doob exactly matched OU at sigma=0 and constrained normalized displacement to approximately 1.0 for nonzero noise, but it did not show the paper's claimed inverted-U retention curve and sacrificed current-task plasticity, so no benchmark win is claimed.

Agent confidence
9/10
Baseline
Matched noisy OU anchoring: task-A loss [2.7834, 2.7535, 2.6076, 2.7056, 2.9411] and task-B loss [0.1162, 0.1162, 0.1122, 0.1296, 0.1376] for sigma [0, 0.03, 0.06, 0.1, 0.16]; max normalized displacement 2.42-2.73 at nonzero noise.
Idea
Doob steering: task-A loss [2.7834, 0.6529, 0.6712, 0.6427, 0.6339] and task-B loss [0.1162, 1.4368, 1.3053, 1.4564, 1.6592] for the same sigma sweep; max normalized displacement was 1.0 due to barrier protection. Formula prediction errors: sigma^2 2.85e-11, boundary ratio 1.84e-16, b^-2 0, log-h derivative 1.72e-10.

Limitations: Only a small one-dimensional-per-coordinate quadratic continual-learning toy was tested, with one seed and finite-step clipping; no MLP, Split-MNIST, Fisher estimation, multi-task stream, statistical significance, Adam/SGD comparison, or hardware-faithful noise model was evaluated.

How to run: python3 doob_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.06924)

Significantly WORSE than the tuned baseline (+60.7%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark
Tabular regression (Friedman)
Model
mlp_tiny
Paired seeds
8
Baseline mean
6.7742
Idea mean
10.8888
Effect (Δ)
4.1146 (+60.7%; negative = idea better)
Wins
0 / 8 paired seeds
p-value
0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect
n/a
Mechanism
Confirmed ✓
Practical effect
Hurts
Baseline tuning
swept over 0 configs
Limitations:

Only the tabular Friedman#1 track with mlp_tiny, 1200 training samples, 400 test samples, 20 epochs, and three Doob settings was tested. No continual multi-task stream, Fisher importance estimation, AdamW variant, or other tracks/models were tested.

How to run: python3 run_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark latest Failed ✗

Agent confidence: 10/10

Implemented Doob barrier consolidation as an Euler-Maruyama optimizer update on the structurally appropriate tabular Friedman#1 track with matched mlp_tiny architecture. The predicted near-anchor restoring stiffness was confirmed with relative error 4.42e-08, but the idea significantly worsened test MSE: paired delta +4.1146, permutation p=0.0081, and zero wins across eight paired seeds. No benchmark win was observed.

Baseline
Tuned SGD, lr=0.01, weight_decay=0.0001; mean test MSE 6.7742, std 0.3514.
Idea
Best Doob setting, lr=0.01, weight_decay=0.0001, sigma=0.002, b=0.5, s=0.01; mean test MSE 10.8888, std 3.8150; paired delta +4.1146, p=0.0081, 0/8 wins.
Mechanism signature
Predicted stiffness 6.16850275e-05; observed 6.16850302e-05; relative error 4.42e-08; confirmed true.

Benchmark result

Significantly WORSE than the tuned baseline (+60.7%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark
Tabular regression (Friedman)
Model
mlp_tiny
Paired seeds
8
Baseline mean
6.7742
Idea mean
10.8888
Effect (Δ)
4.1146 (+60.7%; negative = idea better)
Wins
0 / 8 paired seeds
p-value
0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect
n/a
Mechanism
Confirmed ✓
Practical effect
Hurts
Baseline tuning
swept over 0 configs
Limitations:

Only the tabular Friedman#1 track with mlp_tiny, 1200 training samples, 400 test samples, 20 epochs, and three Doob settings was tested. No continual multi-task stream, Fisher importance estimation, AdamW variant, or other tracks/models were tested.

How to run: python3 run_bench.py

Artifacts

Implementation overview ⬇ Download all as ZIP 6 files · code, reports and structured results