✗ Failed on benchmark 2026

Damkohler-Controlled Optimizer

Implementation & benchmark of arXiv:2607.09416 — The Statistical physics of unsaturated soil water: kinetic theory and non commutative pore water dynamics

Usefulness7/10

Difficulty5/10

Novelty7/10

Dynamics Stability Speedup Optimizer Scheduler Training dynamics Dynamical systems Control theory Statistical mechanics Optimization

Source paper: The Statistical physics of unsaturated soil water: kinetic theory and non commutative pore water dynamics arXiv:2607.09416 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Augment an optimizer with a measurable redistribution time for its internal state and compare it with the time scale of the changing gradient field. Use the resulting Damkohler number to interpolate between a fast quasi-static preconditioner and a history-preserving, non-equilibrium update, rather than applying one optimizer regime throughout training.

Formulas

$$\mathrm{Da}_k=\frac{\tau_{{\rm redis},k}}{\tau_{{\rm forcing},k}},\qquad \tau_{{\rm redis},k}=-\frac{1}{\log(1-\alpha_k)}$$

$$\tau_{{\rm forcing},k}=\frac{\|s_{t,k}\|}{\|s_{t,k}-s_{t-1,k}\|+\varepsilon},\qquad w_k=\frac{\mathrm{Da}_k}{1+\mathrm{Da}_k}$$

$$\theta_{t+1,k}=\theta_{t,k}-\eta\left[(1-w_k)P^{\rm eq}_{t,k}g_{t,k}+w_kP^{\rm neq}_{t,k}g_{t,k}\right]$$

Mathematical statement

The paper defines $\mathrm{Da}(r,\mathbf{x})=\tau_{\rm redis}(r)/\tau_{\rm forcing}(\mathbf{x})$, where $\tau_{\rm redis}$ is the relaxation time of pore occupancy and $\tau_{\rm forcing}$ is the time scale of external forcing. Transfer this to training by defining $\tau_{\rm redis}$ as the exponential relaxation time of optimizer-state bins and $\tau_{\rm forcing}$ as the time over which the gradient or loss curvature changes. For $\mathrm{Da}\ll1$, optimizer state is approximately equilibrated and a quasi-static update is appropriate; for $\mathrm{Da}\gtrsim1$, state lag is substantial and the optimizer should preserve distributional memory. The control law is a smooth interpolation controlled by $w=\mathrm{Da}/(1+\mathrm{Da})$.

Implementation notes

Integrate this into AdamW or a diagonal natural-gradient optimizer, separately for parameter blocks or logarithmic bins of parameter magnitude. Maintain an optimizer-state vector $s_t$, such as the first-moment estimate, and optionally a second state $q_t$ representing variance or curvature. Estimate the redistribution coefficient $\alpha_k$ from the observed relaxation of block state $s_k$: fit $s_{t+1,k}-s^*_k\approx(1-\alpha_k)(s_{t,k}-s^*_k)$ with an exponential moving average, clipping $\alpha_k$ to $[10^{-4},0.99]$. Compute the forcing time from the relative change in state, gradient norm, or diagonal curvature. Pseudocode is: compute gradient $g_t$; update $s_t,q_t$; estimate $\alpha_k$ and $\tau_{{\rm forcing},k}$; calculate $\mathrm{Da}_k$ and $w_k$; form $P^{\rm eq}$ from bias-corrected Adam; form $P^{\rm neq}$ from a lag-preserving update using previous $s,q$; mix both updates with $w_k$; apply decoupled weight decay. The theory supplies the time-scale ratio and regime interpretation, while relaxation and forcing times are empirically estimated. First run CIFAR-10 with a 3-layer MLP against AdamW at identical learning rates, sweeping learning rate and gradient-noise level. The falsifiable prediction is a regime transition near $\mathrm{Da}=1$: below 0.3 the method should match the quasi-static branch, while above 1 it should reduce oscillations or loss spikes. The measured transition in update variance should occur within roughly 20 percent of $\mathrm{Da}=1$.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 7/10

Built a NumPy Damköhler-controlled AdamW prototype with relaxation/forcing estimates and smooth equilibrium/non-equilibrium interpolation. The mechanism checks passed: exponential relaxation reached approximately 1/e at the predicted tau_redis, the Da=1 transition matched the predicted forcing frequency across alpha values, and w was monotone with w(1)=0.5. The optimizer itself did not improve loss on the nonstationary quadratic task; it was slightly worse than AdamW, while update variability was reduced only in the fastest regimes.

Agent confidence: 7/10

Baseline: AdamW final losses for forcing periods 10, 20, 50, 100: 0.2549, 0.2704, 0.3342, 0.3105; update standard deviations: 0.00693, 0.00762, 0.00600, 0.00539.
Idea: Damköhler optimizer final losses: 0.2567, 0.2738, 0.3452, 0.3286; update standard deviations: 0.00668, 0.00745, 0.00622, 0.00609. Relative final-loss differences versus AdamW were +0.0018, +0.0034, +0.0110, +0.0182. Observed/predicted Da=1 transition ratios were 1.000 across alpha=0.02–0.40; relaxation ratios near tau were 0.372, 0.377, 0.387, 0.410, 0.360, 0.300 versus target 0.368.

Limitations: No CIFAR-10/MLP experiment, learning-rate sweep, repeated-seed statistical test, GPU implementation, blockwise state fitting, or validated distributional non-equilibrium branch was tested.

How to run: python3 damkohler_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.09416) Sep 2, 2026

Significantly WORSE than the tuned baseline (+293.1%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0013
Idea mean: 0.0051
Effect (Δ): 0.0038 (+293.1%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±196.7%
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the built-in dynamics track was tested, with 800 training samples, 300 test samples, 8 epochs, and three shared learning rates. The baseline sweep selected using four seeds and was then re-evaluated across all eight seeds; no larger models, longer training, alternative beta settings, weight decay, or other tracks were tested. The generated mechanism signature is model-training-derived but does not confirm the predicted Da-to-w relationship.

How to run: python3 stage2_damkohler_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented Damköhler-controlled AdamW as a local training-loop intervention on the structurally matched dynamics track using the shared rnn_small architecture. The idea's best configuration had test MSE 0.0050747 versus the tuned AdamW baseline at 0.0012908; paired delta was +0.0037839 with permutation p=0.0081, so the idea was significantly worse. The measured trained-model signature also failed the quantitative control-law retest: observed mean Da=3.9111, observed mean w=0.5960, versus w(Da)=0.7964.

Baseline: AdamW tuned lr=0.006; full 8-seed mean MSE 0.0012908305, std 0.0005098155
Idea: Damköhler AdamW best lr=0.0015; full 8-seed mean MSE 0.0050747443, std 0.0027867023; paired delta +0.0037839139; p=0.0081

Benchmark result