△ Mechanism confirmed, baseline not beaten 2026

Distributional Gradient-Flow Memory

Implementation & benchmark of arXiv:2607.09416 — The Statistical physics of unsaturated soil water: kinetic theory and non commutative pore water dynamics

Usefulness7/10

Difficulty6/10

Novelty8/10

Dynamics Stability Generalization Accuracy Optimizer Training dynamics Regularization Dynamical systems Optimization Statistical mechanics Probability

Source paper: The Statistical physics of unsaturated soil water: kinetic theory and non commutative pore water dynamics arXiv:2607.09416 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace a single scalar optimizer memory per parameter block with a small occupancy distribution whose bins represent distinct relaxation or gradient-history regimes. Train this state using a conservative redistribution operator and an energy-decreasing correction, allowing the optimizer to represent non-equilibrium lag and hysteresis that cannot be captured by one momentum variable.

Formulas

$$\frac{\partial g}{\partial t}=\mathcal{C}[g]-\mathcal{E}[g]-\mathcal{T}[g],\qquad \frac{\partial g}{\partial t}=-\mathcal{M}(g)\frac{\delta \mathcal{F}}{\delta g}$$

$$\mathcal{F}[g]=\sum_{k,j}g_{k,j}\left(\log(g_{k,j}+\varepsilon)-1\right)+\lambda\sum_{k,j}g_{k,j}E_{k,j},\qquad \frac{d\mathcal{F}}{dt}=-\sum_{k,j}M_{k,j}\left(\frac{\partial\mathcal{F}}{\partial g_{k,j}}\right)^2\le0$$

$$g_{t+1}=\Pi_{\Delta}\left[g_t+\rho(C_t-I)g_t-\rho M_t\nabla_g\mathcal{F}(g_t)\right],\qquad \sum_jg_{k,j}=1$$

Mathematical statement

The paper treats $g(r,\mathbf{x},t)\in[0,1]$ as the irreducible state outside the quasi-static limit and writes its evolution as a balance between transport, redistribution, and source terms. Its kinetic dynamics are an Onsager gradient flow, $\partial_t g=-\mathcal{M}(g)\delta\mathcal{F}/\delta g$, with nonnegative mobility $\mathcal{M}$ and $d\mathcal{F}/dt\le0$ in the unforced case. Transfer this by storing occupancies $g_{k,j}$ over $J$ bins for parameter block $k$, with bin coordinate $r_j$ representing gradient magnitude or optimizer relaxation scale. A conservative Markov redistribution matrix $C$ preserves total occupancy, while the free-energy-like functional $F=\sum_{k,j}g_{k,j}(\log g_{k,j}-1)+\lambda\sum_{k,j}g_{k,j}E_{k,j}$ penalizes diffuse and high-cost states. The correction must satisfy $\Delta F\le0$ when gradients are held fixed.

Implementation notes

Use this as a plug-in optimizer state for a small number of parameter blocks rather than every scalar weight initially. For each block $k$, create $J=8$ or 16 nonnegative bins $g_k$, with logarithmically spaced coordinates $r_j$ representing gradient magnitude or estimated relaxation time. At each minibatch, compute the block gradient norm $u_k$ and softly deposit unit mass into nearby bins using a Gaussian kernel; this is the forcing/source term. Apply a nearest-neighbor redistribution matrix $C_k$ with nonnegative off-diagonal entries and columns summing to one, then apply the entropy-plus-energy gradient in the displayed formula. Project onto the probability simplex after every state update. Construct the parameter update as a bin-weighted sum of step sizes: $d_k=-\sum_jg_{k,j}\eta_j\widehat g_k$, where $\eta_j$ is larger for fast bins and smaller for slow bins. During a gradient-hold diagnostic, disable source deposition and verify that the free-energy functional decreases; damp or reject any state update violating this condition. The paper supplies the distribution-valued state, conservative kinetic balance, and Onsager Lyapunov condition; the matrix, energy values, and deposition kernel are empirical design choices. First test on a 2-layer MLP and CIFAR-10 against SGD with momentum, AdamW, and an 8-bin version without energy correction. Alternate large- and small-noise training phases. The predicted signature is hysteresis: after returning to the original forcing, effective step size depends on the preceding phase, and the small-cycle loop area should scale as $H\propto I^2$, where $I$ is forcing amplitude.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built an 8-bin distributional-memory optimizer with conservative nearest-neighbor redistribution, Gaussian gradient deposition, entropy-plus-energy correction, simplex projection, and energy-decrease rejection. Three toy predictions were confirmed: the local stability boundary rho*=2/J=0.25 matched the observed transition, equilibria matched softmax(-lambda E) with L1 error <=1.2e-5, and all 300 source-free correction trials decreased free energy. The optimizer did not produce a learning win on seeded digits: it reached 80.44% versus SGD 95.11% and AdamW 94.00%, so the mechanism is validated but the chosen training design is not competitive.

Agent confidence: 8/10

Baseline: SGD momentum: final train loss 0.0746, test accuracy 95.11%; AdamW: final train loss 0.1649, test accuracy 94.00%.
Idea: Distributional memory: final train loss 1.3363, test accuracy 80.44%. Toy stability: predicted rho*=0.25, stable at rho=0.24 and unstable at rho=0.26; equilibrium L1 error <=1.2e-5; Lyapunov violations 0/300, maximum measured free-energy change -0.00607.

Limitations: Only a small 2-layer MLP on sklearn digits was tested, not CIFAR-10 or multiple seeds. Hysteresis and the proposed loop-area scaling H proportional to forcing amplitude squared were not tested. The stability sweep tests the local source-free Euler correction, while the full optimizer includes forcing and empirical hyperparameters; no FLOP-normalized or wall-clock comparison was performed.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.09416) Sep 2, 2026

No significant difference vs. the tuned baseline (+4.5%, p=0.2047, wins 3 / 8 paired seeds).

Benchmark: Tabular regression (Friedman)
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 7.9907
Idea mean: 8.3503
Effect (Δ): 0.3596 (+4.5%; negative = idea better)
Wins: 3 / 8 paired seeds
p-value: 0.2047 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 0 configs

Limitations:

Only the structurally matched tabular track was tested, using mlp_tiny and 25 epochs. The proposed alternating large/small-noise hysteresis phases and loop-area scaling H proportional to I^2 were not tested. No wall-clock or FLOP-normalized comparison was performed; the baseline used the harness Adam path while the idea required a custom optimizer loop.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

Built the distributional-memory optimizer as a local modification of the tabular Friedman regression training loop, using the matched mlp_tiny architecture and 8-bin conservative redistribution plus energy correction. The full 8-seed benchmark found baseline MSE 7.9907 versus idea MSE 8.3503, paired delta +0.3596 with permutation p=0.2047, so there was no significant win. The trained-model source-free probe confirmed the predicted non-increasing free-energy behavior, but this mechanism did not translate into better task performance.

Baseline: Adam via bench.train_model, best lr=0.006 and weight_decay=0.0001; 8-seed test MSE mean 7.9907144904, std 0.6128875038.
Idea: Distributional memory, best shared lr=0.006 with rho=0.08, source=0.10, lambda=0.08; 8-seed test MSE mean 8.3503233194, std 1.1372111740; paired delta +0.3596088290, p=0.2047, 3/8 paired wins.
Mechanism signature: Predicted source-free energy-decrease fraction 1.0; observed fraction on a trained benchmark model 1.0; confirmed=true.

Benchmark result