Entropy-Gap Optimizer Switch
Implementation & benchmark of arXiv:2607.11669 — Still life in a classic Blume-Capel model: pseudo-transitions in a spin-1 diamond chain
Source paper: Still life in a classic Blume-Capel model: pseudo-transitions in a spin-1 diamond chain arXiv:2607.11669 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Model locally competing neural-network parameter basins as low-energy states with different effective multiplicities, and inject calibrated parameter noise to measure when the optimizer begins switching between them. Use the resulting pseudo-transition peak as a principled trigger for changing learning rate, noise, or regularization rather than relying on a fixed epoch schedule.
Formulas
Mathematical statement
Consider two competing state families with energies E0 < E1, degeneracies g0 and g1, gap ΔE = E1 − E0 > 0, and inverse temperature β = 1/T. Their partition function is Z = g0 exp(−βE0) + g1 exp(−βE1). Equal statistical weight occurs at T* = ΔE / ln(g1/g0) when g1 > g0, so a small gap combined with large multiplicity generates a sharp crossover. For a neural network, replace energy by loss L, temperature by perturbation-noise scale T, and degeneracy by the empirical number or volume of perturbations mapping to a basin. If an observable takes values A0 and A1 in the two basins, its variance is Var(A) = p1(1 − p1)(A1 − A0)^2 and is maximal at p1 = 1/2. This finite variance peak is the measurable pseudo-transition signal.
Implementation notes
1. Integration point: add a low-frequency diagnostic around an existing SGD or Adam training loop. Every K optimizer steps, freeze the current weights θ, sample M Gaussian perturbations εm with the same shape as θ, and evaluate the loss, accuracy, gradient norm, or another observable at θ + √(2T) εm for a logarithmic grid of noise temperatures T. Do not backpropagate through these probes. 2. Pseudocode: for every checkpoint and every T, compute Am = A(θ + √(2T) εm), calculate the sample variance χ[T], and select the temperature maximizing χ. Estimate a local two-basin gap by clustering probe losses into two modes; define ΔL as the difference between cluster means and estimate the multiplicity ratio using cluster counts. This gives the predicted crossover Tpred = ΔL / ln(n1/n0). If the measured peak is reproducible across adjacent checkpoints, reduce the learning rate or increase weight decay after the peak. Alternatively, retain noise if the high-loss basin still dominates. 3. Taken from the paper: the two-state partition function, entropy-gap crossover, and variance-peak criterion. Estimated empirically: basin losses, multiplicities, and response curves. 4. First cheap experiment: train a two-layer MLP on MNIST with Adam, comparing this controller against fixed cosine decay and constant Adam, using M = 32, K = 100, and eight T values. The falsifiable prediction is a finite response peak near Tpred, with measured and predicted peak temperatures agreeing within approximately a factor of two when two clusters are present. The peak should become sharper as ΔL decreases or the multiplicity ratio increases. If no reproducible peak appears across seeds, disable the controller.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.