Thermodynamic Confidence Controller for SGD
Implementation & benchmark of arXiv:2609.04162 — Thermodynamic Concentration Inequalities: Controlling Uncertainty in Finite-Time and Small-Sample Thermodynamic Inference
Source paper: Thermodynamic Concentration Inequalities: Controlling Uncertainty in Finite-Time and Small-Sample Thermodynamic Inference arXiv:2609.04162 ⓘ · analyzed Sep 4, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Treat a scalar projection of the stochastic training trajectory as a generalized current and use a finite-time concentration bound to decide when its mean estimate is reliable. Increase batch size, reduce the learning rate, or stop collecting samples when the bound predicts that the probability of a misleading gradient estimate is below a target confidence level.
Formulas
Mathematical statement
Let \(\theta_t\) denote parameters evolving under stochastic optimization, let \(J(\theta,\xi)\) be a scalar current computed from a minibatch \(\xi\), and let \(\overline J_T=T^{-1}\sum_{k=0}^{T-1}J_k\). For a geometrically ergodic effective diffusion with invariant law \(\mu\), initial law \(\nu\), stationary mean \(m_J=\mathbb E^\mu[J]\), and deviation \(a\geq 0\), the paper gives the tail form \(\mathbb P^\nu(|\overline J_T-m_J|\geq a)\leq 2N_\nu\exp[-T\mathcal I^J(a)]\). Here \(N_\nu\) accounts for initialization and \(\mathcal I^J(a)\) is the concentration rate determined by the relaxation or spectral-gap scale \(\lambda_{\mathrm{gap}}\), locally observed entropy-production rate \(\Sigma^U\), and local fluctuation statistics of \(J\). Define the confidence radius \(a_\delta(T)=\inf\{a:2N_\nu e^{-T\mathcal I^J(a)}\leq\delta\}\). The controller changes training only when this radius is large relative to the estimated current mean.
Implementation notes
1. Integration point: add a monitoring wrapper around SGD or Adam after every optimizer step. Choose \(v\) as a fixed random unit direction, the normalized gradient at the beginning of a monitoring window, or the leading principal direction of recent updates. Record the scalar current \(J_k=v^Tg_k\) and optionally the update current \(J_k=v^T(\theta_{k+1}-\theta_k)/\eta_k\). Maintain a rolling window of length \(T\).
2. Pseudocode: compute the minibatch gradient \(g_k\), update the optimizer, and append \(J_k\). Estimate the local mean, variance, and integrated autocorrelation time from the window. Set \(\widehat\lambda_{gap}=1/\max(1,\widehat\tau_{corr})\). Estimate local dissipation using a scalar proxy such as \(\widehat\Sigma^U=\max(0,\langle g_k^T(\theta_k-\theta_{k-1})/\eta_k\rangle)\), with a reversible-drift baseline subtracted if available. Evaluate or numerically approximate the paper's rate function \(\mathcal I^J(a;\widehat\lambda_{gap},\widehat\Sigma^U,\widehat{\mathrm{Var}}(J))\), then solve \(2N_\nu e^{-T\mathcal I^J(a)}=\delta\) for \(a_\delta\). If \(a_\delta>0.25|\widehat m_J|\), double the minibatch size or halve the learning rate. If it remains below \(0.1|\widehat m_J|\) for four windows, allow a 10 percent learning-rate increase.
3. Computed versus estimated: the exponential tail structure and its dependence on relaxation, dissipation, and local fluctuations come from the paper. The spectral gap, entropy-production proxy, current variance, rate-function parameters, and prefactor \(N_\nu\) must be estimated online. Use block bootstrap or independent reruns to calibrate \(N_\nu\) conservatively.
4. First cheap experiment: train a two-layer MLP on MNIST with SGD, comparing vanilla training and the controller at batch sizes 32, 128, and 512. For held-out windows, estimate \(-T^{-1}\log P(|\overline J_T-\widehat m_J|>a)\) versus \(a\). The quantitative prediction is an approximately exponential tail whose empirical slope follows the estimated \(\mathcal I^J(a)\), with failure probability below the target \(\delta\) up to a factor of two. The confidence radius should decrease approximately as \(T^{-1/2}\) near equilibrium and should become larger when the estimated autocorrelation time or local gradient fluctuations increase.
Verification
Stage 1 · Toy mechanism gate: Failed ✗
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built an online current monitor using the paper's sub-gamma rate function, confidence-radius inversion, autocorrelation estimate, and batch-size trigger. The mathematical sanity check reproduced the claimed T^-1/2 radius scaling exactly (slope -0.50), and quadrupling variance doubled the radius. However, at equal sample budget the controller reached 0.9133 test accuracy versus 0.9489 for vanilla SGD, with higher final loss (0.3978 vs 0.1171), so no training win was observed; the controller's estimates are heuristic proxies rather than a rigorous bound for SGD.
- Agent confidence
- 9/10
- Baseline
- Digits MLP, SGD batch 32, 1102 steps / 35,264 examples: final loss 0.1171, test accuracy 0.9489
- Idea
- Digits MLP, confidence controller, 300 steps / 35,264 examples, batches 32->64->128: final loss 0.3978, test accuracy 0.9133, first/last radius 0.00460/0.00178; synthetic radius log-log slope -0.50
Limitations: Only one fixed-seed digits-MLP comparison was run; no MNIST, multiple seeds, wall-clock or FLOP accounting, Adam comparison, learning-rate adaptation, bootstrap calibration, or empirical tail-coverage validation was performed. The implemented C and D estimates are heuristic adaptations and are not proven concentration constants for discrete SGD.
How to run: python3 thermo_controller.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- report.md 1.6 KB View
- results.json 8.3 KB View Raw
- results2.json 1.1 KB View Raw
- results_fair.json 1.1 KB View Raw
- thermo_controller.py 5.8 KB View Raw