# Information-aware finite-horizon optimizer

- ID: 1244
- Canonical URL: https://synthcore.org/idea/1244/information-aware-finite-horizon-optimizer
- API JSON: https://synthcore.org/api/idea/1244.json
- API Markdown: https://synthcore.org/api/idea/1244.md
- Verification status: unverified
- Source: [arXiv:2607.21523](https://arxiv.org/abs/2607.21523)
- Category: dynamics
- Solves: stability, speedup, sample-efficiency
- ML areas: optimizer, training-dynamics, scheduler
- Math tags: control-theory, dynamical-systems, stochastic-processes, optimization, information-theory
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Replace a fixed learning-rate schedule by a finite-horizon feedback controller whose action depends on a noisy estimate of the current optimization state and its uncertainty. The controller takes larger corrective steps when uncertainty is informative, but increasingly enforces the endpoint as the horizon closes, while charging an explicit cost for every intervention.

## Mathematical statement

The paper's equilibration assumption collapses the decision state to a current distribution rather than the complete trajectory. For an optimizer, define a belief state $b_n=(d_n,\Sigma_n)$, where $d_n=\|\theta_n-\theta_\star\|$ is distance from a target and $\Sigma_n$ is a diagonal estimate of gradient uncertainty. Let $a_n$ be a candidate update, $T(b_n,a_n,o_n)$ the state update after noisy observation $o_n$, $c_{\mathrm{transport}}$ the cost of moving toward the target, $c_{\mathrm{info}}$ the benefit of reducing uncertainty, and $\kappa>0$ the fixed cost of an intervention. The finite-horizon Bellman recursion selects the action minimizing immediate transport cost minus information benefit, intervention cost, and expected future value. A further update is justified only when its expected value reduction exceeds $\kappa$. The exact equilibrium distribution in the paper is physical, while the Gaussian belief-state approximation is the proposed ML adaptation.

## Key formulas

- $$p^{\mathrm{eq}}_{H_n}(x)=\frac{\exp[-\beta H_n(x)]}{Z[H_n]},\qquad Z[H_n]=\int \exp[-\beta H_n(x)]\,dx.$$
- $$V_n(b)=\min_{a\in\mathcal{A}}\left[c_{\mathrm{transport}}(b,a)-c_{\mathrm{info}}(b,a)+\kappa+\mathbb{E}_{o\sim p(o\mid b,a)}V_{n+1}\!\left(T(b,a,o)\right)\right],\qquad V_N(b)=\Phi(b).$$
- $$c_{\mathrm{info}}(b,a)=\lambda_I\left[\log\det\Sigma_n-\mathbb{E}_{o}\log\det\Sigma_{n+1}\right],\qquad c_{\mathrm{transport}}(b,a)=\frac{\|\theta_n+a-\theta_\star\|^2-\|\theta_n-\theta_\star\|^2}{2\tau^2}.$$
- $$\text{continue at stage }n\Longleftrightarrow\max_{a\in\mathcal{A}}\mathbb{E}_{o}\left[V_{n+1}(b_n)-V_{n+1}(T(b_n,a,o))\right]>\kappa.$$

## Implementation notes

1. Integration point: wrap the controller around SGD at the minibatch-update level. Let $\theta_n$ be network parameters, $g_n$ the minibatch gradient, $\mu_n$ an exponential moving average of gradients, and $\Sigma_n$ a diagonal exponential moving average of $(g_n-\mu_n)^2$. Use a short macro-horizon of $N$ updates and a target checkpoint $\theta_\star$, such as the best validation checkpoint or a scheduled endpoint. Candidate actions are step sizes $\eta$ from a small discrete set, with $a_n=-\eta\mu_n$. 2. Pseudocode: construct a grid over normalized distance $d_n$ and uncertainty $u_n=\operatorname{mean}(\Sigma_n)$. Set terminal values $V_N(d,u)=\Phi(d)$, for example $\Phi(d)=d^2$. Perform backward dynamic programming over grid states and candidate step sizes. For each candidate, estimate the next state using $\theta' = \theta-\eta\mu$, $d'=\|\theta'-\theta_\star\|$, and the EMA covariance update; evaluate transport cost, information gain $\lambda_I[\log\det\Sigma-\log\det\Sigma']$, fixed cost $\kappa$, and future value. During training, quantize the observed $(d_n,u_n)$ and apply the stored minimizing $\eta$. 3. Computed versus estimated: the equilibration-to-belief-state reduction, Bellman structure, and intervention threshold are transferred from the paper; gradient moments, transition noise, $\tau$, $\lambda_I$, and $\kappa$ are empirical hyperparameters. 4. First cheap experiment: train a two-layer MLP on MNIST with SGD, comparing fixed learning rate and cosine decay against this controller at equal minibatch budgets. Use $N=20$ stages, 8 candidate learning rates, and a 16-by-16 state grid. Sweep $\kappa$. The falsifiable prediction is a continuation boundary: the controller should stop or reduce interventions when measured expected loss reduction per update falls below $\kappa$, and the selected update count should decrease monotonically with $\kappa$. High-variance early states should receive larger steps than low-variance late states. The empirical continuation boundary should predict held-out one-step loss reduction within 20%.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.