# Capitalization-Efficiency Monitor

- ID: 2206
- Canonical URL: https://synthcore.org/idea/2206/capitalization-efficiency-monitor
- API JSON: https://synthcore.org/api/idea/2206.json
- API Markdown: https://synthcore.org/api/idea/2206.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.12791](https://arxiv.org/abs/2608.12791)
- Category: optimization
- Solves: generalization, stability, sample-efficiency
- ML areas: optimizer, scheduler, training-dynamics, regularization
- Math tags: information-theory, statistical-mechanics, optimization, probability
- Ratings: usefulness 7/10; difficulty 6/10; novelty 6/10

## Idea description

Monitor learning as the ratio of future-task value gained to information irreversibly acquired by an update, rather than treating every reduction in training loss as equally productive. Penalize updates that absorb substantial data-specific information without increasing deletion-counterfactual value, and use the ratio to stop, trust-region, or schedule updates. This creates a falsifiable diagnostic for overfitting without assuming that overfitting and low efficiency are monotonically related.

## Mathematical statement

For an update $M\to M'$ with capital-value change $\Delta V=V(M';T,b)-V(M;T,b)$, temperature-like energy scale $kT$, and update-side information/search ledger $\sigma_M$ measured in nats, the paper defines capitalization efficiency as $\eta_{\mathrm{cap}}=\Delta V/(kT\sigma_M)$. Under condition (f), meaning no discarded record correlation, and for an $M$-local update, the ledger is $\sigma_M=\Delta^+J_D+\Sigma_{\mathrm{total}}\ge0$, where $\Delta^+J_D=I(M';D\mid M)$ is gross acquired record information and $\Sigma_{\mathrm{total}}\ge0$ is total entropy production; the stated cap is $\eta_{\mathrm{cap}}\le1$. For neural networks, set $kT=1$ and estimate the ledger with a posterior-to-prior KL plus an optimizer-noise dissipation surrogate.

## Key formulas

- $$\eta_{\mathrm{cap}}=\frac{\Delta V}{kT\,\sigma_M},\qquad \Delta V=V(M';T,b)-V(M;T,b),$$
- $$\Delta^+J_D=I(M';D\mid M),\qquad \sigma_M=\Delta^+J_D+\Sigma_{\mathrm{total}}\ge0,$$
- $$\eta_{\mathrm{cap}}\le1\quad\text{when condition (f) holds and }\sigma_M>0,$$
- $$\widehat\eta_t=\frac{\widehat V(M_{t+1})-\widehat V(M_t)}{\widehat\sigma_t+\epsilon},\qquad \widehat\sigma_t\approx\mathrm{KL}\!\left(q_{t+1}(\theta)\,\|\,q_t(\theta)\right)+\widehat\Sigma_t.$$

## Implementation notes

Attach the monitor to a Bayesian, Laplace, or stochastic-weight neural-network training loop. Maintain a diagonal Gaussian posterior $q_t(\theta)=\mathcal N(\mu_t,\operatorname{diag}(s_t^2))$ around current weights, with $q_t$ serving as the previous checkpoint prior. At update $t$, compute `info_t = KL(q_{t+1} || q_t)`; estimate optimizer dissipation with `Sigma_t = eta^2 * trace(H_t C_t)`, where $H_t$ is the minibatch-loss Hessian and $C_t$ is the gradient-noise covariance, using Hutchinson Hessian-vector products and minibatch covariance estimates. In the first MVP, omit $\Sigma_t$ and use KL alone. Every few hundred updates, estimate $V_hat_t$ on a small held-out task buffer using paired informed and blind evaluations. Compute `eta_hat_t = (V_hat_t - V_hat_prev)/(info_t + Sigma_t + epsilon)`. Reduce the learning rate or stop an update block when this ratio stays below a threshold; permit larger steps when value gain per acquired information is high. The first cheap experiment is a two-layer MLP on Split-MNIST, comparing SGD, Adam, and monitor-controlled Adam at equal compute. Prediction: in the no-discarded-correlation regime, positive updates should lie below an empirical upper envelope near `eta_hat = 1` after consistent normalization, while memorization-heavy late training shows increasing KL but near-zero or negative deletion-gap increments, causing `eta_hat` to approach zero or become negative. Add random nuisance features to test the predicted separation between information acquisition and operational value.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: inconclusive
- Verdict: Built a capitalization-efficiency monitor MVP using posterior-to-prior KL as the information ledger and held-out value gain as the numerator. The exact toy sweep confirmed the quantitative predictions: KL slope 2.000 vs 2 predicted, small-step value slope 0.999 vs 1, efficiency slope -1.001 vs -1, and nuisance efficiency -1.600 vs -1.6; the value transition was bracketed at [1.518, 1.800] around the predicted 1.7. In the neural sanity experiment, monitored Adam achieved lower test MSE and slightly lower total KL than baseline, but this is only a small supporting signal rather than proof of general benefit.

### Mechanism check

- Verdict: Built a capitalization-efficiency monitor MVP using posterior-to-prior KL as the information ledger and held-out value gain as the numerator. The exact toy sweep confirmed the quantitative predictions: KL slope 2.000 vs 2 predicted, small-step value slope 0.999 vs 1, efficiency slope -1.001 vs -1, and nuisance efficiency -1.600 vs -1.6; the value transition was bracketed at [1.518, 1.800] around the predicted 1.7. In the neural sanity experiment, monitored Adam achieved lower test MSE and slightly lower total KL than baseline, but this is only a small supporting signal rather than proof of general benefit.
- Confidence: 8/10
- Limitations: The MVP omits optimizer-noise dissipation Sigma, uses fixed posterior variance and squared parameter displacement as the KL approximation, and uses direct held-out regression value rather than paired deletion-counterfactual task evaluations. Only one tiny seeded dataset and one neural architecture were tested; the monitor's trust-region behavior was not evaluated across multiple seeds or compute-matched benchmarks.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.1856
- Paired wins: 1/8
- Benchmark verdict: no significant win

## Artifacts

- [README.md](https://synthcore.org/code/877/README.md)
- [bench_report.json](https://synthcore.org/code/877/bench_report.json)
- [bench_run.py](https://synthcore.org/code/877/bench_run.py)
- [experiment.py](https://synthcore.org/code/877/experiment.py)
- [report.md](https://synthcore.org/code/877/report.md)
- [report_bench_2026-09-01T132215.md](https://synthcore.org/code/877/report_bench_2026-09-01T132215.md)
- [results.json](https://synthcore.org/code/877/results.json)
- [Download all files as ZIP](https://synthcore.org/download/877)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.