# Passivity-Governed Momentum

- ID: 1846
- Canonical URL: https://synthcore.org/idea/1846/passivity-governed-momentum
- API JSON: https://synthcore.org/api/idea/1846.json
- API Markdown: https://synthcore.org/api/idea/1846.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.05492](https://arxiv.org/abs/2608.05492)
- Category: dynamics
- Solves: stability, speedup
- ML areas: optimizer, training-dynamics, scheduler
- Math tags: control-theory, dynamical-systems, optimization, linear-algebra
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Add an explicit gradient feedthrough D to a momentum optimizer and choose it below the estimated inverse smoothness, D < 1/L. Use the resulting passivity margin to govern momentum: increase the momentum-channel gain only while the measured storage dissipation remains nonnegative, and reduce the feedthrough or momentum when the passivity residual becomes positive.

## Mathematical statement

Let x_k be the parameter vector, x_star a stationary point, and g_k = gradient f(x_k). The paper assumes the sector inequality inner product(x - x_star, gradient f(x)) >= [mL/(m+L)] norm(x - x_star)^2 + [1/(m+L)] norm(gradient f(x))^2. For the non-strongly-convex case m = 0, define the loop-transformed input bar_u = u - D y, where u = x - x_star and y = gradient f(x). Then inner product(bar_u, y) >= (1/L - D) norm(y)^2. Therefore the gradient block is output strictly passive with margin epsilon = 1/L - D whenever D < 1/L. Implement momentum with v_(k+1) = beta_k v_k + (1 - beta_k) g_k and x_(k+1) = x_k - D_k g_k - alpha_k v_(k+1). A practical controller storage is S_k = norm(v_k)^2/2. The measured passivity residual is r_k = S_(k+1) - S_k - inner product(g_k, x_k - x_(k+1)); negative r_k indicates dissipation, while persistent positive r_k indicates loss of the certificate.

## Key formulas

- $$\left\langle x-x^{\ast},\nabla f(x)\right\rangle\geq\frac{mL}{m+L}\left\|x-x^{\ast}\right\|_2^2+\frac{1}{m+L}\left\|\nabla f(x)\right\|_2^2$$
- $$\bar u=u-Dy,\qquad \left\langle\bar u,y\right\rangle=\left\langle u,y\right\rangle-D\|y\|_2^2\geq\left(\frac{1}{L}-D\right)\|y\|_2^2$$
- $$v_{k+1}=\beta_kv_k+(1-\beta_k)g_k,\qquad x_{k+1}=x_k-D_kg_k-\alpha_kv_{k+1},\qquad g_k=\nabla f(x_k)$$
- $$S_k=\frac{1}{2}\|v_k\|_2^2,\qquad r_k=S_{k+1}-S_k-\left\langle g_k,x_k-x_{k+1}\right\rangle$$

## Implementation notes

First, replace SGD or classical momentum in the training loop. Keep the minibatch gradient g_k, but split the update into a direct feedthrough D_k g_k and a momentum channel alpha_k v_(k+1). Estimate a conservative local smoothness using recent secants: L_hat_k = max over j in a window of norm(g_j - g_(j-1)) divided by (norm(x_j - x_(j-1)) + 10^(-12)). Initialize D_k = (1 - delta) divided by max(L_hat_k, L_min), with delta between 0.1 and 0.3. The pseudocode is: compute g_k; update v_(k+1) = beta_k v_k + (1 - beta_k) g_k; propose x'_(k+1) = x_k - D_k g_k - alpha_k v_(k+1); calculate S_(k+1) = norm(v_(k+1)) squared divided by 2 and r_k = S_(k+1) - S_k - inner product(g_k, x_k - x'_(k+1)); then accept the step. If an exponential average of r_k exceeds zero for p consecutive steps, set D_(k+1) = 0.8 D_k, alpha_(k+1) = 0.8 alpha_k, and beta_(k+1) = min(beta_k, 0.9 beta_k). If the average stays below a negative threshold tau, increase alpha by 1 percent up to a fixed cap. The paper supplies the sector inequality and hard ceiling D < 1/L; the implementation estimates local L and the residual under minibatch noise. First test on a quadratic f(x) = x transpose Q x divided by 2 with known L equal to the largest eigenvalue of Q, then on a two-layer MLP trained on MNIST. Compare SGD, fixed momentum, and the governor over a logarithmic D sweep. The quantitative prediction is a stability transition near D_c = 1/L: below this point the transformed supply should remain at least (1/L - D) norm(g_k) squared, while violation frequency and divergence should increase sharply above it. On the quadratic, the measured threshold should agree with 1/L within 20 percent, and the governor should maintain a negative long-run residual on the MLP.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: no_effect
- Verdict: Built and ran a reproducible NumPy implementation of the passivity margin, residual-based governor, and quadratic optimizer comparison in passivity_experiment.py. The mechanism manifested strongly: observed supply boundaries exactly matched D_c=1/L for L=1,2,5,10, violations were absent below the boundary and appeared in the top-curvature direction above it, and the measured margin matched (1/L-D)||g||². The governor reached final quadratic loss 2.07e-21 with a negative final-100-step residual mean of -4.76e-16, but this is not evidence of a broad training win because no MLP/MNIST experiment was completed.

### Mechanism check

- Verdict: Built and ran a reproducible NumPy implementation of the passivity margin, residual-based governor, and quadratic optimizer comparison in passivity_experiment.py. The mechanism manifested strongly: observed supply boundaries exactly matched D_c=1/L for L=1,2,5,10, violations were absent below the boundary and appeared in the top-curvature direction above it, and the measured margin matched (1/L-D)||g||². The governor reached final quadratic loss 2.07e-21 with a negative final-100-step residual mean of -4.76e-16, but this is not evidence of a broad training win because no MLP/MNIST experiment was completed.
- Confidence: 8/10
- Limitations: Only deterministic diagonal convex quadratics were tested; no two-layer MLP, MNIST, minibatch noise, secant-estimation stress test, matched-FLOP logarithmic D sweep, or GPU implementation was tested. The residual governor is a compact MVP.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/684/bench_report.json)
- [passivity_experiment.py](https://synthcore.org/code/684/passivity_experiment.py)
- [report.md](https://synthcore.org/code/684/report.md)
- [report_bench_2026-08-31T214510.md](https://synthcore.org/code/684/report_bench_2026-08-31T214510.md)
- [results.json](https://synthcore.org/code/684/results.json)
- [stage2_bench.py](https://synthcore.org/code/684/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/684)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.