✗ Failed on benchmark 2026

Saturation-Persistence Trust Region

Implementation & benchmark of arXiv:2608.04740 — Analytical Prediction of Voltage Collapse in Current-Limited Grid-Forming Inverters

Usefulness7/10

Difficulty6/10

Novelty8/10

Training Stability Generalization Optimizer Scheduler Training dynamics Control theory Dynamical systems Bifurcations Optimization

Source paper: Analytical Prediction of Voltage Collapse in Current-Limited Grid-Forming Inverters arXiv:2608.04740 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the distinction between persistent saturated equilibria and immediate equilibrium loss to adapt the clipping threshold or learning rate. Increase the allowable update only when saturation is locally persistent and attracting; reduce it when saturation produces a nonpositive branch slope, a shrinking stability margin, or a sharp increase in clipped residual variance.

Formulas

$$e(\theta;C)=\|\nabla_\theta L(\theta)\|_2-C,\qquad \theta_{t+1}=\theta_t-\eta\,\tilde g(\theta_t;C).$$

$$\hat s_\eta=\frac{e(\bar\theta_{+};C)-e(\bar\theta_{-};C)}{2\delta_\eta},\qquad \hat s_C=\frac{e(\bar\theta;C+\delta_C)-e(\bar\theta;C-\delta_C)}{2\delta_C},$$

$$\eta_{k+1}=\begin{cases}\min\{\eta_k(1+\alpha),\eta_{\max}\},&m_k>m_{\min}\ \wedge\ \hat s_\eta>0,\\\eta_k(1-\beta),&\hat s_\eta\le0\ \vee\ \operatorname{Var}(L)_{k-W:k}>V_{\max},\\\eta_k,&\text{otherwise}.\end{cases}$$

Mathematical statement

For a parameter update $\dot\theta=-\eta\tilde g(\theta)$, define the saturation residual $e(\theta)=\|g(\theta)\|_2-C$. The clipped region is $e>0$. Let $\theta_C^*(\eta,C)$ denote a locally averaged stationary state under clipping, obtained from $\mathbb E[\Delta\theta]=0$. Estimate the persistence score $s_\eta=\partial e(\theta_C^*)/\partial\eta$ or $s_C=\partial e(\theta_C^*)/\partial C$ using paired perturbations. A positive slope with a positive margin $m=e(\theta_C^*)$ predicts that the saturated branch persists. A nonpositive slope predicts an impending boundary fold, so the controller must reduce the learning rate or increase the clipping margin. The criterion is applied as an empirical local continuation test around the current training state.

Implementation notes

Add a slow supervisory controller around an existing optimizer; do not modify backpropagation. Maintain a primary run at threshold $C_k$, and create two cheap shadow probes every K optimizer steps. The probes reuse the same minibatch sequence and optimizer state but apply $\eta_k+\delta_\eta$ and $\eta_k-\delta_\eta$ for H short steps, with H between 5 and 20. Compute the averaged endpoint $\bar\theta_+$ and $\bar\theta_-$, their unclipped gradient norms, loss variance, fraction of clipped steps, and update-direction cosine. Estimate the branch slope $\hat s_\eta$ from the finite difference in $e=\|g\|-C$. If the primary run is clipped and $e$ has a positive margin while $\hat s_\eta>0$, increase the learning rate by $1+\alpha$, for example $\alpha=0.02$. If $\hat s_\eta\le0$, loss variance rises, or update cosine falls below zero, reduce the learning rate by $1-\beta$, for example $\beta=0.1$. Optionally increase $C$ only when the issue is persistent clipping without instability. The exact equilibrium continuation and slope criterion come from the paper; branch states, slopes, and thresholds are estimated empirically. First experiment: compare this controller with fixed clipping and cosine-decay SGD on CIFAR-10 using a small ResNet-18, restricting probes to 10 steps every 200 training steps. Sweep initial learning rates around the fixed-clipping divergence boundary. The predicted signature is a sign change in $\hat s_\eta$ near the fixed-run instability boundary, with the controller reducing catastrophic loss spikes by at least 50 percent while retaining a learning rate within 20 percent of the largest stable fixed rate.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a readable NumPy MVP with the paper-derived saturation branch and a supervisory clipped-SGD controller using paired shadow probes. The mechanism manifested: the predicted branch-sign transition was P*=0.600 versus observed 0.600, persistence classification matched across the sweep, and the collapse-voltage formula matched numerical inversion to 1.1e-16 with a mean boundary-to-collapse gap of 0.403 on persistent branches. The optimizer controller did not reduce peak loss in this toy setup and generally worsened final loss at moderate/high learning rates, so there is no evidence of a practical training win yet.

Agent confidence: 8/10

Baseline: Fixed clipping, eta0=0.24: final loss 0.0870, peak loss 97.26, clip fraction 0.9917. Across eta0=0.16..0.32, peak losses were 100.93, 99.09, 97.26, 95.45, 93.66.
Idea: Persistence controller, eta0=0.24: final loss 0.1702, peak loss 97.26, clip fraction 0.9925, final eta 0.35. Across eta0=0.16..0.32, peak losses were identical to baseline; final losses were 2.33e-05, 0.189, 0.170, 0.154, 0.159 respectively. Math predictions: transition error <=1.1e-16; collapse inversion error <=1.1e-16.

Limitations: Only the required toy mechanism and a two-dimensional noisy quadratic were tested; no CIFAR-10, ResNet, CUDA run, wall-clock/FLOP accounting, multiple random seeds, or comparison with cosine-decay SGD was performed. The exact paper-equation sweep is an analytical reduced model and does not establish that the empirical optimizer controller estimates branch persistence reliably in neural-network training.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.04740) Sep 2, 2026

Ahead of the baseline (−1.4%) but not statistically significant (p=0.6166, wins 4 / 8 paired seeds).

Benchmark: Tabular regression (Friedman)
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 7.1104
Idea mean: 0
Effect (Δ): -0.1021 (−1.4%; negative = idea better)
Wins: 4 / 8 paired seeds
p-value: 0.6166 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±5.5%
Mechanism: Not confirmed ✗
Practical effect: No effect
Baseline tuning: swept over 6 configs

Limitations:

Only the tabular track was tested because the idea is an optimizer/scheduler intervention. The benchmark used a reduced 1000-train/300-test subset and 15 epochs rather than the largest possible harness budget; no vision, sequence, or dynamics transfer was tested. The controller's probe implementation is an empirical approximation and its toy slope check failed, so the mechanism claim remains unconfirmed.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 31, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented the Saturation-Persistence Trust Region as a clipped-Adam supervisory controller on the structurally matched tabular Friedman regression track, using the shared mlp_tiny architecture and 8 paired seeds. The idea slightly improved mean MSE from 7.1104 to 7.0084, but the paired permutation test was not significant (delta=-0.1021, p=0.61655), so there is no benchmark win. The toy sanity slope contradicted the expected sign, while trained-model probes showed positive slopes; this does not provide robust mechanism confirmation.

Baseline: Fixed global gradient clipping, swept lr in {0.001,0.003,0.006} and C in {0.5,1.0}; best lr=0.006,C=0.5. Full mean MSE=7.110435, std=0.950749, per-seed=[6.655011,6.247303,6.305510,6.786790,6.036830,8.425214,7.887880,8.538945].
Idea: Persistence-controlled clipped Adam, best lr=0.006,C=0.5. Full mean MSE=7.008376, std=0.606246, per-seed=[6.638780,6.576178,6.431657,6.971906,6.361382,7.409494,7.436182,8.241431]. Paired delta=-0.102059, 4/8 wins, permutation p=0.61655. Idea sweep means: lr=0.003 -> 7.156046, lr=0.001 -> 14.842320, lr=0.006 -> 7.008376.

Benchmark result