# Normal-Form Degeneracy Trust Region

- ID: 2134
- Canonical URL: https://synthcore.org/idea/2134/normal-form-degeneracy-trust-region
- API JSON: https://synthcore.org/api/idea/2134.json
- API Markdown: https://synthcore.org/api/idea/2134.md
- Verification status: unverified
- Source: [arXiv:2608.11494](https://arxiv.org/abs/2608.11494)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: optimizer, training-dynamics, scheduler, regularization
- Math tags: dynamical-systems, bifurcations, optimization, linear-algebra, geometry
- Ratings: usefulness 6/10; difficulty 7/10; novelty 8/10

## Idea description

Use a low-dimensional polynomial model of local training dynamics to detect when the leading nonlinear restoring behavior becomes degenerate. Shrink the optimizer step in that region, or fit higher-order terms before restoring it, because the paper shows that quartic nondegeneracy determines whether local nonlinear stability can be certified and that sixth-order terms resolve inconclusive cases.

## Mathematical statement

Let y_t be a low-dimensional projected training state around a local stationary regime, and fit a polynomial vector field dy/dt = A y + B(y,y) + C(y,y,y) plus higher-order terms. Transform the fitted model into modal coordinates z_j and define modal actions I_j = |z_j|² divided by 2. The normal-form Hamiltonian has the structure H(I) = ω dot I + H_4(I) + H_6(I) plus higher-order terms. Define D_4 as the determinant, or more robustly the smallest singular value, of the Hessian of H_4 at zero action. Small absolute D_4 marks a quartic degeneracy where the low-order stability estimate is unreliable. Use a trust-region radius proportional to the square root of the normalized absolute D_4.

## Key formulas

- dy/dt = A y + B(y,y) + C(y,y,y) + higher-order terms.
- I_j = |z_j|² divided by 2; H(I) = ω dot I + H_4(I) + H_6(I) + higher-order terms.
- D_4 = determinant of the Hessian of H_4 evaluated at I = 0; treat absolute D_4 less than τ_D as a quartic degeneracy region.
- η_t = η_0 times the minimum of 1 and ρ_t divided by ρ_0; ρ_t = ρ_0 times the square root of (absolute D_4 + ε) divided by (τ_D + ε).

## Implementation notes

Integrate this as a periodically recomputed trust-region controller around SGD, momentum, or a second-order optimizer. Choose a random projection R from the full parameter space to m dimensions, with m between 4 and 16, and define y_t = R x_t after subtracting a running local center. Collect short trajectory segments while optimizer hyperparameters remain fixed. Fit A, B, and C by ridge regression from finite differences of the projected trajectory. Diagonalize A to obtain approximate modal coordinates, estimate actions from modal amplitudes, and fit the quartic action polynomial H_4 by regression. Calculate the smallest singular value of its Hessian; use the determinant only in the two-mode case. If the value falls below threshold τ_D, reduce the learning rate according to the square-root rule above. Optionally collect a longer trajectory and fit sixth-order terms before restoring the rate. Coefficients are estimated empirically, while the transferable mechanism is the distinction between nondegenerate quartic stability and degeneracy requiring higher-order analysis. The first cheap experiment should use a two-dimensional toy loss with a tunable quartic valley, followed by a small ResNet on CIFAR-10. Compare fixed learning rate, gradient clipping, and this controller. The predicted signature is a kink in the safe learning-rate boundary as D_4 approaches τ_D, with the trust radius scaling as the square root of absolute D_4. The predicted instability threshold should agree with an independently measured threshold within 20 percent.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.