# Boundary-normal trust-region flow

- ID: 2800
- Canonical URL: https://synthcore.org/idea/2800/boundary-normal-trust-region-flow
- API JSON: https://synthcore.org/api/idea/2800.json
- API Markdown: https://synthcore.org/api/idea/2800.md
- Verification status: unverified
- Source: [arXiv:2608.28976](https://arxiv.org/abs/2608.28976)
- Category: dynamics
- Solves: stability, training
- ML areas: optimizer, training-dynamics, regularization
- Math tags: dynamical-systems, differential-geometry, control-theory, optimization
- Ratings: usefulness 6/10; difficulty 4/10; novelty 6/10

## Idea description

Replace the assumption that strong convexity keeps optimization inside a valid parameter chart with an explicit viability condition on the chart boundary. For Lie-group neural-network parameters or bounded latent coordinates, modify each update so its velocity has nonpositive outward radial component, using either a radial barrier or projection onto the tangent cone.

## Mathematical statement

Let \(x=\log R\in\mathbb{R}^{3}\), \(\mathcal C_\rho=\{x:\|x\|\leq\rho\}\), and \(f(x)=\tfrac12(x-x^*)^\top H(x-x^*)\), with \(H=\begin{bmatrix}1&0.7&0\\0.7&1&0\\0&0&1\end{bmatrix}\), \(x^*=(0.23,0.16,0)^\top\), and \(\rho=0.3\). The paper certifies \(\mathrm{Hess}_{\mathrm{SO}(3)}f\succeq0.172I\) throughout the ball, yet the gradient flow has an exact outward radial velocity \(21/500=0.042\) at a boundary point. For a Euclidean chart, forward invariance requires \(\frac{d}{dt}\|x\|^2=2x^\top v(x)\leq0\) whenever \(\|x\|=\rho\), where \(v\) is the parameter velocity. A differentiable barrier \(B(x)=-\log(1-\|x\|^2/\rho^2)\) gives \(\nabla B(x)=2x/[\rho^2(1-\|x\|^2/\rho^2)]\), whose coefficient diverges at the boundary and suppresses outward motion.

## Key formulas

- $$f(x)=\frac{1}{2}(x-x^*)^\top H(x-x^*),\qquad H=\begin{bmatrix}1&0.7&0\\0.7&1&0\\0&0&1\end{bmatrix},\qquad x^*=\begin{pmatrix}0.23\\0.16\\0\end{pmatrix}.$$
- $$\mathrm{Hess}_{\mathrm{SO}(3)}f\succeq0.172I\ \text{on}\ \|x\|\leq0.3,\qquad \left.\frac{d}{dt}\|x\|\right|_{\|x\|=0.3}=\frac{21}{500}>0\ \text{at the certified boundary point}.$$
- $$\frac{d}{dt}\|x\|^2=2x^\top v(x)\leq0\quad(\|x\|=\rho),\qquad v_{\mathrm{barrier}}(x)=-\nabla f(x)-\lambda_B\nabla B(x),\quad B(x)=-\log\left(1-\frac{\|x\|^2}{\rho^2}\right).$$
- $$v_{\mathrm{proj}}(x)=v(x)-\max\left(0,\frac{x^\top v(x)}{\|x\|^2}\right)x.$$

## Implementation notes

1. Integration point: apply this to any neural module whose parameters are represented in a Lie-algebra chart, such as rotation-valued layers, equivariant networks, neural ODE states constrained to a ball, or a low-dimensional latent code with a required trust region. Compute the ordinary minibatch gradient velocity \(v=-g\), then modify it before the optimizer update. Start with the cheap projection rule; add the barrier as a differentiable alternative. For a discrete step, use \(x_{k+1}=x_k+\eta v_{\mathrm{proj}}(x_k)\), or retract a Lie-group parameter with \(R_{k+1}=R_k\exp(\eta\widehat v_{\mathrm{proj}})\). 2. Pseudocode: compute loss and gradient \(g\); set \(v=-g\); if \(\|x\|\geq\rho-\epsilon\), compute \(a=x^\top v/\|x\|^2\) and replace \(v\leftarrow v-\max(0,a)x\); update using the exponential-map retraction. For the barrier version, use \(v\leftarrow-g-\lambda_B 2x/[\rho^2(1-\|x\|^2/\rho^2)]\). 3. Computed versus estimated: \(x\), its norm, and the boundary projection are computed exactly per batch; the paper's curvature certificate is not needed by the implementation, while the actual outward-rate distribution is estimated by logging \(x^\top v/\|x\|\) on near-boundary iterates. 4. First experiment: train a small rotation-equivariant MLP or SO(3)-parameterized linear layer on synthetic rotated-MNIST, comparing Adam, unconstrained SGD, projected flow, and barrier flow with \(\rho=0.3\). Initialize near the paper's quadratic example and use the same \(H\) as a controlled test. The falsifiable signature is that the unconstrained method exhibits positive boundary-normal velocity near the boundary, with a controlled reproduction near \(0.042\), and exits the chart; projection should make every logged boundary-normal velocity nonpositive up to numerical tolerance and produce zero chart exits. In discrete time, measure the largest safe learning rate and test whether it scales approximately as the distance-to-boundary divided by the maximum outward velocity.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.