# Complete Log-Barrier Natural Gradient

- ID: 384
- Canonical URL: https://synthcore.org/idea/384/complete-log-barrier-natural-gradient
- API JSON: https://synthcore.org/api/idea/384.json
- API Markdown: https://synthcore.org/api/idea/384.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.03036](https://arxiv.org/abs/2607.03036)
- Category: optimization
- Solves: stability, accuracy, generalization
- ML areas: optimizer, training-dynamics, initialization
- Math tags: differential-geometry, geometry, spectral-theory, optimization
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Constrain a neural parameter block to a bounded open domain and replace its Euclidean optimizer with a Riemannian gradient induced by the Hessian of the logarithmic barrier g=-log(-rho). The metric diverges near the boundary, so updates automatically become small when parameters approach saturation or an invalid region, while the logarithmic exhaustion has bounded intrinsic gradient.

## Mathematical statement

Let rho:Omega -> [-1,0) be a smooth strictly plurisubharmonic exhaustion on a bounded domain Omega in complex dimension n, and define g=-log(-rho) and omega=dd^c g. The paper uses partial g=-partial rho/rho and omega= -dd^c rho/rho + (i partial rho wedge bar-partial rho)/rho^2. The key estimate is |partial g|_omega^2 <= 1, so the barrier coordinate has bounded intrinsic gradient despite diverging at the boundary. The paper also proves lambda_0(Delta_omega,Omega)=n^2 when the weighted Monge–Ampere condition integral_Omega (-rho)^epsilon (dd^c rho)^n < infinity holds for every epsilon>0. For implementation, use a real bounded parameter domain and the analogous real Hessian metric G(x)=nabla^2[-log(-rho(x))]. The spectral equality is a motivation for a uniform geometric scale, not an assumption that holds for every neural parameter domain.

## Key formulas

- $$g=-\log(-\rho),\qquad \partial g=-\frac{\partial\rho}{\rho}$$
- $$\omega=dd^cg=-\frac{dd^c\rho}{\rho}+\frac{\sqrt{-1}\,\partial\rho\wedge\bar{\partial}\rho}{\rho^2}$$
- $$\lvert\partial g\rvert_{\omega}^{2}\leq 1$$
- $$\lambda_0(\Delta_{\omega},\Omega)=\inf_{0\neq f\in C_0^\infty(\Omega)}\frac{4\int_\Omega\sum_{i,j}g^{\bar ji}f_i f_{\bar j}\,\omega^n}{\int_\Omega |f|^2\,\omega^n}=n^2$$

## Implementation notes

(1) Integration point: apply this to a bounded, semantically meaningful parameter block, such as normalized attention scales, LoRA coefficients, mixture weights, or a small adapter. Choose Omega=(-R,R)^d and rho(x)=||x||_2^2/R^2-1. Define g(x)=-log(-rho(x)) and G(x)=nabla^2 g(x)+delta I. Replace the AdamW update for this block by a metric-preconditioned update x_{t+1}=Projection_Omega[x_t-eta G(x_t)^(-1) grad_x L], using a Cholesky solve rather than an explicit inverse. For high-dimensional blocks, use diagonal or block-diagonal G. (2) Pseudocode: compute rho, g, and G; compute v=grad_x L; solve Gs=v; set x=x-eta s; if rho(x)>=-epsilon, backtrack eta until rho(x)<-epsilon; apply decoupled weight decay separately. Optionally add lambda*g(x) to the loss. (3) The logarithmic potential and metric are computed exactly; delta, eta, projection tolerance, and block approximation are engineering choices. Estimate the condition number of G, intrinsic update norm sqrt(v^T G^(-1)v), boundary-hit rate, and empirical Rayleigh quotients. (4) First experiment: train a two-layer MLP and a small ViT on CIFAR-10, constraining only 64-dimensional LoRA or normalization-scale parameters. Compare AdamW, projected AdamW, and the barrier update at matched steps and wall-clock time. Measure loss descent, validation accuracy, gradient spikes, NaNs, boundary proximity, and optimizer overhead. A successful result is fewer unstable updates and faster loss reduction at equal FLOPs, with less than 20 percent overhead for a diagonal metric.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: no_effect
- Verdict: Built a real bounded-ball logarithmic-barrier natural-gradient MVP with an exact Hessian metric, Cholesky-equivalent solves, and backtracking safety. The intrinsic barrier-gradient bound held with maximum squared norm 0.999413 and radial formula error 9.0e-12; natural-gradient steps shrank dramatically near the boundary. The stability phenomenon was reproducible, but projected Euclidean descent achieved lower final loss on this toy problem, so no optimization-quality win was demonstrated.

### Mechanism check

- Verdict: Built a real bounded-ball logarithmic-barrier natural-gradient MVP with an exact Hessian metric, Cholesky-equivalent solves, and backtracking safety. The intrinsic barrier-gradient bound held with maximum squared norm 0.999413 and radial formula error 9.0e-12; natural-gradient steps shrank dramatically near the boundary. The stability phenomenon was reproducible, but projected Euclidean descent achieved lower final loss on this toy problem, so no optimization-quality win was demonstrated.
- Confidence: 9/10
- Limitations: Only a 2D quadratic toy problem was tested; no neural network, CIFAR-10, AdamW comparison, wall-clock overhead, high-dimensional block approximation, or generalization evaluation was performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.46775
- Paired wins: 6/8
- Benchmark verdict: no measurable effect

## Artifacts

- [barrier_experiment.py](https://synthcore.org/code/1281/barrier_experiment.py)
- [bench_report.json](https://synthcore.org/code/1281/bench_report.json)
- [report.md](https://synthcore.org/code/1281/report.md)
- [report_bench_2026-09-04T111832.md](https://synthcore.org/code/1281/report_bench_2026-09-04T111832.md)
- [stage2_bench.py](https://synthcore.org/code/1281/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/1281)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.