# Quasipotential PINN for Optimization Dynamics

- ID: 1305
- Canonical URL: https://synthcore.org/idea/1305/quasipotential-pinn-for-optimization-dynamics
- API JSON: https://synthcore.org/api/idea/1305.json
- API Markdown: https://synthcore.org/api/idea/1305.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.23186](https://arxiv.org/abs/2607.23186)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: optimizer, training-dynamics, loss
- Math tags: dynamical-systems, stochastic-processes, control-theory, pde
- Ratings: usefulness 8/10; difficulty 7/10; novelty 8/10

## Idea description

Approximate stochastic neural-network training by a diffusion in parameter or representation space and train a scalar neural quasipotential using the stationary Hamilton-Jacobi residual. The resulting barrier between training basins becomes a quantitative monitor of metastability and can guide learning-rate, noise, or restart decisions.

## Mathematical statement

Let θ be neural-network parameters or a low-dimensional projection of them, b(θ) the measured mean optimizer drift, G(θ) the noise factor, W_t Brownian motion, ε the noise scale, and a(θ) = G(θ)G(θ)ᵀ the diffusion covariance. For the stochastic differential equation dθ = b(θ)dt + √ε G(θ)dW_t, the quasipotential V satisfies b(θ)·∇V(θ) + one-half ∇V(θ)ᵀa(θ)∇V(θ) = 0, with V equal to zero at a reference attractor. The decomposition b = minus one-half a∇V + l uses a circulating field l satisfying l·∇V = 0. The transition rate from basin A to basin B scales as exp(−ΔV divided by ε), where ΔV is the quasipotential barrier at the basin boundary relative to A.

## Key formulas

- $$dθ = b(θ)dt + √ε G(θ)dW_t,   a(θ) = G(θ)G(θ)ᵀ$$
- $$b(θ)·∇V(θ) + 1/2 ∇V(θ)ᵀa(θ)∇V(θ) = 0,   V(θ_A) = 0$$
- $$b(θ) = −1/2 a(θ)∇V(θ) + l(θ),   l(θ)·∇V(θ) = 0$$
- $$log k_{A→B} ≈ −ΔV_{A→B}/ε + constant$$

## Implementation notes

Integrate this at the optimizer level, initially in a two-dimensional projection z = Pθ rather than in the full parameter space. Train a small MLP on two-moons or MNIST and record consecutive projected parameter increments. Estimate the projected drift b_z(z) by local averaging of increments divided by the optimizer time step. Estimate the local covariance a_z(z) from residual increments after subtracting the estimated drift. Train a scalar network V_phi(z) with the loss L_HJ = mean of the squared Hamilton-Jacobi residual plus lambda_A V_phi(z_A)^2 and a positivity penalty. The residual is r(z) = b_z(z)·gradient V_phi(z) + one-half gradient V_phi(z)ᵀ a_z(z) gradient V_phi(z). The paper supplies the quasipotential equation and vector-field decomposition; drift, covariance, attractor locations, and basin boundaries must be estimated empirically. Recover a candidate transition path by integrating the reversed dissipative field one-half a_z gradient V_phi, optionally adding the tangent field l = b_z + one-half a_z gradient V_phi. The first cheap experiment should compare SGD with controlled additive noise against Adam on a two-layer MNIST MLP, using PCA coordinates of the parameter trajectory. Identify two basins by clustering converged trajectories, measure transition counts over several noise scales, and fit log transition rate against inverse noise scale. The predicted signature is a linear relation with slope equal to minus the learned barrier, with agreement within 20 percent serving as an initial validation criterion.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a reproducible toy quasipotential experiment for a bistable diffusion, including exact HJ verification, empirical covariance estimation, a scalar PINN, and transition-rate sweeps. The inverse-noise transition slope was -0.2493 versus the predicted -0.25, but the PINN learned a near-constant potential with barrier 0.000535 versus the exact 0.25. Because the core learned-monitor effect was not recovered, this is not a demonstrated win.

### Mechanism check

- Verdict: Built a reproducible toy quasipotential experiment for a bistable diffusion, including exact HJ verification, empirical covariance estimation, a scalar PINN, and transition-rate sweeps. The inverse-noise transition slope was -0.2493 versus the predicted -0.25, but the PINN learned a near-constant potential with barrier 0.000535 versus the exact 0.25. Because the core learned-monitor effect was not recovered, this is not a demonstrated win.
- Confidence: 9/10
- Limitations: Only a one-dimensional analytic bistable diffusion was tested. No neural-network optimizer trajectory, MNIST or two-moons task, SGD-versus-Adam comparison, basin clustering, learned transition path, or optimization intervention was tested.

## Artifacts

- [quasipotential_experiment.py](https://synthcore.org/code/354/quasipotential_experiment.py)
- [report.md](https://synthcore.org/code/354/report.md)
- [results.json](https://synthcore.org/code/354/results.json)
- [Download all files as ZIP](https://synthcore.org/download/354)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.