Quasipotential PINN for Optimization Dynamics

Integrate this at the optimizer level, initially in a two-dimensional projection z = Pθ rather than in the full parameter space. Train a small MLP on two-moons or MNIST and record consecutive projected parameter increments. Estimate the projected drift b_z(z) by local averaging of increments divided by the optimizer time step. Estimate the local covariance a_z(z) from residual increments after subtracting the estimated drift. Train a scalar network V_phi(z) with the loss L_HJ = mean of the squared Hamilton-Jacobi residual plus lambda_A V_phi(z_A)^2 and a positivity penalty. The residual is r(z) = b_z(z)·gradient V_phi(z) + one-half gradient V_phi(z)ᵀ a_z(z) gradient V_phi(z). The paper supplies the quasipotential equation and vector-field decomposition; drift, covariance, attractor locations, and basin boundaries must be estimated empirically. Recover a candidate transition path by integrating the reversed dissipative field one-half a_z gradient V_phi, optionally adding the tangent field l = b_z + one-half a_z gradient V_phi. The first cheap experiment should compare SGD with controlled additive noise against Adam on a two-layer MNIST MLP, using PCA coordinates of the parameter trajectory. Identify two basins by clustering converged trajectories, measure transition counts over several noise scales, and fit log transition rate against inverse noise scale. The predicted signature is a linear relation with slope equal to minus the learned barrier, with agreement within 20 percent serving as an initial validation criterion.