Symphony Constrained-Velocity Optimizer
Implementation & benchmark of arXiv:2608.09525 — Symphony: Simple Phase Control for Wave Energy Systems
Source paper: Symphony: Simple Phase Control for Wave Energy Systems arXiv:2608.09525 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace direct parameter updates with a hierarchical controller. An upper loop converts the minibatch gradient into a bounded desired parameter velocity, while a lower loop drives the actual velocity toward that reference through feedback and feedforward compensation. This should suppress minibatch-induced velocity spikes, make the maximum parameter displacement explicit, and preserve stable behavior when gradient estimates or curvature models are inaccurate.
Formulas
Mathematical statement
The source model has the controlled second-order form $m\dot v=f_{ex}+f_u-f_h(z)-\tilde f_r(v,\dot v)+\Delta$, where $z$ is position, $v$ velocity, $m>0$ mass, $f_u$ control, $f_h$ restoring force, $\tilde f_r$ damping, and $\Delta$ uncertainty. Transfer this structure to parameters $\theta$: $\dot\theta=v$ and $M\dot v=u$, with diagonal positive mass matrix $M$. The upper loop computes $v^\star=-P\,\widehat g$ and enforces a hard componentwise speed bound using $v^\star\leftarrow V_{\max}\tanh(-P\widehat g/V_{\max})$, where $\widehat g$ is the minibatch gradient, $P\succ0$ is a diagonal preconditioner, and $V_{\max}>0$ is the maximum parameter velocity. The lower loop uses $u=M\dot v^\star-K(v-v^\star)$, with diagonal $K\succ0$. For tracking error $e=v-v^\star$, the ideal continuous-time dynamics are $\dot e=-M^{-1}Ke$, giving exponential decay rate at least $\lambda_{\min}(M^{-1}K)$. With explicit Euler step $h$, tracking is linearly stable when $\rho(I-hM^{-1}K)<1$, equivalently $0<h<2/\lambda_{\max}(M^{-1}K)$ for symmetric positive-definite diagonal gains. Near a local minimum with Hessian eigenvalues $\lambda_i>0$ and unsaturated reference velocity, the position dynamics reduce to $\dot\theta=-P H(\theta-\theta^\star)$ and decay at rates given by eigenvalues of $PH$.
Implementation notes
Integrate this at the optimizer level, without changing the neural-network forward pass or loss. Maintain parameter velocity v and optionally an exponential-moving-average gradient g. At each minibatch: (1) compute the ordinary gradient g_hat; (2) update the gradient estimate g_hat <- beta*g_hat_old+(1-beta)*g_batch; (3) form the constrained reference velocity v_star = Vmax*tanh(-P*g_hat/Vmax), elementwise; (4) estimate its derivative with finite differences, dv_star = (v_star-v_star_old)/h, or omit the feedforward term during the first ablation; (5) calculate acceleration a = dv_star - M^{-1}K*(v-v_star); (6) update v <- v+h*a; (7) update theta <- theta+h*v. Use diagonal M, K, and P, initialized to ones or Adam-style second-moment preconditioners. Clip or smooth dv_star to avoid numerical impulses. The paper supplies the hierarchical feedforward-plus-feedback architecture and bounded reference-trajectory principle; the gradient, Hessian, and disturbance statistics must be estimated empirically. The first cheap experiment should use a 2-layer MLP on MNIST or CIFAR-10, comparing SGD with momentum, Adam, and this optimizer at matched parameter-update budgets. Sweep h and K/M while logging loss, ||v-v_star||, maximum coordinate velocity, and gradient-noise sensitivity. The quantitative prediction is a sharp tracking-divergence boundary near h_c=2/lambda_max(M^{-1}K): measured error should decay for h<h_c and oscillate or diverge for h>h_c. In the stable regime, ||v_t-v_star_t|| should show geometric decay with slope approximately log rho(I-hM^{-1}K), while parameter updates should never exceed the imposed Vmax bound apart from discretization error.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.