# Contact-Splitting Momentum Optimizer

- ID: 1329
- Canonical URL: https://synthcore.org/idea/1329/contact-splitting-momentum-optimizer
- API JSON: https://synthcore.org/api/idea/1329.json
- API Markdown: https://synthcore.org/api/idea/1329.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.23642](https://arxiv.org/abs/2607.23642)
- Category: optimization
- Solves: stability, speedup, accuracy
- ML areas: optimizer, training-dynamics
- Math tags: dynamical-systems, symplectic, geometry, optimization, numerical-analysis
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Implement a momentum optimizer as a contact Hamiltonian splitting rather than as a direct Euler discretization. Introduce an auxiliary scalar contact state and compose exact kinetic, potential, and damping subflows; this produces a second-order conformal integrator whose modified contact energy should decay more reliably at moderately large learning rates.

## Mathematical statement

The paper defines a contact Hamiltonian H(x,p,s), where x is the parameter vector, p is its momentum, and s is a scalar contact coordinate, through the vector field \(\dot{x}=\nabla_pH\), \(\dot{p}=-\nabla_xH-p\,\partial_sH\), and \(\dot{s}=p^T\nabla_pH-H\). Along every exact contact trajectory, \(\frac{d}{dt}H=-H\,\partial_sH\). Use \(H(x,p,s)=K(p)+V(x)+D(s)\), with \(K(p)=\frac12p^TM^{-1}p\), \(V(x)=f(x)\) as the training objective, \(M\succ0\) as a diagonal mass/preconditioner, and \(D(s)=\gamma s\), where \(\gamma>0\). Then \(\partial_sH=\gamma\), giving \(H(t)=e^{-\gamma t}H(0)\). The exact subflows are: kinetic K, \(x\leftarrow x+\tau M^{-1}p\), \(s\leftarrow s+\tau K(p)\), with p unchanged; potential V, \(p\leftarrow p-\tau\nabla f(x)\), \(s\leftarrow s-\tau f(x)\), with x unchanged; and damping D, \(p\leftarrow e^{-\gamma\tau}p\), \(s\leftarrow e^{-\gamma\tau}s\), with x unchanged. The symmetric composition \(\Phi_h=\Phi_K^{h/2}\circ\Phi_V^{h/2}\circ\Phi_D^h\circ\Phi_V^{h/2}\circ\Phi_K^{h/2}\) is order two. Backward-error analysis predicts a modified contact Hamiltonian \(\widetilde H=H+O(h^2)\) and modified conformal factor \(\widetilde\gamma=\gamma+O(h^2)\), up to the finite-horizon shadowing defect.

## Key formulas

- $$\dot{x}=\nabla_{p}H,\quad\dot{p}=-\nabla_{x}H-p\frac{\partial H}{\partial s},\quad\dot{s}=p^{T}\nabla_{p}H-H,$$
- $$\frac{\mathrm{d}}{\mathrm{d}t}H=-H\frac{\partial H}{\partial s},$$
- $$H(x,p,s)=\frac{1}{2}p^{T}M^{-1}p+f(x)+\gamma s,\qquad H(t)=e^{-\gamma t}H(0),$$
- $$\Phi_h=\Phi_K^{h/2}\circ\Phi_V^{h/2}\circ\Phi_D^h\circ\Phi_V^{h/2}\circ\Phi_K^{h/2},\qquad \widetilde H=H+O(h^2),\quad \widetilde\gamma=\gamma+O(h^2).$$

## Implementation notes

Integrate this at the optimizer-update level. Treat the flattened network parameter vector x as position, maintain a same-shaped tensor p as momentum, and maintain one scalar s per optimizer instance. Let f(x) be the current minibatch loss and g=∇x f(x); start with M=I, then test a fixed positive diagonal preconditioner M estimated from a warm-up second moment. For each minibatch and step size h, execute: (1) kinetic half-step: s←s+(h/2)K(p), then x←x+(h/2)M^{-1}p; (2) potential half-step: evaluate f and g at the new x, then p←p−(h/2)g and s←s−(h/2)f; (3) damping: p←exp(−γh)p and s←exp(−γh)s; (4) evaluate f and g again and repeat the potential half-step; (5) apply the second kinetic half-step. The two potential steps require two gradient evaluations; compare against SGD with momentum using equal gradient evaluations and equal wall-clock budgets. Compute the certificate Hn=1/2 p_n^T M^{-1}p_n+f(x_n)+γs_n. When Hn and Hn+1 are nonzero with the same sign, estimate the empirical conformal rate by γhat_n=−[log|H_{n+1}|−log|H_n|]/h. Run a controlled quadratic regression problem and a small MLP or ResNet-18 on MNIST or CIFAR-10. Repeat with h, h/2, and h/4; the predicted second-order behavior is a roughly fourfold reduction in certificate residual after halving h. Success criteria are lower loss at equal updates for large h, fewer divergent runs, and improved final accuracy or time-to-target-loss relative to tuned momentum. Log negative or near-zero certificates, extra gradient cost, and whether any gain remains at matched wall-clock time.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built a deterministic contact-splitting optimizer toy implementation and numerically verified the mechanism. Exact certificate decay matched -gamma to 2.7e-10, state error under step halving decreased by 4.0006x, and certificate-rate residual decreased by 4.0003x; the observed stability boundary was 1.9995 versus the predicted h*sqrt(lambda)=2. The mechanism manifested, although the matched-gradient quadratic comparison did not show an optimization win: contact splitting ended at 1.87e-3 loss versus 1.75e-5 for momentum.

### Mechanism check

- Verdict: Built a deterministic contact-splitting optimizer toy implementation and numerically verified the mechanism. Exact certificate decay matched -gamma to 2.7e-10, state error under step halving decreased by 4.0006x, and certificate-rate residual decreased by 4.0003x; the observed stability boundary was 1.9995 versus the predicted h*sqrt(lambda)=2. The mechanism manifested, although the matched-gradient quadratic comparison did not show an optimization win: contact splitting ended at 1.87e-3 loss versus 1.75e-5 for momentum.
- Confidence: 9/10
- Limitations: Only a separable quadratic toy problem was tested; no MLP/MNIST experiment, learned diagonal preconditioner, wall-clock benchmark, or broad hyperparameter tuning was performed. The positive result supports second-order splitting and certificate behavior, not practical superiority over tuned momentum.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_contact.py](https://synthcore.org/code/523/bench_contact.py)
- [bench_report.json](https://synthcore.org/code/523/bench_report.json)
- [experiment.py](https://synthcore.org/code/523/experiment.py)
- [report.md](https://synthcore.org/code/523/report.md)
- [report_bench_2026-08-31T164844.md](https://synthcore.org/code/523/report_bench_2026-08-31T164844.md)
- [results.json](https://synthcore.org/code/523/results.json)
- [Download all files as ZIP](https://synthcore.org/download/523)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.