# Zero-Crossing Reset Integral Optimizer

- ID: 969
- Canonical URL: https://synthcore.org/idea/969/zero-crossing-reset-integral-optimizer
- API JSON: https://synthcore.org/api/idea/969.json
- API Markdown: https://synthcore.org/api/idea/969.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.15572](https://arxiv.org/abs/2607.15572)
- Category: dynamics
- Solves: stability, speed, accuracy
- ML areas: optimizer, training-dynamics
- Math tags: control-theory, dynamical-systems, optimization
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Replace ordinary momentum-like accumulation with a PI controller whose integral state is reset when the proportional error changes sign, indicating that the trajectory has crossed its local target. Apply the mechanism to each parameter block or to a scalar block residual, and impose a dwell time so that minibatch noise cannot trigger arbitrarily frequent resets.

## Mathematical statement

For a scalar tracking error e(t), the continuous-flow controller uses proportional action u_P(t) = -k_P e(t), integral state derivative dz(t)/dt = e(t), and total action u(t) = u_P(t) - k_I z(t), where k_P and k_I are positive gains and z(t) is accumulated mismatch. At a jump, z is set to zero when e(t) crosses zero, while parameters remain continuous. A dwell-time condition requires consecutive reset times to differ by at least tau_d. For a local quadratic neural-network objective f(theta) = one-half theta-transpose H theta, with Hessian eigenvalue lambda, the no-reset linearized mode obeys theta-double-dot + k_P lambda theta-dot + k_I lambda theta = 0. Its characteristic roots are s = (-k_P lambda plus or minus square-root(k_P-squared lambda-squared minus 4 k_I lambda)) divided by 2. The reset removes accumulated integral memory after a sign crossing and should reduce post-crossing overshoot. In discrete training, z_(k+1) = z_k + g_k and theta_(k+1) = theta_k - eta times (k_P g_k + k_I z_(k+1)), followed by z_(k+1) = 0 when g_(k-1) and g_k have opposite signs and the dwell counter is satisfied.

## Key formulas

- $$u(t)=-k_P e(t)-k_I z(t),\qquad \dot z(t)=e(t),\qquad z^+=0\ \text{when }e(t)\ \text{crosses zero},\qquad t_{j+1}-t_j\geq\tau_d.$$
- $$\ddot\theta+k_P\lambda\dot\theta+k_I\lambda\theta=0,\qquad s^2+k_P\lambda s+k_I\lambda=0,\quad \lambda\in\operatorname{eig}(H).$$
- $$z_{k+1}=z_k+g_k,\qquad \theta_{k+1}=\theta_k-\eta\left(k_Pg_k+k_Iz_{k+1}\right),\qquad z_{k+1}\leftarrow0\ \text{if }g_{k-1}g_k\leq0\ \text{and }k-k_{\rm reset}\geq d.$$
- $$k_P^2\lambda=4k_I\quad \text{is the critical-damping boundary for the linearized mode.}$$

## Implementation notes

1. Integration point: add one integral buffer per parameter tensor or, for a cheaper first test, one scalar buffer per layer. Let G_k be the minibatch gradient. The simplest version uses g_k = G_k directly and stores a tensor z_k with the same shape. To make the zero-crossing rule less sensitive to individual coordinates, use a block residual g_k = inner_product(G_k, U_(k-1)) divided by the norm of U_(k-1) plus epsilon, where U_(k-1) is the previous update direction. 2. Pseudocode: compute G; form the residual g; update z = z + g; if the elementwise or blockwise product g times g_old is nonpositive and the reset counter is at least d, set z = 0 and record the reset; form U = k_P times g plus k_I times z; update theta = theta - eta times U; store g_old = g and increment the dwell counter. Clip the integral buffer only as a secondary safety measure, not in the first ablation. 3. Taken from the paper: PI accumulation, reset at a proportional-error zero crossing, and a minimum dwell time. Estimated empirically: Hessian eigenvalue ranges, minibatch sign-flip probability, eta, k_P, k_I, and d. 4. First cheap experiment: train a two-layer MLP on MNIST and a small quadratic objective with known Hessian, comparing SGD, momentum, Adam, non-reset PI, and reset PI at equal update counts. Sweep the ratio k_P squared times lambda divided by 4 k_I. The predicted signature is a transition near ratio one: below the critical-damping boundary, the quadratic error should oscillate and generate repeated zero crossings; above it, the dominant mode should become non-oscillatory. Reset PI should reduce the post-crossing error peak and settling steps, whereas too-small d should create reset chatter and degrade convergence. On the known quadratic, the measured dominant decay rate should match the real part of the predicted characteristic root within about 20 percent before stochastic-gradient noise dominates.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a deterministic quadratic PI/reset-PI prototype with continuous-flow and discrete-time simulations. The critical-damping boundary R=kP²λ/(4kI)=1 was confirmed at every sweep point: R<1 produced complex roots and R≥1 real roots. Resetting the integral state reduced the post-crossing peak from 0.2984 to 3.0e-5, while increasing dwell from 0 to 500 steps reduced resets from 100 to 5; the mechanism manifested clearly, although this is not evidence of a general neural-network win.

### Mechanism check

- Verdict: Built a deterministic quadratic PI/reset-PI prototype with continuous-flow and discrete-time simulations. The critical-damping boundary R=kP²λ/(4kI)=1 was confirmed at every sweep point: R<1 produced complex roots and R≥1 real roots. Resetting the integral state reduced the post-crossing peak from 0.2984 to 3.0e-5, while increasing dwell from 0 to 500 steps reduced resets from 100 to 5; the mechanism manifested clearly, although this is not evidence of a general neural-network win.
- Confidence: 8/10
- Limitations: Only deterministic scalar quadratic dynamics were tested; no MNIST/MLP, minibatch noise, tensor-wise block residuals, Adam or momentum comparisons, wall-clock/FLOP measurements, or GPU execution. The decay-rate fit was only numerically usable for the lowest underdamped sweep point because non-oscillatory trajectories do not provide local peaks.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/406/bench_report.json)
- [bench_stage2.py](https://synthcore.org/code/406/bench_stage2.py)
- [experiment.py](https://synthcore.org/code/406/experiment.py)
- [report.md](https://synthcore.org/code/406/report.md)
- [report_bench_2026-08-31T113109.md](https://synthcore.org/code/406/report_bench_2026-08-31T113109.md)
- [results.json](https://synthcore.org/code/406/results.json)
- [Download all files as ZIP](https://synthcore.org/download/406)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.