# Phase-Repulsive Worker Throttling

- ID: 1154
- Canonical URL: https://synthcore.org/idea/1154/phase-repulsive-worker-throttling
- API JSON: https://synthcore.org/api/idea/1154.json
- API Markdown: https://synthcore.org/api/idea/1154.md
- Verification status: unverified
- Source: [arXiv:2607.19638](https://arxiv.org/abs/2607.19638)
- Category: dynamics
- Solves: stability, scalability, speedup
- ML areas: optimizer, training-dynamics, scheduler
- Math tags: dynamical-systems, bifurcations, control-theory, statistics
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Treat periodic update bursts from distributed training workers or parameter blocks as oscillator phases, and use a shared adaptive compute or learning-rate cap to create deliberately phase-repulsive coupling. When aggregate demand is high, throttle workers currently near their compute peak and preferentially release workers in low-demand phases, spreading communication and gradient-update bursts instead of allowing them to lock together. The controller should be disabled or retuned when its measured feedback delay crosses the phase-lag boundary at which effective coupling changes sign.

## Mathematical statement

For worker phases \(\theta_i\), intrinsic iteration frequencies \(\omega_i\), and coupling strength \(K_1\), use \(\dot{\theta}_i=\omega_i+(K_1/N)\sum_j\sin(\theta_j-\theta_i+\Phi)\). Coherence is measured by \(r e^{\mathrm{i}\psi}=N^{-1}\sum_j e^{\mathrm{i}\theta_j}\), where \(r=0\) denotes dispersed phases and \(r=1\) perfect locking. Let \(\Omega\) be the dominant worker-cycle frequency, \(\tau\) the control delay, and \(\Phi_0\) the phase shift from the workload waveform and controller. The synchronizing component is \(K_{\mathrm{sync}}=K_1\cos(\Phi_0+\Omega\tau)\); its sign changes when the total lag reaches a quarter cycle modulo \(\pi\). For a unimodal natural-frequency density \(g(\omega)\) centered at \(\omega_0\), the incoherent state loses stability near \(K_{\mathrm{sync},c}=2/[\pi g(\omega_0)]\). The neural controller applies a phase-dependent per-worker resource multiplier, with saturation kept inactive during the small-signal regime.

## Key formulas

- $$\dot{\theta}_i=\omega_i+\frac{K_1}{N}\sum_{j=1}^{N}\sin(\theta_j-\theta_i+\Phi),\qquad i=1,\ldots,N.$$
- $$r(t)e^{\mathrm{i}\psi(t)}=\frac{1}{N}\sum_{j=1}^{N}e^{\mathrm{i}\theta_j(t)},\qquad K_{\mathrm{sync}}=K_1\cos\Phi.$$
- $$K_{\mathrm{sync}}(\tau)=K_1\cos\bigl(\Phi_0+\Omega\tau\bigr),\qquad K_{\mathrm{sync},c}=\frac{2}{\pi g(\omega_0)}.$$
- $$u_i(t+1)=\operatorname{clip}\left(u_0\left[1-\alpha\bigl(p(t)-\bar p(t)\bigr)\sin\bigl(\theta_i(t)-\psi(t)\bigr)\right],u_{\min},u_{\max}\right).$$

## Implementation notes

1. Integration point: instrument each data-parallel worker, pipeline stage, or selected parameter block with a scalar phase. Define a worker phase by band-pass filtering its recent GPU-power, step-time, gradient-norm, or all-reduce-traffic trace and applying a Hilbert transform; a simpler prototype can assign phase zero at each observed compute-burst onset and linearly advance it between bursts. Let \(p_t\) be normalized aggregate demand and \(\bar p_t\) its low-pass average. Let \(u_i\) be a worker's microbatch allowance, token budget, local learning-rate multiplier, or compute quota. 2. Pseudocode: at every control interval, compute \(z=N^{-1}\sum_i e^{\mathrm{i}\theta_i}\), \(r=|z|\), and \(\psi=\arg z\). Estimate the dominant frequency \(\Omega\) using an autocorrelation peak or FFT, estimate delay \(\tau\) from the cross-correlation between demand and applied throttling, and calculate \(K_{\mathrm{sync}}=K_1\cos(\Phi_0+\Omega\tau)\). If \(K_{\mathrm{sync}}<0\), update each quota with \(u_i\leftarrow\operatorname{clip}(u_0[1-\alpha(p_t-\bar p_t)\sin(\theta_i-\psi)],u_{\min},u_{\max})\). If it is positive, reduce \(\alpha\), compensate for delay, or disable the controller. 3. Computed versus estimated: \(r\), \(\psi\), \(\Omega\), and \(\tau\) are estimated online. Calibrate \(K_1\) and \(\Phi_0\) by injecting a small sinusoidal quota modulation and fitting the first-harmonic amplitude and phase of aggregate demand. The Kuramoto threshold is a small-signal prediction, not a formal theorem for stochastic NN training. 4. First cheap experiment: run 8 asynchronous PyTorch workers training ResNet-18 on CIFAR-10 with deliberately synchronized 20-step compute/communication bursts. Compare no controller, conventional global token throttling, and this controller over several artificial delays \(\tau\). Measure phase order parameter \(r\), all-reduce peak-to-mean traffic, step-time variance, throughput, and validation accuracy. The predicted signature is a sharp change from decreasing to increasing \(r\) when \(\Phi_0+\Omega\tau\) crosses a zero of the cosine, followed by coherence growth when measured \(K_{\mathrm{sync}}\) exceeds approximately \(2/[\pi g(\omega_0)]\). The empirical transition should agree with the calibrated prediction within about 20%.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.