# Phase-Blind Checkpoint Scheduling

- ID: 1534
- Canonical URL: https://synthcore.org/idea/1534/phase-blind-checkpoint-scheduling
- API JSON: https://synthcore.org/api/idea/1534.json
- API Markdown: https://synthcore.org/api/idea/1534.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.28377](https://arxiv.org/abs/2607.28377)
- Category: dynamics
- Solves: stability, scalability
- ML areas: training-dynamics, scheduler, optimizer
- Math tags: dynamical-systems, control-theory, linear-algebra, bifurcations
- Ratings: usefulness 7/10; difficulty 5/10; novelty 7/10

## Idea description

Design distributed training workers so checkpoint service is anonymous: every active writer receives a throughput determined only by the current number of active writers, not by worker identity, age, or phase. For identical compute periods and checkpoint durations shorter than the period, this removes pairwise phase attraction and prevents deterministic checkpoint synchronization; controlled timing jitter can then be added when rapid phase mixing is desired.

## Mathematical statement

Consider identical workers with compute period T, checkpoint duration d less than T, and phase difference delta between two checkpoint starts. Let F(delta) be the phase-gap return map after one recurrence and let g(delta) = F(delta) - delta. An anonymous resource supplies rate f(n) to each active writer when n writers are active, where f depends only on n. For two identical workers, storage contention, a shared cap, or both imply F(delta) = delta and g(delta) = 0, so F'(delta) = 1: there is no pairwise contraction. For N simultaneous write windows, let a_j be the interval between consecutive ordered write starts, for j from 1 to N minus 1. In the uncapped unit-throughput case, the local map is a_j' = ((N-j)/j) a_j. Its eigenvalues are lambda_j = (N-j)/j, with reciprocal eigenvalues and determinant one. More generally, anonymous throughput gives lambda_j = ((N-j) f(j)) / (j f(N-j)), and the product of all eigenvalues remains one. Thus synchrony is not an attractor. With independent per-cycle timing noise of standard deviation sigma, the phase-margin lifetime is predicted to scale as t_mix approximately C times (m/sigma) squared, with C between 0.11 and 0.24 in the reported fleets.

## Key formulas

- $$g(\delta)=F(\delta)-\delta,\qquad F(\delta)=\delta\ \text{for identical jobs on an anonymous resource}$$
- $$a_j'=\frac{N-j}{j}a_j,\qquad \lambda_j=\frac{N-j}{j},\qquad \prod_{j=1}^{N-1}\lambda_j=1$$
- $$\lambda_j=\frac{(N-j)f(j)}{j\,f(N-j)},\qquad \prod_{j=1}^{N-1}\lambda_j=1$$
- $$t_{\mathrm{mix}}\approx C\left(\frac{m}{\sigma}\right)^2,\qquad C\in[0.11,0.24]$$

## Implementation notes

1. Integration point: implement a checkpoint coordinator for synchronous or semi-synchronous data-parallel training. Workers request checkpoints periodically, and the storage gateway estimates the active-writer count n. Allocate every active writer the same throughput f(n), with no priority based on worker identity, request age, or prior service. Optionally add independent bounded noise to each worker's next checkpoint time. 2. Pseudocode: initialize random phases phi[i]. At each cycle, schedule worker i at phase phi[i] modulo period T, collect the active writer set, compute n, assign rate f(n) to each active writer, and update phi[i] by adding Gaussian noise with standard deviation sigma after completion. Record sorted checkpoint phases, pairwise gaps, active-writer counts, and completion latency. 3. Compute from the paper's mechanism: estimate the return map F by regressing each next-cycle pairwise gap on the current gap; estimate f(n) from gateway byte counters; construct the fleet Jacobian from consecutive ordered checkpoint-start intervals a_j. The identity pairwise map and unit determinant are predictions, not fitted assumptions. 4. First cheap experiment: run 8 identical PyTorch workers training a small ResNet on CIFAR-10, checkpointing every 60 seconds with checkpoint duration below 20 seconds. Compare anonymous equal-share service against FIFO or identity-prioritized service and a shared bandwidth cap for 500 cycles from random phases. The predicted signature is pairwise-gap regression slope 1.0 within 20 percent, no systematic decay of the smallest gap, and fleet Jacobian determinant near 1 under anonymous service. Priority or heterogeneity should create nonzero pairwise drift. With jitter, doubling sigma should reduce the phase-margin lifetime by approximately fourfold, because lifetime scales as (m/sigma)^2.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built a compact phase-blind checkpoint scheduling toy verifier with anonymous and priority pair maps, fleet eigenvalue sweeps, and noisy phase-margin lifetime simulation. The three mechanism predictions were confirmed: pair return-map slope 1.0 across T,d settings, unit fleet determinant and reciprocal eigenvalues for N=2..10 and three throughput laws, and noise lifetime exponent -1.962 versus predicted -2. Doubling noise gave a measured lifetime ratio 3.83 versus the predicted 4, so the mathematical mechanism manifested; this is not evidence of production-training throughput gains.

### Mechanism check

- Verdict: Built a compact phase-blind checkpoint scheduling toy verifier with anonymous and priority pair maps, fleet eigenvalue sweeps, and noisy phase-margin lifetime simulation. The three mechanism predictions were confirmed: pair return-map slope 1.0 across T,d settings, unit fleet determinant and reciprocal eigenvalues for N=2..10 and three throughput laws, and noise lifetime exponent -1.962 versus predicted -2. Doubling noise gave a measured lifetime ratio 3.83 versus the predicted 4, so the mathematical mechanism manifested; this is not evidence of production-training throughput gains.
- Confidence: 8/10
- Limitations: No real PyTorch distributed workers, checkpoint storage gateway, network contention, filesystem I/O, wall-clock throughput, or CIFAR-10 training was tested. The anonymous return map and priority comparator are idealized toy maps; the results verify the supplied formulas and noise scaling rather than demonstrating end-to-end operational benefit.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 1
- Paired wins: 0/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_report.json](https://synthcore.org/code/597/bench_report.json)
- [phase_blind.py](https://synthcore.org/code/597/phase_blind.py)
- [phase_blind_bench.py](https://synthcore.org/code/597/phase_blind_bench.py)
- [phase_track.py](https://synthcore.org/code/597/phase_track.py)
- [report.md](https://synthcore.org/code/597/report.md)
- [report_bench_2026-08-31T193326.md](https://synthcore.org/code/597/report_bench_2026-08-31T193326.md)
- [results.json](https://synthcore.org/code/597/results.json)
- [Download all files as ZIP](https://synthcore.org/download/597)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.