# Thermal Homotopy Training

- ID: 1611
- Canonical URL: https://synthcore.org/idea/1611/thermal-homotopy-training
- API JSON: https://synthcore.org/api/idea/1611.json
- API Markdown: https://synthcore.org/api/idea/1611.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.29590](https://arxiv.org/abs/2607.29590)
- Category: dynamics
- Solves: stability, speedup, sample-efficiency
- ML areas: optimizer, scheduler, training-dynamics, diffusion
- Math tags: dynamical-systems, optimization, numerical-analysis, probability
- Ratings: usefulness 8/10; difficulty 4/10; novelty 5/10

## Idea description

Train a neural model through a sequence of progressively harder objectives, analogous to descending temperature from the exactly solvable infinite-temperature heat kernel. At stage k, initialize from the parameters learned at the previous stage and increase the continuation parameter only when the current residual and sampling diagnostics are stable. This should reduce optimization shocks and avoid repeatedly entering poor basins.

## Mathematical statement

Let \(\beta\ge 0\) be inverse temperature, \(\rho_\theta(x,x';\beta)\) the neural approximation to a density matrix, and \(H\) the system Hamiltonian or, in the ML adaptation, a parameterized objective operator. The Bloch evolution is \(\partial_\beta\rho_\beta=-H\rho_\beta\), giving residual \(R_\theta=\partial_\beta\rho_\theta+H\rho_\theta\). The reference at \(\beta=0\) is the infinite-temperature heat kernel \(\rho_0\), which supplies an analytically known initialization. Use a discrete continuation sequence \(0=\beta_0<\beta_1<\cdots<\beta_K\), train \(\theta_k\) at \(\beta_k\) from \(\theta_{k-1}\), and accept a step only when the normalized residual and parameter update remain below prescribed thresholds. The transferable mechanism is homotopy continuation: nearby objectives should require smaller parameter changes than a direct jump to the final objective.

## Key formulas

- $$\partial_\beta \rho_\beta(x,x')=-\left(H_x\rho_\beta\right)(x,x'),\qquad R_\theta(x,x';\beta)=\partial_\beta\rho_\theta(x,x';\beta)+\left(H_x\rho_\theta\right)(x,x';\beta).$$
- $$\beta_{k+1}=\beta_k+\Delta\beta_k,\qquad \theta_{k+1}\leftarrow\operatorname*{argmin}_{\theta}\;\mathbb{E}_{z\sim q_{\beta_k}}\left[|R_\theta(z;\beta_{k+1})|^2\right]+\lambda\mathcal{L}_{\mathrm{anchor}}(\theta,\theta_k).$$
- $$\frac{\|\theta_{k+1}-\theta_k\|}{\|\theta_k\|+\epsilon}<\tau_\theta,\qquad \frac{\|R_{k+1}\|}{\|R_k\|+\epsilon}<\tau_R.$$

## Implementation notes

1. Integration point: use this as a scheduler around SGD or Adam for a model whose objective can be continuously softened, such as a diffusion noise level, inverse-temperature-weighted energy, label smoothing, or sequence-horizon difficulty. Add a scalar continuation input \(\beta\) to the network and expose the residual or training loss at that value. 2. Pseudocode: initialize \(\beta=0\) and \(\theta\) from the easy reference solution; repeat: sample a minibatch at \(\beta\), compute the task loss plus \(\lambda\|\theta-\theta_{\mathrm{old}}\|^2\), take optimizer steps, and estimate normalized residual \(r=\operatorname{mean}|R_\theta|^2/(\operatorname{mean}|H\rho_\theta|^2+\epsilon)\). If r is below tolerance for M consecutive evaluations, set \(\beta\leftarrow\beta+\Delta\beta\), save \(\theta_{\mathrm{old}}\), and otherwise halve \(\Delta\beta\). 3. Computed from the paper mechanism: the easy reference, continuation direction, and residual-based stage logic. Estimated empirically: residual norms, gradient variance, acceptance rate, and the largest stable \(\Delta\beta\). 4. First cheap experiment: train a small MLP or diffusion denoiser on MNIST with a noise or temperature parameter. Compare direct training at the hardest setting against five to ten continuation stages with equal total optimizer steps. The predicted signature is a sharp reduction in failure rate when the direct objective is unstable, a transient residual spike after each stage followed by decay, and approximately linear parameter displacement in \(\Delta\beta\) for sufficiently small increments. Test whether final-stage loss variance is at least 20% lower and whether the measured maximum stable stage increment is larger than for direct training.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a quadratic homotopy verification and a matched-step PyTorch regression comparison. Predictions manifested: displacement per Δβ stayed 1.1277707129 for Δβ=0.01–0.25 (predicted constant ||A⁻¹b||=1.1277707129), old-solution residual per Δβ stayed 1.2206555616 (predicted ||b||=1.2206555616), and GD diverged at η=0.50 while remaining stable at η=0.49, versus the predicted boundary η=2/λmax=0.495014. The neural task itself showed no win: homotopy hard MSE was higher and variance was 19.3% higher than baseline, so the demonstrated success is the mathematical mechanism rather than improved task performance.

### Mechanism check

- Verdict: Built a quadratic homotopy verification and a matched-step PyTorch regression comparison. Predictions manifested: displacement per Δβ stayed 1.1277707129 for Δβ=0.01–0.25 (predicted constant ||A⁻¹b||=1.1277707129), old-solution residual per Δβ stayed 1.2206555616 (predicted ||b||=1.2206555616), and GD diverged at η=0.50 while remaining stable at η=0.49, versus the predicted boundary η=2/λmax=0.495014. The neural task itself showed no win: homotopy hard MSE was higher and variance was 19.3% higher than baseline, so the demonstrated success is the mathematical mechanism rather than improved task performance.
- Confidence: 8/10
- Limitations: The neural experiment used synthetic full-batch sine regression rather than MNIST or a density-matrix residual, did not implement adaptive step halving or residual-based acceptance, and used only five seeds and one architecture. The stability boundary was sampled on a coarse eta grid rather than solved by binary search.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/478/bench_report.json)
- [bench_stage2.py](https://synthcore.org/code/478/bench_stage2.py)
- [experiment.py](https://synthcore.org/code/478/experiment.py)
- [report.md](https://synthcore.org/code/478/report.md)
- [report_bench_2026-08-31T151431.md](https://synthcore.org/code/478/report_bench_2026-08-31T151431.md)
- [results.json](https://synthcore.org/code/478/results.json)
- [Download all files as ZIP](https://synthcore.org/download/478)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.