Dynamical phase selection controls compute scaling in looped transformers

What the math gives to ML

The paper treats a weight-tied transformer as a discrete dynamical system and argues that inference cost is controlled by the bifurcation phase selected during training, not merely by architecture or final accuracy. The most transferable object is the local relaxation law near a saddle-node fold: the iteration time and spectral gap obey the parameter-free asymptotic relation \(\tau(\varepsilon)[1-\lambda_{\max}(-\varepsilon)]\to\pi\). This suggests measuring local Jacobian criticality during inference to predict required iterations and implementing adaptive halting rather than using a fixed loop depth. A second practical direction is phase-aware training or initialization that penalizes undesirable near-unit Jacobian spectra, reducing unpredictable critical slowing down while preserving the option of deliberately targeting compute-adaptive behavior.