{
 "artifacts": [
  {
   "name": "REPORT.md",
   "url": "https://synthcore.org/code/1059/REPORT.md"
  },
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1059/bench_report.json"
  },
  {
   "name": "experiment.py",
   "url": "https://synthcore.org/code/1059/experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1059/report.md"
  },
  {
   "name": "report_bench_2026-09-01T232745.md",
   "url": "https://synthcore.org/code/1059/report_bench_2026-09-01T232745.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1059/results.json"
  },
  {
   "name": "smoke.py",
   "url": "https://synthcore.org/code/1059/smoke.py"
  },
  {
   "name": "spectral_bench.py",
   "url": "https://synthcore.org/code/1059/spectral_bench.py"
  }
 ],
 "category": "dynamics",
 "description": "Use the local Jacobian of a looped transformer to estimate its remaining relaxation time and stop the recurrent computation when the predicted residual reduction is sufficient. Near a saddle-node fold, the paper's asymptotic relation converts an estimated dominant eigenvalue into a compute forecast, allowing dynamic iteration budgets instead of a conservative fixed maximum.",
 "download_zip": "https://synthcore.org/download/1059",
 "formulas_latex": [
  "$$z_{t+1}=F_{\\theta}(z_t;h)$$",
  "$$\\tau(\\varepsilon)\\,[1-\\lambda_{\\max}(-\\varepsilon)]\\to\\pi$$",
  "$$J_t=\\frac{\\partial F_{\\theta}(z_t;h)}{\\partial z_t},\\qquad \\hat{\\tau}_t=\\frac{\\pi}{\\max(\\delta,1-\\hat\\lambda_t)}$$",
  "$$\\hat\\lambda_t\\approx\\frac{\\|J_t v_t\\|_2}{\\|v_t\\|_2},\\qquad v_{t+1}=\\frac{J_t v_t}{\\|J_t v_t\\|_2}$$"
 ],
 "id": 2748,
 "implementation": "Integrate this into the halting controller of a looped transformer, immediately after each application of the tied block. Keep the usual task output head, but expose the hidden state and a scalar residual such as \\(r_t=\\|z_t-z_{t-1}\\|_2/(\\|z_t\\|_2+10^{-6})\\). At every few iterations, estimate local criticality with one or two power iterations: initialize a random vector \\(v\\), compute the Jacobian-vector product \\(u=J_t v\\) using forward-mode autodiff or reverse-mode VJP tricks, set \\(\\hat\\lambda_t=\\|u\\|_2/\\|v\\|_2\\), and normalize \\(v=u/\\|u\\|_2\\). Predict the remaining relaxation time as \\(\\hat\\tau_t=\\pi/\\max(\\delta,1-\\hat\\lambda_t)\\), with \\(\\delta\\) around 0.01 to avoid numerical explosion. Stop when both \\(r_t\u003cr_{\\mathrm{tol}}\\) and the predicted remaining time is below a small threshold; otherwise continue until \\(t_{\\max}\\). Track the last four residuals and disable the fold predictor when they oscillate or increase, using a fixed safe cap for that example. The mathematical quantity is the asymptotic \\(\\pi\\)-calibrated spectral-gap predictor; the empirical choices are the residual threshold, sampling frequency, cap, and oscillation detector. First test on a 4-8 layer weight-tied transformer solving small algorithmic reasoning tasks, comparing fixed-depth inference and standard residual-based early exit at equal accuracy. Success means lower mean iterations and wall-clock cost at the same accuracy, with the predicted \\(\\hat\\tau_t\\) correlating with actual iterations and no instability on oscillatory examples.",
 "math_summary": "The recurrent model is the discrete dynamical system \\(z_{t+1}=F_{\\theta}(z_t;h)\\), where \\(z_t\\) is the hidden state, \\(h\\) is the input representation, and \\(F_\\theta\\) is one weight-tied transformer block. Let \\(J_t=\\partial F_\\theta(z_t;h)/\\partial z_t\\) be its local Jacobian and let \\(\\lambda_{\\max}(-\\varepsilon)\\) denote the dominant eigenvalue in the paper's fold normal-form parameterization, with \\(\\varepsilon\\) measuring distance from the saddle-node transition. The paper states that the relaxation time \\(\\tau(\\varepsilon)\\) and spectral gap satisfy \\(\\tau(\\varepsilon)[1-\\lambda_{\\max}(-\\varepsilon)]\\to\\pi\\). We operationalize this by estimating the dominant magnitude eigenvalue \\(\\hat\\lambda_t\\) of \\(J_t\\) with Jacobian-vector products and using \\(\\hat\\tau_t=\\pi/(1-\\hat\\lambda_t)\\) as a remaining-iteration estimate. This law is expected only near the fold phase; oscillatory or Neimark-Sacker-like dynamics should be detected by non-monotone residuals and handled by a fallback cap.",
 "math_tags": [
  "dynamical-systems",
  "bifurcations",
  "linear-algebra"
 ],
 "ml_areas": [
  "transformer",
  "inference-speedup",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2608.26556",
  "arxiv_url": "https://arxiv.org/abs/2608.26556",
  "summary_what_math_gives_to_ml": "The paper treats a weight-tied transformer as a discrete dynamical system and argues that inference cost is controlled by the bifurcation phase selected during training, not merely by architecture or final accuracy. The most transferable object is the local relaxation law near a saddle-node fold: the iteration time and spectral gap obey the parameter-free asymptotic relation \\(\\tau(\\varepsilon)[1-\\lambda_{\\max}(-\\varepsilon)]\\to\\pi\\). This suggests measuring local Jacobian criticality during inference to predict required iterations and implementing adaptive halting rather than using a fixed loop depth. A second practical direction is phase-aware training or initialization that penalizes undesirable near-unit Jacobian spectra, reducing unpredictable critical slowing down while preserving the option of deliberately targeting compute-adaptive behavior.",
  "title": "Dynamical phase selection controls compute scaling in looped transformers",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 8
 },
 "solves": [
  "inference-speedup",
  "stability"
 ],
 "title": "Spectral-gap adaptive halting",
 "url": "https://synthcore.org/idea/2748/spectral-gap-adaptive-halting",
 "verification": {
  "benchmark": {
   "track": "dynamics",
   "model": "rnn_small",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 0,
    "idea_mean": 0.004345412977272645,
    "delta_mean": 0,
    "per_seed_diffs": [
     0,
     0,
     0,
     0,
     0,
     0,
     0,
     0
    ],
    "idea_wins": 0,
    "n_pairs": 8,
    "p_value": 1,
    "mde": 0,
    "mde_rel_pct": 0,
    "verdict": "no measurable effect",
    "system_worked": false
   },
   "baseline": {
    "best_cfg": {
     "cap": 4,
     "lr": 0.01,
     "rtol": 0.025,
     "tau": 3
    },
    "sweep": null,
    "full": {
     "mean": 0,
     "std": 0,
     "per_seed": null,
     "n": 0
    }
   },
   "idea": {
    "mean": 0.004345412977272645,
    "std": 0,
    "per_seed": [],
    "n": 0
   },
   "mechanism_signature": {
    "confirmed": false,
    "correlation": 0,
    "n_examples": 6400,
    "observed_iterations_mean": 4,
    "predicted_tau_mean": 7.158651609644294,
    "quantity": "trained-model predicted local tau vs observed adaptive iterations"
   },
   "custom_track": null
  },
  "mechanism_confirmed": false,
  "peer_reviewed": false,
  "practical_verdict": "no_effect",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 9,
   "verdict": "Built a readable MVP for spectral-gap prediction and adaptive halting in the fold normal form. The predicted epsilon and b exponents matched closely (+0.500 for the gap and about -0.489 for passage time versus predicted -0.500), while the calibrated product Pi averaged 3.189 versus pi=3.142, a 1.51% error. The controller did not yield a speedup: fixed and residual methods averaged 449 steps, whereas the strict spectral controller hit its 50,000-step cap, so a practical transformer inference win was not demonstrated.",
   "metrics": {
    "baseline": "Fixed/residual stable-branch halting: mean 449 steps over 20 tasks.",
    "idea": "Spectral controller: mean 50,000 steps; no speedup. Mechanism checks: gap-epsilon exponent 0.500, passage-epsilon exponent -0.489, gap-b exponent 0.500, passage-b exponent -0.490, Pi mean 3.189 with 1.51% relative error."
   },
   "how_to_run": "python3 experiment.py",
   "files": [
    "experiment.py",
    "results.json",
    "REPORT.md"
   ],
   "limitations": "This is a scalar fold-normal-form experiment, not a trained looped transformer. It does not test transformer task accuracy, GPU wall-clock cost, autodiff JVP overhead, learned power-iteration error, or tuned controller thresholds; the practical adaptive-halting benefit remains unverified and was absent in this toy controller."
  },
  "status": "failed_benchmark",
  "status_label": "Failed on benchmark",
  "updated_at": "2026-09-01T23:27:45",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}