{
 "artifacts": [
  {
   "name": "bench_experiment.py",
   "url": "https://synthcore.org/code/1081/bench_experiment.py"
  },
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1081/bench_report.json"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1081/report.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1081/results.json"
  },
  {
   "name": "slow_mode_experiment.py",
   "url": "https://synthcore.org/code/1081/slow_mode_experiment.py"
  }
 ],
 "category": "dynamics",
 "description": "Train two parameter replicas with symmetric coupling, treating one replica as a prepared thermalization packet for the other. Estimate the slow local Hessian direction and initialize or periodically reset the packet so that the coupled state has zero projection onto that mode; the target should then relax according to the next-slowest mode rather than the original bottleneck.",
 "download_zip": "https://synthcore.org/download/1081",
 "formulas_latex": [
  "$$\\dot{x}=-Hx+\\kappa(y-x),\\qquad \\dot{y}=-Hy+\\kappa(x-y),\\qquad \\kappa\u003e0.$$",
  "$$Hv_i=\\lambda_i v_i,\\quad c_i=\\frac{v_i^\\top x+v_i^\\top y}{2},\\quad d_i=\\frac{v_i^\\top x-v_i^\\top y}{2},\\qquad \\dot c_i=-\\lambda_i c_i,\\quad \\dot d_i=-(\\lambda_i+2\\kappa)d_i.$$",
  "$$v_1^\\top y(0)=-v_1^\\top x(0)\\ \\Rightarrow\\ c_1(0)=0,\\qquad r_{\\mathrm{after}}\\approx\\min\\{\\lambda_2,\\lambda_1+2\\kappa\\}.$$",
  "$$\\theta_{t+1}=\\theta_t-\\eta[g_\\theta+\\kappa(\\theta_t-\\phi_t)],\\qquad \\phi_{t+1}=\\phi_t-\\eta[g_\\phi+\\kappa(\\phi_t-\\theta_t)].$$"
 ],
 "id": 2732,
 "implementation": "1. Integration point: add a second trainable parameter vector phi with the same architecture as the ordinary model parameter vector theta. At every minibatch, compute both losses and update both replicas with gradients plus a parameter-coupling term. Use the target prediction from theta; use phi only as the packet and optionally average predictions late in training. The discrete Euler updates are theta \u003c- theta-eta[g_theta+kappa(theta-phi)] and phi \u003c- phi-eta[g_phi+kappa(phi-theta)]. 2. Pseudocode: initialize theta and phi, estimate a low-curvature direction v1 using 5--20 Hessian-vector products and Lanczos or inverse iteration, then estimate the displacement x relative to a nearby reference theta_star or an initial checkpoint. Set phi so that v1^T(phi-theta_star)=-v1^T(theta-theta_star), while leaving orthogonal components unchanged. Run the coupled updates and re-estimate v1 every 100--1000 steps. 3. Computed versus estimated: the modal equations and cancellation condition are analytic; v1, lambda_1, and lambda_2 are estimated from Hessian-vector products or by fitting exponential decay rates of projected errors. Minibatch noise will continually repopulate the cancelled mode, so measure its replenishment rate. 4. First cheap experiment: use a two-layer MLP on MNIST and compare SGD, an unprepared two-replica optimizer, and the prepared optimizer at matched total parameter-update FLOPs. Measure loss, validation accuracy, and error projected onto v1. The falsifiable prediction is that the late-time projected error changes from A exp(-lambda_1 t) to approximately B exp(-min(lambda_2,lambda_1+2kappa)t). Increasing kappa should improve the decay rate only until lambda_1+2kappa approximately equals lambda_2. A cancellation ratio |c_1(0)|/|a_1(0)| below 0.1 should yield at least a tenfold reduction in slow-mode amplitude. For discrete updates, instability should appear when eta(lambda_max+2kappa) reaches approximately 2.",
 "math_summary": "Near a stationary point, let the target and packet parameter errors be x=theta-theta^star and y=phi-theta^star, and approximate the loss gradient by grad L(theta^star+x) approximately Hx, where H is the local Hessian. Use coupled gradient-flow dynamics dot x=-Hx+kappa(y-x) and dot y=-Hy+kappa(x-y), with coupling strength kappa\u003e0. For a Hessian eigenvector v_i satisfying Hv_i=lambda_i v_i, define scalar coordinates a_i=v_i^T x and b_i=v_i^T y. Their common and difference coordinates are c_i=(a_i+b_i)/2 and d_i=(a_i-b_i)/2, obeying dot c_i=-lambda_i c_i and dot d_i=-(lambda_i+2kappa)d_i. If lambda_1\u003clambda_2 is the slowest nonzero Hessian eigenvalue, ordinary initialization has asymptotic error proportional to c_1(0) exp(-lambda_1 t), while packet preparation b_1(0)=-a_1(0) gives c_1(0)=0 and removes this mode exactly. The resulting asymptotic rate is at least min(lambda_2,lambda_1+2kappa), subject to nonlinear and stochastic errors.",
 "math_tags": [
  "dynamical-systems",
  "spectral-theory",
  "optimization",
  "linear-algebra"
 ],
 "ml_areas": [
  "optimizer",
  "training-dynamics",
  "initialization"
 ],
 "paper": {
  "arxiv_id": "2608.25141",
  "arxiv_url": "https://arxiv.org/abs/2608.25141",
  "summary_what_math_gives_to_ml": "The paper provides a transferable spectral mechanism: an auxiliary system can be prepared so that, after coupling to a target, its initial state has zero overlap with the slowest relaxation eigenmode. The optimal preparation is therefore determined by eigenvector-overlap cancellation rather than by maximal cooling or maximal distance from equilibrium. A direct neural-network analogue is a coupled pair of parameter replicas, where a deliberately prepared auxiliary replica cancels the slow Hessian mode of training dynamics and accelerates local relaxation without increasing the learning rate.",
  "title": "Thermalization packets and optimal ice cubes",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 8,
  "usefulness": 7
 },
 "solves": [
  "speedup",
  "stability"
 ],
 "title": "Slow-Mode-Canceling Optimizer Packet",
 "url": "https://synthcore.org/idea/2732/slow-mode-canceling-optimizer-packet",
 "verification": {
  "peer_reviewed": false,
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 9,
   "verdict": "Built and numerically verified the coupled two-replica optimizer on a diagonal quadratic Hessian. Exact slow-mode cancellation was observed, the measured rate matched the discrete prediction across the kappa sweep to numerical precision, saturation occurred at kappa=0.3 as predicted, and the Euler stability boundary was predicted at kappa=19.0 with divergence observed at kappa=19.05. The prepared packet reached 1.05% of the unprepared final loss in this toy case.",
   "metrics": {
    "baseline": "Unprepared coupled replicas: final loss 0.0370561 at eta=0.05, kappa=0.35, 100 steps; slow-mode amplitude after 30 steps was 0.860384 of initial target amplitude.",
    "idea": "Prepared packet: final loss 0.000388824 under identical settings, or 0.01049x baseline; exact initial c1 cancellation and c1=0 after 30 steps. Rate sweep predicted/observed continuous-equivalent rates matched: 0.100251/0.100251, 0.201007/0.201007, 0.404054/0.404054, 0.712544/0.712544, and 0.712544/0.712544 for kappa 0, 0.05, 0.15, 0.3, and 0.6. Stability critical kappa was predicted as 19.0; kappa=19.05 and 19.5 diverged while kappa\u003c=19.0 did not."
   },
   "how_to_run": "python3 slow_mode_experiment.py",
   "files": [
    "slow_mode_experiment.py",
    "results.json"
   ],
   "limitations": "Only a deterministic diagonal quadratic/local-Hessian model was tested; no MLP, MNIST, stochastic minibatch replenishment, Hessian-vector estimation, nonlinear effects, wall-clock or FLOP comparison, or practical packet-reset schedule was evaluated."
  },
  "status": "unverified",
  "status_label": "Unverified",
  "updated_at": "2026-09-01T23:58:36",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}