{
 "artifacts": null,
 "category": "dynamics",
 "description": "Replace a fixed stack of expensive residual blocks with a sequence of increasingly accurate correction stages. Each stage starts from the previous representation, applies a cheap low-capacity update, and optionally performs a more expensive correction; inference halts when the normalized change between successive approximations falls below a tolerance. This creates an anytime network whose computation adapts to the difficulty of each input.",
 "formulas_latex": [
  "$$\\frac{d\\bm{u}}{dt}=\\bm{S}(t,\\bm{u}(t))+\\bm{N}(t,\\bm{u}(t)),\\qquad \\bm{u}(0)=\\bm{z}.$$",
  "$$\\frac{\\left\\lVert\\underline{\\bm{u}}^{M^{(p)},(p)}-\\underline{\\bm{u}}^{M^{(p-1)},(p-1)}\\right\\rVert_{2}}{\\left\\lVert\\underline{\\bm{u}}^{M^{(p)},(p)}\\right\\rVert_{2}}\\leq\\varepsilon.$$",
  "$$h^{(p)}=I_{p-1\\to p}\\!\\left(h^{(p-1)}\\right)+C_{p,\\theta_p}\\!\\left(I_{p-1\\to p}\\!\\left(h^{(p-1)}\\right),x\\right),\\qquad r_p=\\frac{\\left\\lVert h^{(p)}-I_{p-1\\to p}(h^{(p-1)})\\right\\rVert_2}{\\left\\lVert h^{(p)}\\right\\rVert_2+\\delta}.$$",
  "$$\\text{halt at level }p\\text{ if }r_p\\leq\\varepsilon;\\qquad p\\leftarrow p+1\\text{ otherwise}.$$"
 ],
 "id": 116,
 "implementation": "Integrate this at inference time as a replacement for a fixed-depth residual stack, preferably first in a small MLP or Transformer feed-forward sublayer. Let $h^{(0)}$ be the output of a cheap base block. Build levels $p=1,\\ldots,P$ with increasing capacity, such as hidden widths $[64,128,256]$, increasing low-rank rank, or progressively larger token-mixing kernels. Define $I_{p-1\\to p}$ as an explicit projection from the previous width to the next width; initialize it as nearest-width copying plus a learned linear projection. At each level compute $\\tilde h=I_{p-1\\to p}(h^{(p-1)})$, compute the correction $c=C_{p,\\theta_p}(\\tilde h,x)$, and set $h^{(p)}=\\tilde h+c$. Initialize the correction branches near zero so that successive levels are refinements rather than unrelated networks. Compute $r_p=\\|h^{(p)}-\\tilde h\\|_2/(\\|h^{(p)}\\|_2+10^{-6})$ per example, detach $r_p$ only for the halting decision, and stop evaluating later levels when $r_p\\leq\\varepsilon$. During training, use a fixed maximum $P$ with a weighted loss on every level, or use straight-through halting together with a compute penalty $\\lambda p$. The paper supplies the relative-change stopping rule; cost and accuracy thresholds must be estimated empirically. Begin with CIFAR-10 and a three-level residual MLP or small ViT, comparing against a fixed-depth model with equal maximum capacity. Measure accuracy at equal average FLOPs, average executed levels, worst-case latency, and correlation between $r_p$ and final prediction error. Success means at least 25% lower average inference FLOPs with no more than 0.2 percentage points accuracy loss, while difficult examples automatically use deeper levels.",
 "math_summary": "The paper starts from a split dynamical system $\\frac{d\\bm{u}}{dt}=\\bm{S}(t,\\bm{u})+\\bm{N}(t,\\bm{u})$, where $\\bm{u}(t)\\in\\mathbb{R}^{Q}$ is the state, $\\bm{S}$ is the stiff component, and $\\bm{N}$ is the nonstiff component. Its adaptive criterion is $\\frac{\\left\\lVert\\underline{\\bm{u}}^{M^{(p)},(p)}-\\underline{\\bm{u}}^{M^{(p-1)},(p-1)}\\right\\rVert_{2}}{\\left\\lVert\\underline{\\bm{u}}^{M^{(p)},(p)}\\right\\rVert_{2}}\\leq\\varepsilon$, where $p$ is the current correction/order, $M^{(p)}$ is the number of discretization nodes used at level $p$, $\\underline{\\bm{u}}^{M^{(p)},(p)}$ is the current approximation, and $\\varepsilon$ is a user tolerance. The neural adaptation uses $h^{(p)}$ for the level-$p$ hidden approximation and $I_{p-1\\to p}$ for interpolation or prolongation from the cheaper level to the current representation. The transferable property is the a posteriori relative correction estimate: additional computation is conditional on observed convergence rather than imposed at a fixed depth.",
 "math_tags": [
  "numerical-analysis",
  "dynamical-systems",
  "approximation-theory"
 ],
 "ml_areas": [
  "rnn",
  "training-dynamics",
  "inference-speedup"
 ],
 "paper": {
  "arxiv_id": "2608.23919",
  "arxiv_url": "https://arxiv.org/abs/2608.23919",
  "summary_what_math_gives_to_ml": "The transferable asset is the paper's view of high-order computation as a sequence of progressively more accurate correction iterates, rather than as a fixed-depth computation graph. Its interpolation between lower- and higher-order discretizations suggests a neural module that begins with cheap approximate updates and only spends additional computation when the current representation has not converged. The paper's explicit relative-change stopping criterion provides a simple, architecture-independent halting rule for adaptive-depth residual or recurrent networks. The strongest first experiment is therefore an anytime refinement block with shared or nested parameters and per-example early stopping, rather than a literal ODE solver embedded in a network.",
  "title": "New Efficient Implicit-Explicit Deferred Correction methods",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 5,
  "usefulness": 6
 },
 "solves": [
  "speedup",
  "stability",
  "scalability"
 ],
 "title": "Deferred-Correction Adaptive-Depth Block",
 "url": "https://synthcore.org/idea/116/deferred-correction-adaptive-depth-block",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}