Deferred-Correction Adaptive-Depth Block

Integrate this at inference time as a replacement for a fixed-depth residual stack, preferably first in a small MLP or Transformer feed-forward sublayer. Let $h^{(0)}$ be the output of a cheap base block. Build levels $p=1,\ldots,P$ with increasing capacity, such as hidden widths $[64,128,256]$, increasing low-rank rank, or progressively larger token-mixing kernels. Define $I_{p-1\to p}$ as an explicit projection from the previous width to the next width; initialize it as nearest-width copying plus a learned linear projection. At each level compute $\tilde h=I_{p-1\to p}(h^{(p-1)})$, compute the correction $c=C_{p,\theta_p}(\tilde h,x)$, and set $h^{(p)}=\tilde h+c$. Initialize the correction branches near zero so that successive levels are refinements rather than unrelated networks. Compute $r_p=\|h^{(p)}-\tilde h\|_2/(\|h^{(p)}\|_2+10^{-6})$ per example, detach $r_p$ only for the halting decision, and stop evaluating later levels when $r_p\leq\varepsilon$. During training, use a fixed maximum $P$ with a weighted loss on every level, or use straight-through halting together with a compute penalty $\lambda p$. The paper supplies the relative-change stopping rule; cost and accuracy thresholds must be estimated empirically. Begin with CIFAR-10 and a three-level residual MLP or small ViT, comparing against a fixed-depth model with equal maximum capacity. Measure accuracy at equal average FLOPs, average executed levels, worst-case latency, and correlation between $r_p$ and final prediction error. Success means at least 25% lower average inference FLOPs with no more than 0.2 percentage points accuracy loss, while difficult examples automatically use deeper levels.