Win-Martingale Adaptive Router

Integrate this into the per-token adaptive-depth controller or the MoE router, rather than into the main hidden-state update. Let \(h_k\) be the token representation after layer or routing step \(k\), and initialize \(p_0=\operatorname{sigmoid}(w^Th_0+b)\). Use a positive volatility such as \(\sigma_\theta(p,h)=\sigma_{\min}+\operatorname{softplus}(a^T[h,p,p(1-p)]+c)\), clipped to \(\sigma_{\max}\). At each step sample \(\varepsilon_k\sim\mathcal N(0,1)\), compute \(q=p_k+\sigma_\theta(p_k,h_k)\sqrt{\Delta t}\varepsilon_k\), and apply absorption: set \(p_{k+1}=0\) when \(q\le0\), set \(p_{k+1}=1\) when \(q\ge1\), and otherwise set \(p_{k+1}=q\). For adaptive depth, continue while \(p_k\in(\delta,1-\delta)\); halt with the positive branch at \(p_k\ge1-\delta\), and halt or skip at \(p_k\le\delta\). For MoE, use the terminal branch to select an expert and use a straight-through gate during training. The paper supplies the SDE, absorption rule, and martingale constraint; the implementation estimates \(\sigma_\theta\) from hidden states and approximates the diffusion by Euler-Maruyama. Add a calibration loss on the terminal decision and optionally a small variance penalty if trajectories are excessively noisy. First test a 6-layer Transformer on CIFAR-10 or a small language-model benchmark against fixed-depth inference and a standard sigmoid halting head. Measure accuracy at equal average layers, expected compute at equal accuracy, calibration error, absorption time, and empirical conditional drift \(\operatorname{mean}(p_{k+1}-p_k\mid p_k\text{ bin})\). Success means lower average depth at unchanged accuracy, bounded confidence trajectories, and near-zero conditional drift before absorption.