Mesh-Stable Residual Gain Chain

Integrate this at every residual block of a 50- to 100-layer MLP, ResNet, or sequence SSM. Add a scalar parameter \(u_i\) per block and map it to a positive gain with \(\gamma_i=\operatorname{softplus}(u_i)+\epsilon\). After every optimizer step, project neighboring gains onto the feasible set defined by \(\gamma_i\leq1/\gamma_{i-1}\) and \(\gamma_i+1/\gamma_{i-1}\geq2\), or use a penalty \(\lambda\sum_i[\max(0,\gamma_i\gamma_{i-1}-1)^2+\max(0,2-\gamma_i-1/\gamma_{i-1})^2]\). In the forward pass, set the communicated state to the previous block output; for a delayed experiment, maintain a queue and use a randomly delayed predecessor activation. Pseudocode is: compute \(c_i=\text{queue}[i]\); compute \(r_i=F_i(h_{i-1},c_i)\); set \(h_i=h_{i-1}+\gamma_i r_i\); enqueue \(h_i\); backpropagate; project or penalize gains. The paper supplies the gain inequalities; the Jacobian norm and frequency response must be estimated empirically with power iteration or finite perturbations. First experiment: train 20-, 50-, and 100-layer residual MLPs on CIFAR-10 or a synthetic long-horizon copy task, comparing unconstrained gains, constant gain, and constrained gains. Inject a perturbation at layer 1 and measure \(\|\delta h_L\|/\|\delta h_1\|\). The quantitative prediction is a sharp reduction in depth-induced amplification: constrained models should keep median amplification below 1 or approximately flat in depth, while unconstrained models should show exponential growth; violating \(\gamma_i\gamma_{i-1}\leq1\) should produce a measurable increase in amplification.