Jacobian Tube Training

Integrate this at the activation level of a recurrent network, state-space model, or transformer unrolled over depth. The nominal state is the hidden activation $z_k$ at layer or recurrent step $k$; the disturbance bound represents quantization error, dropout perturbation, bounded gradient noise, or a measured adversarial activation perturbation. First run the nominal model to cache activations. Estimate $L_k$ with one or two power iterations through Jacobian-vector products, rather than forming the full Jacobian. Set $\delta_k$ from a calibrated perturbation experiment: inject Gaussian, uniform, or quantization noise and use the 99th percentile of the resulting one-step residual. Initialize $r_0$ from the input perturbation radius and recursively compute $r_{k+1}=L_k r_k+\delta_k$. Add $\mathcal L_{\mathrm{tube}}$ to the task loss, or use it as a gate that clips the learning rate when $r_k>m_k$. Pseudocode is: forward and cache $z_k$; estimate $L_k$; update $r\leftarrow L_k r+\delta_k$; accumulate the barrier and gain penalties; backpropagate; then update parameters. The paper-derived components are nominal-plus-tube decomposition and bounded-disturbance propagation; Jacobian estimates, disturbance quantiles, and margins are empirical. The first cheap experiment should use a 4-layer MLP on MNIST, compare SGD, spectral normalization, and this method under activation noise. Sweep a global gain multiplier and predict a sharp stability boundary near $L_{\mathrm{eff}}=1$: for $L_{\mathrm{eff}}<1$, measured radii should converge to $\delta/(1-L_{\mathrm{eff}})$, while for $L_{\mathrm{eff}}>1$ they should grow exponentially with depth. Test whether the measured boundary agrees within 20 percent.