Diffusive fast-slow recurrent block

(1) Integration point: implement a recurrent or state-space layer operating on a sequence $u_s$, with hidden state $h_s\in\mathbb{R}^{d}$, fast state $z_s\in\mathbb{R}^{d}$, and scalar phase $\phi_s$. Interpret feature channels as nodes of a fixed graph and construct a normalized graph Laplacian $L=I-D^{-1/2}AD^{-1/2}$, where $A$ is a sparse nonnegative channel adjacency matrix and $D_{ii}=\sum_jA_{ij}$. Apply diffusion only to $z$, not directly to the input or slow state. Use small $\eta$ but integrate the fast equation with either two or four substeps per recurrent step. (2) Pseudocode: initialize $h,z,\phi$; for each input $u_s$, compute $g=g_\theta(h,\phi,u_s)$; compute $f=f_\theta(h,\phi,z,u_s)$; for $j=1,\ldots,m$, update $z\leftarrow z+(\Delta t/(m\eta))f-(\Delta t/m)\kappa Lz$, recomputing $f$ after each substep; update $h\leftarrow h+\Delta t g$ and $\phi\leftarrow\phi+\Delta t$. Add the paper-inspired symmetric initialization by pairing channels and initializing paired values identically, or test this as an ablation. (3) Compute $Lz$ exactly from the sparse graph. Estimate the spectral damping rate empirically with the Rayleigh quotient $\widehat\lambda(z)=z^{T}Lz/(z^{T}z)$ after removing the mean component; the expected linear damping of a disagreement mode is approximately $\kappa\lambda_i$. Tune $\kappa$ so that the diffusion timescale is comparable to the slow update, while keeping $\Delta t/\eta$ numerically stable. Do not assume the paper's continuum stability theorem transfers automatically; verify Jacobian spectra numerically. (4) First experiment: train a 2-layer, 128-channel block on sequential-MNIST or permuted-MNIST, comparing a GRU and a vanilla fast-slow block against the diffusive version at matched parameters and FLOPs. Measure validation accuracy, gradient norm through 500-1000 steps, sensitivity to Gaussian hidden-state perturbations, and convergence to a periodic attractor when inputs are held constant. Success means lower exploding-gradient frequency and lower output variance under perturbations without reducing clean accuracy; a stronger result is equal accuracy with fewer recurrent steps or a larger stable $\Delta t$.