Boundary-Coupled Spectral Memory Layer

Integrate the layer immediately before the output projection of an SSM, RNN, or compact sequence model. Given an input sequence $u_0,\ldots,u_{T-1}$, maintain a state $x_t\in\mathbb{R}^{d}$ split into mode pairs and optionally a small refractory-history block. Construct $J$ as a block diagonal matrix whose blocks are either damped rotation blocks $J_k=[[-a_k,-\omega_k],[\omega_k,-a_k]]$ or near-defective blocks $[[-a_k,1],[-\delta_k^2,-a_k]]$. Store unconstrained parameters $\tilde a_k,\tilde\delta_k$ and set $a_k=\operatorname{softplus}(\tilde a_k)+10^{-4}$ and $\delta_k=\operatorname{softplus}(\tilde\delta_k)$ to enforce continuous-time stability. Create two input projections, $B_{\mathrm{bulk}}u_t$ and $B_{\mathrm{bdry}}u_t$, with independent weights. For a fixed step $\Delta$, discretize with $A=\exp(\Delta J)$ and $K=J^{-1}(A-I)$; use the augmented matrix exponential when $J$ is singular, and set $\bar B_{\mathrm{bulk}}=KB_{\mathrm{bulk}}$, $\bar B_{\mathrm{bdry}}=KB_{\mathrm{bdry}}$. The recurrent update is $x_{t+1}=Ax_t+\bar B_{\mathrm{bulk}}u_t+\bar B_{\mathrm{bdry}}u_t$, followed by $y_t=Cx_t$. In pseudocode: initialize $x=0$; for each $u_t$, compute $z_b=B_bu_t$ and $z_r=B_ru_t$, update $x=Ax+K_bz_b+K_rz_r$, then emit $Cx$. The paper-derived quantities are the stable semigroup-style transition and the separation of generator and boundary forcing; estimate mode frequencies, damping, projections, and relative pathway scales by gradient descent. First run copy, adding, delayed-copy, and permuted sequential MNIST tasks with a 64-state layer against diagonal S4-style dynamics, a GRU, and a standard linear SSM at equal parameter count. Measure validation accuracy, gradient norm, spectral radius, and loss after a fixed number of updates. A useful signal is better delayed-copy accuracy or faster loss descent without exploding gradients. Ablate the boundary pathway and near-defective blocks separately, and test impulse responses: the expected signature is a damped oscillatory response with no unbounded growth.