Dissipative Completely-Monotone Memory Layer

Integrate this as a recurrent or state-space block between Transformer or MLP layers, operating on a sequence tensor with shape [batch, time, d]. Choose $m$ memory modes with logarithmically spaced rates $\lambda_j$ covering the task horizon, and positive weights $w_j$ normalized so $\sum_jw_j=M_0$. The trainable parameters are $B\in\mathbb{R}^{p\times d}$, input projection $W$, and optionally a positive semidefinite damping matrix $Q=LL^\top$; do not train the return coupling independently, because it must remain $-B^\top$ to preserve cancellation. For each time step, use implicit Euler on the coupled system: solve $x_{t+1}=x_t+h[-Qx_{t+1}-B^\top\sum_jw_jz_{j,t+1}+Wr_t]$ and $z_{j,t+1}=z_{j,t}+h[Bx_{t+1}-\lambda_jz_{j,t+1}]$. A cheaper semi-implicit variant first computes $z_{j,t+1}=e^{-h\lambda_j}z_{j,t}+((1-e^{-h\lambda_j})/\lambda_j)Bx_t$, using the limit $h$ when $\lambda_j=0$, and then updates $x$. Add a nonlinear gated readout after the stable state rather than placing an unconstrained nonlinearity inside the certified homogeneous dynamics. The exact structural checks are $w_j\ge0$, $\lambda_j\ge0$, $Q\succeq0$, and tied adjoint couplings. Estimate the useful rate range from sequence lengths, for example $\lambda_j\in[1/T,1]$, and optionally learn positive rates using $\lambda_j=\operatorname{softplus}(a_j)$. First test on sequential MNIST and a small WikiText character model against a vanilla GRU and an unconstrained diagonal SSM with equal parameter count. Measure hidden-state norm growth, exploding-gradient frequency, long-context validation loss, and robustness when the integration step size is increased. Success means bounded energy in homogeneous tests, fewer NaNs and gradient spikes, and lower perplexity or classification error at sequence lengths beyond training horizons. Include an ablation with $Q=0$ to verify that stability comes from memory damping and adjoint cancellation rather than ordinary visible-state regularization.