Passivity-Constrained Neural State-Space Layer

(1) Integration point: replace the transition in a GRU-free recurrent model, latent ODE, or small state-space model with a port-Hamiltonian vector field. Let the input sequence be $u_k\in\mathbb{R}^m$ and latent state be $x_k\in\mathbb{R}^d$. Use a linear decoder $\hat z_k=Cx_k$, or attach the block before an existing prediction head. Start with constant $J$, $R$, $Q$, and $G$ for each layer; later test input-conditioned low-rank factors. (2) Pseudocode: initialize unconstrained parameters $A$, $B$, $L_Q$, vector $r$, and input matrix $G$. Compute $Q=L_QL_Q^\top+10^{-4}I$, $J=A-A^\top$, and $R=BB^\top+\operatorname{diag}(\operatorname{softplus}(r))$. For each time step, compute $g_k=Qx_k$, $v_k=(J-R)g_k+Gu_k$, and update $x_{k+1}=x_k+\Delta t v_k$. Decode $x_{k+1}$ and backpropagate through the rollout. For numerical stability, use two half-step updates or a differentiable adaptive ODE solver while retaining the same vector field. (3) Compute from the mathematics: construct $J$ and $R$ exactly from the antisymmetric and Gram-factor parameterizations, and calculate the certificate $\Delta H_k=H(x_{k+1})-H(x_k)$. Estimate empirically only discretization error, prediction loss, and the fraction of unforced steps with $\Delta H_k>0$. Initially train with prediction loss alone; optionally add a small penalty on positive energy increments to compensate for Euler discretization. (4) First cheap experiment: compare this layer against a vanilla tanh RNN and a diagonal SSM with matched hidden width on sequential MNIST or PhysioNet waveform forecasting. Measure one-step MSE, free-running 100- to 500-step rollout error, gradient norms, and latent energy drift after setting $u=0$. The expected signal is similar short-horizon accuracy but fewer exploding or chaotic rollouts, monotone or near-monotone unforced energy, and lower long-horizon error at equal parameter count. Also scale input amplitude and verify that energy growth tracks the supplied power $y_k^\top u_k$ rather than growing autonomously.