# Incrementally Passive Monotone RNN

- ID: 586
- Canonical URL: https://synthcore.org/idea/586/incrementally-passive-monotone-rnn
- API JSON: https://synthcore.org/api/idea/586.json
- API Markdown: https://synthcore.org/api/idea/586.md
- Verification status: beats_baseline
- Source: [arXiv:2607.08637](https://arxiv.org/abs/2607.08637)
- Category: dynamics
- Solves: stability, generalization, accuracy
- ML areas: rnn, ssm, training-dynamics
- Math tags: optimization, convex-analysis, dynamical-systems, control-theory
- Ratings: usefulness 7/10; difficulty 6/10; novelty 5/10

## Idea description

Replace the recurrent transition by a dissipative linear state update minus a maximal monotone nonlinear damping operator. Couple the hidden-state update to an output map so that the cell satisfies a discrete analogue of the paper's scattering-passivity inequality, controlling both hidden-state energy and output energy by initial-state energy plus input energy.

## Mathematical statement

The paper defines scattering passivity through the energy inequality \(\|x(\tau)\|^{2}+\int_{0}^{\tau}\|y(t)\|^{2}\,dt\leq \|x(0)\|^{2}+\int_{0}^{\tau}\|u(t)\|^{2}\,dt\), where \(x\) is the state, \(u\) is the input, and \(y\) is the output. Its nonlinear systems are formed by subtracting a maximal monotone operator \(\mathcal M\) from the generator of a scattering-passive linear system. An operator \(\mathcal M:H\to 2^H\) is monotone if \(\langle m_2-m_1,z_2-z_1\rangle\geq0\) for all \(m_i\in\mathcal M(z_i)\), and maximal if it cannot be enlarged while preserving monotonicity. For implementation, use \(\mathcal M=\partial\phi\), the subgradient of a convex function \(\phi\), or the smooth special case \(\mathcal M=\nabla\phi\). The resolvent \((I+\Delta t\mathcal M)^{-1}\) is single-valued and nonexpansive for maximal monotone \(\mathcal M\). Parameterize the linear generator as \(A=S-S^\top-\alpha I\), with \(\alpha\geq0\), so \(A+A^\top\preceq0\). The paper's mechanical example demonstrates that nonsmooth sign damping is admissible: \(\mathcal N([\zeta_1,\zeta_2]^\top)=[0,\frac{\beta}{\sqrt{m_2}}\operatorname{sign}(\zeta_2)]^\top\).

## Key formulas

- $$\|x(\tau)\|^{2}+\int_{0}^{\tau}\|y(t)\|^{2}\,dt\leq \|x(0)\|^{2}+\int_{0}^{\tau}\|u(t)\|^{2}\,dt.$$
- $$\left\{\begin{array}{ll}\dot{x}(t)=Ax(t)+Bu(t),\\ y(t)=\overline{C}x(t)+Du(t),\end{array}\right.$$
- $${\mathcal N}\left(\begin{bmatrix}\zeta_{1}\\\zeta_{2}\end{bmatrix}\right)=\begin{bmatrix}0\\\frac{\beta}{\sqrt{m_{2}}}\,\mathrm{sign}(\zeta_{2})\end{bmatrix},\qquad D_{0}=0.$$
- $$h_{t+1}=\operatorname{prox}_{\Delta t\phi}\!\left((I-\Delta t A)^{-1}(h_t+\Delta t Bu_t)\right),\qquad y_t=Ch_{t+1}+Du_t.$$

## Implementation notes

(1) Integrate at the recurrent or state-space layer, replacing the usual explicit update \(h_{t+1}=\tanh(W_hh_t+W_uu_t+b)\). Parameterize \(A=S-S^\top-\alpha I\), where \(S\) is unconstrained and \(\alpha=\operatorname{softplus}(a)\). Use trainable \(B,C,D\), and initially enforce \(\|D\|_2\leq0.9\) and \(\Delta t\|C\|_2\|B\|_2\leq0.9\) by spectral-norm rescaling. (2) Let \(\phi(h)=\lambda\sum_i h_i^2/2\) for a closed-form baseline, \(\phi(h)=\lambda\|h\|_1\) for soft-thresholding, or a separable convex function with a numerically computed proximal map. Given hidden state \(h\) and input \(u\), compute \(r=(I-\Delta t A)^{-1}(h+\Delta t Bu)\), then solve \(h_{new}=\operatorname{prox}_{\Delta t\phi}(r)\), and output \(y=C h_{new}+Du\). The linear solve can use a cached LU factorization when \(A\) is fixed during a sequence. (3) Backpropagate through 3-10 unrolled proximal iterations, or use implicit differentiation through the proximal optimality condition \(0\in h_{new}-r+\Delta t\partial\phi(h_{new})\). The paper supplies the monotone-operator structure and energy guarantee; norm bounds, timestep, damping strength, and proximal iteration count are empirical settings. Log the residual \(E_t=\|h_{t+1}\|^2+\|y_t\|^2-\|h_t\|^2-\|u_t\|^2\), cumulative energy, hidden norms, and Jacobian-vector amplification. (4) First experiment: compare a 128-dimensional version against GRU, vanilla linear SSM, and an explicit dissipative RNN on sequential MNIST, adding, and long-delay copy tasks. Train with identical AdamW and FLOP budgets. Success means stable hidden norms and gradients on sequences 5-10 times longer than training, equal or better validation accuracy, and lower loss or gradient variance at equal compute.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built an implicit Incrementally Passive Monotone RNN using A=S-S^T-alpha I and a closed-form quadratic proximal damping step, plus a tanh RNN comparator. The mechanism manifested: the resolvent norm stayed below the predicted dissipative bound across alpha, the proximal contraction matched 1/(1+dt lambda) to 2.2e-16, and no-input hidden energy decreased monotonically with stronger damping. On one fixed-seed length-30 sum-regression toy task, the passive model also achieved lower validation MSE (0.388 vs 4.156), though this is only a small secondary signal.

### Mechanism check

- Verdict: Built an implicit Incrementally Passive Monotone RNN using A=S-S^T-alpha I and a closed-form quadratic proximal damping step, plus a tanh RNN comparator. The mechanism manifested: the resolvent norm stayed below the predicted dissipative bound across alpha, the proximal contraction matched 1/(1+dt lambda) to 2.2e-16, and no-input hidden energy decreased monotonically with stronger damping. On one fixed-seed length-30 sum-regression toy task, the passive model also achieved lower validation MSE (0.388 vs 4.156), though this is only a small secondary signal.
- Confidence: 9/10
- Limitations: Only a quadratic smooth proximal operator was tested; no nonsmooth subgradient prox, output-energy scattering inequality with nonzero C/D, GRU/SSM baselines, long-horizon extrapolation, multiple seeds, FLOP-matched training, or broad statistical evaluation was performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_passive.py](https://synthcore.org/code/268/bench_passive.py)
- [bench_report.json](https://synthcore.org/code/268/bench_report.json)
- [experiment.py](https://synthcore.org/code/268/experiment.py)
- [report.md](https://synthcore.org/code/268/report.md)
- [report_bench_2026-08-30T235045.md](https://synthcore.org/code/268/report_bench_2026-08-30T235045.md)
- [results.json](https://synthcore.org/code/268/results.json)
- [Download all files as ZIP](https://synthcore.org/download/268)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.