✓✓ Beats tuned baseline 2026

Incrementally Passive Monotone RNN

Implementation & benchmark of arXiv:2607.08637 — Aclass of incrementally scattering-passive nonlinear systems

Usefulness7/10

Difficulty6/10

Novelty5/10

Dynamics Stability Generalization Accuracy Rnn Ssm Training dynamics Optimization Convex analysis Dynamical systems Control theory

Source paper: Aclass of incrementally scattering-passive nonlinear systems arXiv:2607.08637 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace the recurrent transition by a dissipative linear state update minus a maximal monotone nonlinear damping operator. Couple the hidden-state update to an output map so that the cell satisfies a discrete analogue of the paper's scattering-passivity inequality, controlling both hidden-state energy and output energy by initial-state energy plus input energy.

Formulas

$$\|x(\tau)\|^{2}+\int_{0}^{\tau}\|y(t)\|^{2}\,dt\leq \|x(0)\|^{2}+\int_{0}^{\tau}\|u(t)\|^{2}\,dt.$$

$$\left\{\begin{array}{ll}\dot{x}(t)=Ax(t)+Bu(t),\\ y(t)=\overline{C}x(t)+Du(t),\end{array}\right.$$

$${\mathcal N}\left(\begin{bmatrix}\zeta_{1}\\\zeta_{2}\end{bmatrix}\right)=\begin{bmatrix}0\\\frac{\beta}{\sqrt{m_{2}}}\,\mathrm{sign}(\zeta_{2})\end{bmatrix},\qquad D_{0}=0.$$

$$h_{t+1}=\operatorname{prox}_{\Delta t\phi}\!\left((I-\Delta t A)^{-1}(h_t+\Delta t Bu_t)\right),\qquad y_t=Ch_{t+1}+Du_t.$$

Mathematical statement

The paper defines scattering passivity through the energy inequality $\|x(\tau)\|^{2}+\int_{0}^{\tau}\|y(t)\|^{2}\,dt\leq \|x(0)\|^{2}+\int_{0}^{\tau}\|u(t)\|^{2}\,dt$, where $x$ is the state, $u$ is the input, and $y$ is the output. Its nonlinear systems are formed by subtracting a maximal monotone operator $\mathcal M$ from the generator of a scattering-passive linear system. An operator $\mathcal M:H\to 2^H$ is monotone if $\langle m_2-m_1,z_2-z_1\rangle\geq0$ for all $m_i\in\mathcal M(z_i)$, and maximal if it cannot be enlarged while preserving monotonicity. For implementation, use $\mathcal M=\partial\phi$, the subgradient of a convex function $\phi$, or the smooth special case $\mathcal M=\nabla\phi$. The resolvent $(I+\Delta t\mathcal M)^{-1}$ is single-valued and nonexpansive for maximal monotone $\mathcal M$. Parameterize the linear generator as $A=S-S^\top-\alpha I$, with $\alpha\geq0$, so $A+A^\top\preceq0$. The paper's mechanical example demonstrates that nonsmooth sign damping is admissible: $\mathcal N([\zeta_1,\zeta_2]^\top)=[0,\frac{\beta}{\sqrt{m_2}}\operatorname{sign}(\zeta_2)]^\top$.

Implementation notes

(1) Integrate at the recurrent or state-space layer, replacing the usual explicit update $h_{t+1}=\tanh(W_hh_t+W_uu_t+b)$. Parameterize $A=S-S^\top-\alpha I$, where $S$ is unconstrained and $\alpha=\operatorname{softplus}(a)$. Use trainable $B,C,D$, and initially enforce $\|D\|_2\leq0.9$ and $\Delta t\|C\|_2\|B\|_2\leq0.9$ by spectral-norm rescaling. (2) Let $\phi(h)=\lambda\sum_i h_i^2/2$ for a closed-form baseline, $\phi(h)=\lambda\|h\|_1$ for soft-thresholding, or a separable convex function with a numerically computed proximal map. Given hidden state $h$ and input $u$, compute $r=(I-\Delta t A)^{-1}(h+\Delta t Bu)$, then solve $h_{new}=\operatorname{prox}_{\Delta t\phi}(r)$, and output $y=C h_{new}+Du$. The linear solve can use a cached LU factorization when $A$ is fixed during a sequence. (3) Backpropagate through 3-10 unrolled proximal iterations, or use implicit differentiation through the proximal optimality condition $0\in h_{new}-r+\Delta t\partial\phi(h_{new})$. The paper supplies the monotone-operator structure and energy guarantee; norm bounds, timestep, damping strength, and proximal iteration count are empirical settings. Log the residual $E_t=\|h_{t+1}\|^2+\|y_t\|^2-\|h_t\|^2-\|u_t\|^2$, cumulative energy, hidden norms, and Jacobian-vector amplification. (4) First experiment: compare a 128-dimensional version against GRU, vanilla linear SSM, and an explicit dissipative RNN on sequential MNIST, adding, and long-delay copy tasks. Train with identical AdamW and FLOP budgets. Success means stable hidden norms and gradients on sequences 5-10 times longer than training, equal or better validation accuracy, and lower loss or gradient variance at equal compute.

Verification

Beats tuned baseline

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significant win ✓

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built an implicit Incrementally Passive Monotone RNN using A=S-S^T-alpha I and a closed-form quadratic proximal damping step, plus a tanh RNN comparator. The mechanism manifested: the resolvent norm stayed below the predicted dissipative bound across alpha, the proximal contraction matched 1/(1+dt lambda) to 2.2e-16, and no-input hidden energy decreased monotonically with stronger damping. On one fixed-seed length-30 sum-regression toy task, the passive model also achieved lower validation MSE (0.388 vs 4.156), though this is only a small secondary signal.

Agent confidence: 9/10

Baseline: tanh RNN: train MSE 3.0201, validation MSE 4.1563
Idea: Passive proximal RNN: train MSE 0.4317, validation MSE 0.3880; resolvent and energy checks confirmed

Limitations: Only a quadratic smooth proximal operator was tested; no nonsmooth subgradient prox, output-energy scattering inequality with nonzero C/D, GRU/SSM baselines, long-horizon extrapolation, multiple seeds, FLOP-matched training, or broad statistical evaluation was performed.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.08637) Sep 2, 2026

Beats the tuned baseline by 94.2% (p=0.0081, wins 8 / 8 paired seeds; metric: lower is better).

Benchmark: Dynamics forecasting
Model: rnn_small_matched
Paired seeds: 8
Baseline mean: 0.0017
Idea mean: 0.0001
Effect (Δ): -0.0016 (−94.2%; negative = idea better)
Wins: 8 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±14.5%
Mechanism: Confirmed ✓
Practical effect: Helps
Baseline tuning: swept over 3 configs

Limitations:

Only the registered dynamics track was tested; no vision, tabular, or sequence transfer was evaluated. The passive cell uses a smooth quadratic proximal operator rather than nonsmooth sign damping, and separate FLOP or wall-clock measurements were not collected.

How to run: python3 bench_passive.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 30, 2026 latest Worked ✓

Agent confidence: 9/10

The passive implicit monotone RNN significantly outperformed the canonical GRU rnn_small baseline on the structurally matched registered dynamics track. It won all 8 paired seeds with mean test MSE 0.00009577 versus 0.00165214 and permutation p=0.0081. The trained-model zero-input hidden-energy contraction signature was also confirmed.

Baseline: GRU rnn_small, tuned lr=0.006: mean test MSE 0.001652143953833729
Delta mean: -0.0015563727831704455
Idea: Passive implicit monotone RNN, tuned lr=0.006, alpha=0.1, lambda=0.05: mean test MSE 0.00009577117066328356
Idea wins: 8/8
P value: 0.0081

Benchmark result