# ISS-Constrained Modular Recurrent Network

- ID: 891
- Canonical URL: https://synthcore.org/idea/891/iss-constrained-modular-recurrent-network
- API JSON: https://synthcore.org/api/idea/891.json
- API Markdown: https://synthcore.org/api/idea/891.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.14078](https://arxiv.org/abs/2607.14078)
- Category: architecture
- Solves: stability, generalization, accuracy
- ML areas: rnn, ssm, optimizer, training-dynamics
- Math tags: dynamical-systems, control-theory, linear-algebra, numerical-analysis
- Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10

## Idea description

Replace an unconstrained recurrent block with two coupled modules: a contractive perceptual estimator and an input-to-state-stable cognitive state transition. Spectral normalization and a controlled Euler residual step enforce a quantitative gain condition, preventing hidden-state explosion while retaining long memory when the contraction factor is chosen close to one.

## Mathematical statement

Let z_k in R^p be a perceptual latent state, x_k in R^n a cognitive state, and u_k the input. Use z_{k+1}=P_theta(z_k,u_k) and x_{k+1}=C_theta(x_k,z_{k+1},u_k). If P_theta is Lipschitz in z with constant alpha<1, then two perceptual trajectories under the same input satisfy ||z_k-z_tilde_k|| <= alpha^k ||z_0-z_tilde_0||. If the cognitive block satisfies ||C_theta(x,z,u)-C_theta(x_tilde,z_tilde,u)|| <= a||x-x_tilde||+b||z-z_tilde|| with a<1, then its state difference is input-to-state stable. For a residual Euler cognitive update x_{k+1}=x_k+dt[f_theta(x_k,z_{k+1},u_k)-gamma x_k], a sufficient contraction condition is ||I+dt(J_f-gamma I)||_2 <= a<1, where J_f is the Jacobian of f_theta with respect to x and gamma>0 is leak. Under bounded input, the state has a geometric input-to-state bound.

## Key formulas

- $$z_{k+1}=P_\theta(z_k,u_k),\qquad x_{k+1}=x_k+\Delta t\left[f_\theta(x_k,z_{k+1},u_k)-\gamma x_k\right].$$
- $$\|P_\theta(z,u)-P_\theta(\tilde z,u)\|_2\leq\alpha\|z-\tilde z\|_2,\qquad 0\leq\alpha<1,$$
- $$\left\|I+\Delta t\left(J_f(x,z,u)-\gamma I\right)\right\|_2\leq a<1,\qquad J_f=\frac{\partial f_\theta}{\partial x}.$$
- $$\|x_k\|_2\leq a^k\|x_0\|_2+\sum_{j=0}^{k-1}a^{k-1-j}\left(L_z\|z_{j+1}\|_2+L_u\|u_j\|_2\right)\leq a^k\|x_0\|_2+\frac{L_zZ_{\max}+L_uU}{1-a}.$$

## Implementation notes

1. Integration point: implement a recurrent sequence model with a perception block producing z and a cognition block producing x; attach the prediction head to x or to the concatenation [z,x]. Use tanh or another 1-Lipschitz activation. Spectrally normalize every recurrent and input matrix in the perception block so its estimated state Jacobian norm is at most alpha. Implement cognition as a leaky residual update with explicit step size dt and leak gamma. 2. Pseudocode: compute z_next = Perception(z,u); compute f = CognitionMLP(x,z_next,u); compute x_next = x + dt*(f - gamma*x); compute y = Head(x_next); backpropagate task loss plus a penalty max(0,J_bound-a)^2. Estimate J_bound with one or two power iterations on Jacobian-vector products, or conservatively bound it using the product of spectral norms of the cognition layers. Reduce dt or increase gamma whenever the bound exceeds the target a. 3. Computed from the mechanism: the contraction inequalities, geometric decay rate a, and steady-state ISS bound (L_z Z_max + L_u U)/(1-a). Estimated empirically: local Jacobian norms, hidden-state norms, and perturbation decay from paired rollouts. 4. First cheap experiment: compare a vanilla GRU, a spectrally normalized GRU, and this modular model on Sequential MNIST or adding-memory MNIST with matched parameter counts and identical Adam settings. Sweep dt or gamma across the predicted boundary where the measured Jacobian norm reaches one. For two trajectories initialized with perturbation delta_0, the predicted signature is ||delta_k|| approximately proportional to a^k below the boundary and non-decaying or growing above it; fit the slope of log ||delta_k|| and test agreement with log(a) within 20%. Also verify that maximum hidden-state norm remains below the predicted ISS bound and that long-horizon loss degrades more slowly in the stable regime.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built and ran an ISS-constrained modular recurrent MVP with scalar Euler verification and a small sequence-sum comparison against a vanilla GRU. The mechanism manifested: the observed stability transition occurred at the predicted gamma=2.0 boundary, decay slopes matched log|q| with relative error below 6e-16, perceptual decay matched log(alpha) to numerical precision, and forced steady-state values agreed with the ISS bound within 0.66%. The modular model did not beat the GRU on this task, so the result supports the stability mechanism but not an accuracy win.

### Mechanism check

- Verdict: Built and ran an ISS-constrained modular recurrent MVP with scalar Euler verification and a small sequence-sum comparison against a vanilla GRU. The mechanism manifested: the observed stability transition occurred at the predicted gamma=2.0 boundary, decay slopes matched log|q| with relative error below 6e-16, perceptual decay matched log(alpha) to numerical precision, and forced steady-state values agreed with the ISS bound within 0.66%. The modular model did not beat the GRU on this task, so the result supports the stability mechanism but not an accuracy win.
- Confidence: 9/10
- Limitations: The mechanism verification is an analytically matched scalar linear Euler system rather than a full Jacobian sweep of a trained high-dimensional network. The learning comparison used one fixed seed, a short synthetic sequence-sum task, CPU execution after shared-CUDA allocation errors, and did not test Sequential MNIST, matched FLOPs, multiple seeds, or long-horizon generalization.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.04685
- Paired wins: 2/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/371/bench_report.json)
- [iss_bench.py](https://synthcore.org/code/371/iss_bench.py)
- [iss_experiment.py](https://synthcore.org/code/371/iss_experiment.py)
- [report.md](https://synthcore.org/code/371/report.md)
- [report_bench_2026-08-31T095443.md](https://synthcore.org/code/371/report_bench_2026-08-31T095443.md)
- [results.json](https://synthcore.org/code/371/results.json)
- [Download all files as ZIP](https://synthcore.org/download/371)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.