✗ Failed on benchmark 2026

Small-gain certified modular network

Implementation & benchmark of arXiv:2607.11982 — Contraction Certification from Streaming Data: Wasserstein Robustness and Compositional Stability for Interconnected Nonlinear System

Usefulness8/10

Difficulty6/10

Novelty7/10

Architecture Stability Scalability Mlp Rnn Graph nn Regularization Control theory Dynamical systems Linear algebra Spectral theory

Source paper: Contraction Certification from Streaming Data: Wasserstein Robustness and Compositional Stability for Interconnected Nonlinear System arXiv:2607.11982 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Partition a neural network into independently trained or independently monitored modules and constrain their cross-module interaction gain using a compositional contraction certificate. This enables stable deep modular MLPs, graph blocks, or recurrent modules without estimating the full network Jacobian, while providing an explicit coupling threshold for when the architecture loses contraction.

Formulas

$$\beta_{\mathrm{net}}=\frac{\beta_A+\beta_B}{2}-\sqrt{\frac{(\beta_A-\beta_B)^2}{4}+\gamma^2}.$$

$$\beta_{\mathrm{net}}>0\quad\Longleftrightarrow\quad \gamma<\sqrt{\beta_A\beta_B},\qquad \gamma_{\mathrm{warn}}=\sqrt{\beta_A\beta_B}.$$

$$\beta_i=1-\sup_{z_i}\left\|\frac{\partial F_i}{\partial z_i}\right\|_2,\qquad \gamma\geq\max\left(\sup\left\|\frac{\partial C_A}{\partial z_B}\right\|_2,\sup\left\|\frac{\partial C_B}{\partial z_A}\right\|_2\right).$$

Mathematical statement

Let modules A and B have local contraction margins $\beta_A>0$ and $\beta_B>0$, measured from their local input-output Jacobians or discrete-time residual maps. Let $\gamma\geq0$ bound the cross-module gain, for example the maximum of the cross Jacobian norms. The compositional certificate is $\beta_{\mathrm{net}}=(\beta_A+\beta_B)/2-\sqrt{(\beta_A-\beta_B)^2/4+\gamma^2}$. The network is certified contractive when $\beta_{\mathrm{net}}>0$, equivalently $\gamma<\sqrt{\beta_A\beta_B}$. For a block $z_i^+=F_i(z_i)+C_i(z_j)$, estimate $\beta_i=1-\sup\|\partial F_i/\partial z_i\|_2$ and bound $\gamma$ using cross-module Jacobian norms. The property doing the work is compositional small-gain stability: local certificates and coupling gains predict global contraction without forming the full joint Jacobian.

Implementation notes

Build a two-stream network with states $z_A$ and $z_B$. Each stream contains a local block $F_i(z_i)$ and a cross interaction $C_i(z_j)$, such as a linear projection followed by a 1-Lipschitz activation. At every K training steps, estimate each local Jacobian norm using several random JVPs restricted to that module, taking $\beta_i=1-$ the largest observed norm. Estimate $\gamma$ from cross-module JVPs and multiply the estimate by a safety factor of 1.1-1.3. Compute $\beta_{\mathrm{net}}$ from the displayed formula. If it falls below target margin $\beta_{\mathrm{target}}$, reduce a cross-gate parameter by $\mathrm{clip}(\beta_{\mathrm{net}}/\beta_{\mathrm{target}},0,1)$, or project cross weights to $\gamma_{\mathrm{allowed}}=q\sqrt{\beta_A\beta_B}$ with $q<1$. A differentiable alternative is the penalty $\max(0,\gamma^2-q^2\beta_A\beta_B)^2$. The paper supplies the certificate and threshold; empirical JVPs estimate unknown suprema and should be stress-tested with more directions. First test a two-branch MLP on MNIST or CIFAR-10, sweeping the cross-gate across values around the predicted threshold. Compare unconstrained training, full-Jacobian spectral normalization, and the local certificate controller under clean and spike-corrupted inputs. Measure perturbation decay across depth and training divergence. The predicted signature is a transition near $\gamma_{\mathrm{warn}}=\sqrt{\beta_A\beta_B}$: below it, perturbations decay geometrically at approximately rate $1-\beta_{\mathrm{net}}$; above it, the measured global Jacobian norm should approach or exceed one and long-horizon perturbations should stop decaying.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built and numerically verified the two-module small-gain certificate and projected cross-gain controller in small_gain_experiment.py. The predicted transition gamma_warn=sqrt(beta_A beta_B)=0.244949 matched the measured contraction boundary across equal and asymmetric margins; predicted decay ratios matched observed ratios within 3.46e-7, and zero coupling exactly matched the local norm prediction. The controller kept the norm at 0.976 and reduced 30-step perturbations to 0.480, while the unconstrained gain produced norm 1.305 and perturbation growth to 2921, so the proposed mechanism clearly manifested in the toy setting.

Agent confidence: 9/10

Baseline: Unconstrained cross gain converged to gamma=0.5525; global Jacobian norm=1.3048; 30-step perturbation magnitude=2921.35.
Idea: Projected gain gamma=0.22045 (0.9*gamma_warn); global Jacobian norm=0.9761; 30-step perturbation magnitude=0.4803. Boundary prediction 0.244949 matched the observed transition interval [0.242499, 0.247398].

Limitations: This is an exact low-dimensional linearized two-module toy experiment, not a trained MNIST/CIFAR modular neural network. It does not test empirical JVP estimation under nonlinear saturation, finite-sample certification errors, optimization quality, compute overhead, or robustness on real data.

How to run: python3 small_gain_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.11982) Sep 2, 2026

Significantly WORSE than the tuned baseline (+159.5%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0005
Idea mean: 0.0014
Effect (Δ): 0.0009 (+159.5%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 0 configs

Limitations:

Only the built-in dynamics track was tested; no CIFAR, tabular, sequence, spike-corruption robustness, full-Jacobian spectral-normalization comparator, longer training budget, or alternative certified penalty/controller implementation was evaluated. The local certificate used recurrent weight spectral norms as a conservative proxy rather than exhaustive state-space Jacobian suprema.

How to run: python3 small_gain_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 30, 2026 latest Failed ✗

Agent confidence: 10/10

Built the small-gain certified two-stream recurrent network on the structurally matched dynamics track, using identical data, architecture, epochs, batch size, and paired seeds for baseline and idea. The tuned baseline achieved test MSE 0.0005448 versus 0.0014136 for the idea, with paired delta +0.0008688 and permutation p=0.0081, so the idea was significantly worse. The behavior signature measured negative local margins (beta_A=-0.2352, beta_B=-0.0777), an observed input-output Jacobian norm of 0.9518, and confirmed=false; therefore the contraction certificate's assumptions were not met by the trained model.

Baseline: Full 8-seed test MSE mean=0.0005448212, std=0.0002417870; best config lr=0.006, gate=0.5.
Idea: Full 8-seed test MSE mean=0.0014136387, std=0.0008909184; selected config lr=0.006, gate=0.75, q=0.85.
Mechanism signature: map[beta_A:-0.2352112532 beta_B:-0.0777136087 confirmed:false gamma_after_projection:8.5e-05 observed_input_output_jacobian_norm:0.9518151879 predicted_global_bound_1_minus_beta_net:NaN because beta_A and beta_B were negative projection_scale:7.31e-05]
Paired delta: idea - baseline = +0.0008688174 MSE; idea wins 0/8 pairs; permutation p=0.0081.

Benchmark result