# Small-gain certified modular network

- ID: 776
- Canonical URL: https://synthcore.org/idea/776/small-gain-certified-modular-network
- API JSON: https://synthcore.org/api/idea/776.json
- API Markdown: https://synthcore.org/api/idea/776.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.11982](https://arxiv.org/abs/2607.11982)
- Category: architecture
- Solves: stability, scalability
- ML areas: mlp, rnn, graph-nn, regularization
- Math tags: control-theory, dynamical-systems, linear-algebra, spectral-theory
- Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10

## Idea description

Partition a neural network into independently trained or independently monitored modules and constrain their cross-module interaction gain using a compositional contraction certificate. This enables stable deep modular MLPs, graph blocks, or recurrent modules without estimating the full network Jacobian, while providing an explicit coupling threshold for when the architecture loses contraction.

## Mathematical statement

Let modules A and B have local contraction margins \(\beta_A>0\) and \(\beta_B>0\), measured from their local input-output Jacobians or discrete-time residual maps. Let \(\gamma\geq0\) bound the cross-module gain, for example the maximum of the cross Jacobian norms. The compositional certificate is \(\beta_{\mathrm{net}}=(\beta_A+\beta_B)/2-\sqrt{(\beta_A-\beta_B)^2/4+\gamma^2}\). The network is certified contractive when \(\beta_{\mathrm{net}}>0\), equivalently \(\gamma<\sqrt{\beta_A\beta_B}\). For a block \(z_i^+=F_i(z_i)+C_i(z_j)\), estimate \(\beta_i=1-\sup\|\partial F_i/\partial z_i\|_2\) and bound \(\gamma\) using cross-module Jacobian norms. The property doing the work is compositional small-gain stability: local certificates and coupling gains predict global contraction without forming the full joint Jacobian.

## Key formulas

- $$\beta_{\mathrm{net}}=\frac{\beta_A+\beta_B}{2}-\sqrt{\frac{(\beta_A-\beta_B)^2}{4}+\gamma^2}.$$
- $$\beta_{\mathrm{net}}>0\quad\Longleftrightarrow\quad \gamma<\sqrt{\beta_A\beta_B},\qquad \gamma_{\mathrm{warn}}=\sqrt{\beta_A\beta_B}.$$
- $$\beta_i=1-\sup_{z_i}\left\|\frac{\partial F_i}{\partial z_i}\right\|_2,\qquad \gamma\geq\max\left(\sup\left\|\frac{\partial C_A}{\partial z_B}\right\|_2,\sup\left\|\frac{\partial C_B}{\partial z_A}\right\|_2\right).$$

## Implementation notes

Build a two-stream network with states \(z_A\) and \(z_B\). Each stream contains a local block \(F_i(z_i)\) and a cross interaction \(C_i(z_j)\), such as a linear projection followed by a 1-Lipschitz activation. At every K training steps, estimate each local Jacobian norm using several random JVPs restricted to that module, taking \(\beta_i=1-\) the largest observed norm. Estimate \(\gamma\) from cross-module JVPs and multiply the estimate by a safety factor of 1.1-1.3. Compute \(\beta_{\mathrm{net}}\) from the displayed formula. If it falls below target margin \(\beta_{\mathrm{target}}\), reduce a cross-gate parameter by \(\mathrm{clip}(\beta_{\mathrm{net}}/\beta_{\mathrm{target}},0,1)\), or project cross weights to \(\gamma_{\mathrm{allowed}}=q\sqrt{\beta_A\beta_B}\) with \(q<1\). A differentiable alternative is the penalty \(\max(0,\gamma^2-q^2\beta_A\beta_B)^2\). The paper supplies the certificate and threshold; empirical JVPs estimate unknown suprema and should be stress-tested with more directions. First test a two-branch MLP on MNIST or CIFAR-10, sweeping the cross-gate across values around the predicted threshold. Compare unconstrained training, full-Jacobian spectral normalization, and the local certificate controller under clean and spike-corrupted inputs. Measure perturbation decay across depth and training divergence. The predicted signature is a transition near \(\gamma_{\mathrm{warn}}=\sqrt{\beta_A\beta_B}\): below it, perturbations decay geometrically at approximately rate \(1-\beta_{\mathrm{net}}\); above it, the measured global Jacobian norm should approach or exceed one and long-horizon perturbations should stop decaying.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: harms
- Verdict: Built and numerically verified the two-module small-gain certificate and projected cross-gain controller in small_gain_experiment.py. The predicted transition gamma_warn=sqrt(beta_A beta_B)=0.244949 matched the measured contraction boundary across equal and asymmetric margins; predicted decay ratios matched observed ratios within 3.46e-7, and zero coupling exactly matched the local norm prediction. The controller kept the norm at 0.976 and reduced 30-step perturbations to 0.480, while the unconstrained gain produced norm 1.305 and perturbation growth to 2921, so the proposed mechanism clearly manifested in the toy setting.

### Mechanism check

- Verdict: Built and numerically verified the two-module small-gain certificate and projected cross-gain controller in small_gain_experiment.py. The predicted transition gamma_warn=sqrt(beta_A beta_B)=0.244949 matched the measured contraction boundary across equal and asymmetric margins; predicted decay ratios matched observed ratios within 3.46e-7, and zero coupling exactly matched the local norm prediction. The controller kept the norm at 0.976 and reduced 30-step perturbations to 0.480, while the unconstrained gain produced norm 1.305 and perturbation growth to 2921, so the proposed mechanism clearly manifested in the toy setting.
- Confidence: 9/10
- Limitations: This is an exact low-dimensional linearized two-module toy experiment, not a trained MNIST/CIFAR modular neural network. It does not test empirical JVP estimation under nonlinear saturation, finite-sample certification errors, optimization quality, compute overhead, or robustness on real data.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 0/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/147/bench_report.json)
- [report.md](https://synthcore.org/code/147/report.md)
- [report_bench_2026-08-30T174936.md](https://synthcore.org/code/147/report_bench_2026-08-30T174936.md)
- [results.json](https://synthcore.org/code/147/results.json)
- [small_gain_bench.py](https://synthcore.org/code/147/small_gain_bench.py)
- [small_gain_experiment.py](https://synthcore.org/code/147/small_gain_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/147)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.