✗ Failed on benchmark 2026

Topology-Aware Streaming Jacobian Monitor

Implementation & benchmark of arXiv:2607.10893 — Streaming Contraction Certificates for Nonlinear Networks: Topology-Aware Data Sufficiency with Partial Observation

Usefulness8/10

Difficulty6/10

Novelty7/10

Architecture Stability Scalability Speedup Rnn Graph nn Ssm Training dynamics Graph theory Linear algebra Dynamical systems Control theory Statistics

Source paper: Streaming Contraction Certificates for Nonlinear Networks: Topology-Aware Data Sufficiency with Partial Observation arXiv:2607.10893 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

For a recurrent or graph neural network with known local connectivity, estimate each node's local Jacobian row using only graph neighbors rather than all hidden coordinates. Use the resulting sparse Jacobian both to compute a contraction certificate and to regularize training toward dynamically local interactions, reducing estimator variance and the number of samples required for reliable stability decisions.

Formulas

$$\frac{\partial F_i}{\partial h_j}=0\quad\text{whenever }(j,i)\notin\mathcal E,$$

$$\hat\beta=-\lambda_{\max}\!\left(\frac{\hat J+\hat J^\top}{2}\right),\qquad \beta_{\mathrm{cert}}=\hat\beta-\rho,$$

$$\#\text{estimated coefficients per row}=\mathcal O(d_i)\leq\mathcal O(d_{\max})\quad\text{instead of}\quad\mathcal O(N),$$

$$h_i^{k+1}=G_i\!\left(h_i^k,u_i^k,\{h_j^k:j\in\mathcal N_i\}\right),\qquad \mathcal N_i=\{j:(j,i)\in\mathcal E\}.$$

Mathematical statement

Let the hidden state be partitioned into node states $h_i$, with graph adjacency $A$ known. If $(j,i)\notin\mathcal E$, impose the structural constraint $\partial F_i/\partial h_j=0$. Each Jacobian row then has at most $d_i\leq d_{\max}$ unknown blocks instead of N possible blocks. Estimate the nonzero coefficients from a sliding window of local perturbation or finite-difference observations using recursive least squares, while never fitting forbidden coefficients. Assemble the sparse estimate $\hat J$, compute $\hat\beta=-\lambda_{\max}((\hat J+\hat J^\top)/2)$, and certify $\beta_{cert}=\hat\beta-\rho$. The predicted statistical advantage is that estimation error and required data scale with $d_{\max}$, not total node count N, assuming the topology is correct and local regressors are sufficiently exciting.

Implementation notes

1. Integration point: construct a node-wise recurrent network or graph state-space model, $h_i^{k+1}=G_i(h_i^k,u_i^k,\{h_j^k:j\in\mathcal N_i\})$, with fixed neighbor lists from the known graph. During inference, estimate the Jacobian from state increments and input increments. 2. Pseudocode: for each node i, form regressors $\phi_i^k=[h_i^k,u_i^k,h_j^k:j\in\mathcal N_i]$ and targets $r_i^k=(h_i^{k+1}-h_i^k)/\Delta t$. Update only coefficient blocks associated with $\mathcal N_i$ using recursive least squares, Huber loss, or median-of-means updates; set every non-neighbor block exactly to zero. Assemble $\hat J_k$, estimate the largest eigenvalue of its symmetric part using Lanczos, and set $\beta_{cert,k}=-\lambda_{max}((\hat J_k+\hat J_k^T)/2)-\rho_k$. 3. Computed versus estimated: the zero pattern and degree counts come from the topology prior; coefficients, residual variance, and uncertainty radius are estimated online. If the graph may be wrong, maintain a small monitored escape set of candidate edges rather than deleting all nonedges permanently. 4. First cheap experiment: compare dense versus topology-constrained estimators on 20-node synthetic graph RNNs with degree 2, 4, 8, and 19, using only 2-4 observed nodes and Laplace disturbances. Keep the underlying dynamics identical and vary window size W. The predicted signature is that the topology-aware certificate crosses zero with fewer samples, with required sample count scaling approximately with $d_{max}$ rather than N. Record the smallest W for which false-certification probability is below 5%; the certification-time advantage should increase with N/dmax and disappear on deliberately randomized incorrect graphs.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a topology-constrained Jacobian monitor MVP using masked least squares and the symmetric-part contraction certificate. The numerical math checks confirmed skew-symmetric invariance to 1.3e-15 and the diagonal-shift prediction (-0.370000 observed vs -0.370 predicted); coefficient error stayed near the predicted noise floor (2.68e-6–3.24e-6 observed vs 3.125e-6 predicted). In the synthetic benchmark, sparse MSE was 2–10x lower than dense, certification reached 95% at W=12 for d=2/8 and W=24 for d=16 versus W=48 dense, while an incorrect topology increased MSE from 4.38e-6 to 1.56e-2 and produced an overconfident invalid certificate.

Agent confidence: 8/10

Baseline: Dense least-squares Jacobian: at N=32,d=2,W=12 MSE=0.0388 and positive-certificate rate=0%; at W=48 MSE=8.31e-5 and rate=100%; at N=24,d=4,W=80 MSE=2.19e-5.
Idea: Masked topology-aware least squares: at N=32,d=2,W=12 MSE=1.36e-5 and positive-certificate rate=100%; at N=24,d=4,W=80 MSE=3.45e-6; correct topology at N=32,d=4,W=48 MSE=4.38e-6, versus 1.56e-2 for a randomized mask.

Limitations: The implementation uses batch masked least squares rather than true recursive least squares or a sliding-window streaming update. It tests linear synthetic dynamics with full state observations and small repeated trials, not a trained nonlinear RNN/GNN, partial observations, uncertainty radius rho, Lanczos estimation, or runtime/FLOP savings.

How to run: OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 /home/maxwelhelp/main/bin/python3 topology_monitor.py > results.json

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.10893) Sep 2, 2026

Ahead of the baseline (−30.1%) but not statistically significant (p=0.0555, wins 6 / 8 paired seeds).

Benchmark: Dynamics forecasting
Model: rnn_small
Paired seeds: 8
Baseline mean: 0.0021
Idea mean: 0.0015
Effect (Δ): -0.0006 (−30.1%; negative = idea better)
Wins: 6 / 8 paired seeds
p-value: 0.0555 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±27.1%
Mechanism: Not confirmed ✗
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

Only the dynamics track was tested. The sparse model used a custom masked recurrent implementation rather than an exact block-sparse parameterization, the mechanism signature was not confirmed, uncertainty radius and online recursive least squares were not implemented, and the observed task-metric improvement was not statistically significant at p<0.05.

How to run: python3 bench_topology.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Aug 30, 2026 latest Failed ✗

Agent confidence: 9/10

Built and evaluated a topology-masked recurrent intervention on the structurally matched dynamics track using the benchmark rnn_small task and train_model path. Test MSE improved from 0.0021058194 to 0.0014712255 with 6/8 paired wins, but the permutation p-value was 0.0555, so the required significant-win criterion was not met. The mechanism signature failed because the implementation still uses a dense hidden projection before masking its outputs, so forbidden-coordinate dependencies remain in the measured preactivations.

Baseline: dynamics/rnn_small; best lr=0.006, epochs=12; 8-seed MSE mean=0.0021058194, std=0.0003894366; sweep means lr=0.001: 0.0042987791, lr=0.003: 0.0034615941, lr=0.006: 0.0018282316
Idea: MaskedGRU ring degree=2; same best baseline lr/epochs; 8-seed MSE mean=0.0014712255, std=0.0003449246; paired delta=-0.0006345939, 6/8 wins, permutation p=0.0555, verdict=no significant win

Benchmark result