# Integral Master-Stability Coupling for Heterogeneous RNN Copies

- ID: 3144
- Canonical URL: https://synthcore.org/idea/3144/integral-master-stability-coupling-for-heterogeneous-rnn-cop
- API JSON: https://synthcore.org/api/idea/3144.json
- API Markdown: https://synthcore.org/api/idea/3144.md
- Verification status: unverified
- Source: [arXiv:2609.03698](https://arxiv.org/abs/2609.03698)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, training-dynamics, optimizer
- Math tags: control-theory, dynamical-systems, spectral-theory, linear-algebra
- Ratings: usefulness 7/10; difficulty 6/10; novelty 8/10

## Idea description

Run several heterogeneous recurrent or state-space network copies and couple their hidden states through a directed hypergraph with proportional and integral feedback. The proportional term contracts disagreement, while the integral state rejects persistent replica-specific biases that ordinary consensus coupling can only bound. This creates a controllable synchronization-versus-divergence transition rather than an unstructured regularization coefficient.

## Mathematical statement

Let x_i in R^n be the hidden state of replica i, f_i its vector field, g the coupling output, L^P and L^I directed hypergraph Laplacians, k_P and k_I their coupling gains, and z_i in R^n an integral controller state. Use dx_i/dt = f_i(x_i,mu_i) - k_P sum_j L^P_ij g(x_j) - z_i and dz_i/dt = k_I sum_j L^I_ij g(x_j). The synchronization manifold is x_1 = ... = x_N. For a transverse Laplacian mode with eigenvalues lambda_P and lambda_I, linearization around the synchronized trajectory gives M(lambda_P,lambda_I) = [[J_x f(x_bar,mu_bar) - lambda_P J_x g(0), -I_n], [lambda_I J_x g(0), 0_n]]. Here J_x f is the hidden-state Jacobian, J_x g is the coupling-output Jacobian, and I_n is the n-dimensional identity. The paper's condition is that the maximum Lyapunov exponent of every transverse M-mode is negative. For a discrete Euler implementation with step h, a local stability test is rho(I_2n + h M(lambda_P,lambda_I)) < 1 for every non-consensus mode. Exact synchronization requires the mismatch contribution J_mu f(x_bar,mu_bar) delta_mu to be constant in the transverse dynamics; the integral state can then cancel it.

## Key formulas

- $$\dot{x}_i=f_i(x_i,\mu_i)-k_P\sum_{j=1}^{N}L^P_{ij}g(x_j)-z_i,\qquad \dot{z}_i=k_I\sum_{j=1}^{N}L^I_{ij}g(x_j).$$
- $$\mathcal{M}(\lambda_P,\lambda_I)=\begin{bmatrix}J_xf(\bar{x},\bar{\mu})-\lambda_PJ_xg(0)&-I_n\\ \lambda_IJ_xg(0)&0_n\end{bmatrix},\qquad \chi_{\max}(\lambda_P,\lambda_I)<0.$$
- $$\rho\!\left(I_{2n}+h\mathcal{M}(\lambda_P,\lambda_I)\right)<1\quad\text{for all transverse modes}.$$
- $$\delta\dot{x}=\left[J_xf-\lambda_PJ_xg(0)\right]\delta x-\delta z+J_\mu f\,\delta\mu,\qquad \delta\dot{z}=\lambda_IJ_xg(0)\delta x.$$

## Implementation notes

1. Integration point: create N recurrent or state-space network copies with the same task architecture but deliberately heterogeneous parameters, such as different low-rank perturbations, quantization noise, or expert-specific biases. At every sequence step, construct a directed hypergraph over copies; ordinary directed edges are the MVP, while a hyperedge can aggregate the mean state of several tail copies and inject it into several head copies. Couple a selected hidden state h_i or a low-dimensional projection g(h_i), not the full parameter vector. 2. Pseudocode: initialize h_i and integral states z_i to zero; for each time step compute h_i_next = f_i(h_i,input); compute yP_i = sum_j Lp[i,j] g(h_j) and yI_i = sum_j Li[i,j] g(h_j); update z_i = z_i + h*kI*yI_i; update h_i = h_i + h*(f_i(h_i,input) - kP*yP_i - z_i); compute the task loss from the average or one designated synchronized copy and backpropagate through the coupled rollout. Start with g(h) equal to a linear projection and zero-row-sum Laplacians. 3. Computed from the paper: the transverse matrix and the requirement that its maximum Lyapunov exponent be negative. Estimated empirically: J_x f by Jacobian-vector products along validation sequences, Laplacian eigenvalues numerically, and the maximum Lyapunov exponent by QR-based tangent propagation. 4. First cheap experiment: compare five GRU copies on sequential MNIST or permuted-MNIST, with fixed per-copy hidden biases, against uncoupled training and proportional-only coupling. Sweep kP, kI, and Euler step h. Measure pairwise hidden disagreement and validation accuracy. The predicted signature is a sharp stability boundary: synchronization should persist only where the largest estimated transverse exponent is negative, and the observed divergence boundary should agree with chi_max = 0 within roughly 20%. With a constant injected bias, proportional-only coupling should leave nonzero steady disagreement, whereas PI coupling should drive disagreement toward zero; increasing kI should remove the residual bias until a discrete-time instability boundary is crossed.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.