Integral Master-Stability Coupling for Heterogeneous RNN Copies

1. Integration point: create N recurrent or state-space network copies with the same task architecture but deliberately heterogeneous parameters, such as different low-rank perturbations, quantization noise, or expert-specific biases. At every sequence step, construct a directed hypergraph over copies; ordinary directed edges are the MVP, while a hyperedge can aggregate the mean state of several tail copies and inject it into several head copies. Couple a selected hidden state h_i or a low-dimensional projection g(h_i), not the full parameter vector. 2. Pseudocode: initialize h_i and integral states z_i to zero; for each time step compute h_i_next = f_i(h_i,input); compute yP_i = sum_j Lp[i,j] g(h_j) and yI_i = sum_j Li[i,j] g(h_j); update z_i = z_i + h*kI*yI_i; update h_i = h_i + h*(f_i(h_i,input) - kP*yP_i - z_i); compute the task loss from the average or one designated synchronized copy and backpropagate through the coupled rollout. Start with g(h) equal to a linear projection and zero-row-sum Laplacians. 3. Computed from the paper: the transverse matrix and the requirement that its maximum Lyapunov exponent be negative. Estimated empirically: J_x f by Jacobian-vector products along validation sequences, Laplacian eigenvalues numerically, and the maximum Lyapunov exponent by QR-based tangent propagation. 4. First cheap experiment: compare five GRU copies on sequential MNIST or permuted-MNIST, with fixed per-copy hidden biases, against uncoupled training and proportional-only coupling. Sweep kP, kI, and Euler step h. Measure pairwise hidden disagreement and validation accuracy. The predicted signature is a sharp stability boundary: synchronization should persist only where the largest estimated transverse exponent is negative, and the observed divergence boundary should agree with chi_max = 0 within roughly 20%. With a constant injected bias, proportional-only coupling should leave nonzero steady disagreement, whereas PI coupling should drive disagreement toward zero; increasing kI should remove the residual bias until a discrete-time instability boundary is crossed.