Local connectivity balance shapes population dynamics in random recurrent networks
arXiv:2608.30008
2026
Dynamics
1 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper identifies local excitatory-inhibitory balance as a dynamical control parameter that is invisible to the usual connectivity spectrum but strongly changes recurrent-state behavior through suppression of self-generated feedback. This suggests a practical initialization and regularization strategy for RNNs and state-space sequence models: control each neuron's incoming row sum independently of global spectral radius, rather than relying only on spectral normalization. The effect should be especially measurable with saturating or sub-linear non-odd activations, where the paper predicts that balance can change growth, chaos, and effective dynamical dimension. A small-scale ablation over balance strength and activation type can directly test whether balanced recurrent networks train more stably or retain richer dynamics.
Ideas from this paper
Unverified
2026
Initialize or regularize recurrent matrices so that each unit receives an approximately cancelling sum of positive and negative weights, while keeping the global variance and spectral radius fixed. Sweep a continuous balance parameter instead of imposing balance blindly, because the paper predicts qualitatively different behavior for saturating, sub-linear, and odd nonlinearities.
Useful6/10
Difficulty4/10
Novelty6/10