Behavior-Gap Clustered Neural Controllers

Use this at the level of recurrent cells, SSM layers, or MoE experts. During a calibration pass, feed every module the same batch of excitation sequences, record its hidden-state or output sequence, and build a block Hankel matrix with L consecutive time windows. Reset hidden states between sequences so that the measured response is comparable. For each module, compute B_i=col(H_L(u_i),H_L(y_i)), take a rank-r SVD B_i=U_i Sigma_i V_i^T, and set P_i=U_iU_i^T. Cluster modules greedily using ||P_i-P_leader||_2; share the leader's optimizer moments, controller, or low-rank adapter with followers. Recompute clusters every E epochs and use hysteresis so a module must violate epsilon for two recalculations before moving. The paper supplies the projector-gap construction; rank r, epsilon, mu_min, and finite-horizon responses must be estimated empirically. First experiment: train 8-16 small GRUs on sequential MNIST or permuted-MNIST with varied recurrent spectral radii. Compare independent Adam states against four behavior-gap clusters sharing optimizer moments or adapters. Plot validation loss, hidden-state norms, trajectory error, and mu_i versus gap. The predicted signature is a sharp increase in error and exploding hidden norms when gap exceeds epsilon or mu_i approaches zero. Repeat for L=16,32,64; the transition should move with the observation horizon.