Projector-Gap Trust Region for Shared Updates

Place the mechanism after gradient aggregation and before the optimizer update for each cluster of GRU, SSM, or MoE parameters. All cluster members receive a proposed common Adam or SGD update, but a scalar trust coefficient alpha_k scales that update before it is committed. Maintain calibration trajectories and a rank-r orthonormal basis U_i for every module; form P_i=U_iU_i^T every E training steps. Calculate the proposed update Delta theta_k, predict the updated basis using a finite-difference Jacobian-vector estimate or by applying the update to shadow copies, and compute the maximum follower-leader projector gap. Set alpha_k=min(1,epsilon/(max_gap+delta)); update theta_i with alpha_k Delta theta_k. If the measured post-update gap exceeds epsilon, halve alpha and retry. The paper supplies the projector-gap metric and its graph-subspace interpretation; behavior projectors, their parameter sensitivity, epsilon, and delta are implementation estimates. First experiment: train a four-layer stack of heterogeneous GRUs on adding and copy-memory tasks, sharing one update per two modules. Compare ordinary shared updates with the gap trust region at equal wall-clock cost while sweeping learning rate and epsilon. Record loss spikes, gradient norms, hidden-state norms, actual gaps, and clipped-update frequency. The quantitative prediction is that loss-spike probability rises sharply when the proposed gap exceeds epsilon, while adaptive alpha keeps measured gaps below epsilon and increases the largest stable learning rate. Test reproducibility across seeds and horizons L.