Contracting Countable-Branch Router

1. Integration point: implement this as a recurrent latent block or mixture-of-experts router. Let \(h_t\in\mathbb{R}^d\) be the hidden state, let \(r_\theta(h_t,u_t)\) produce branch logits, and let each branch \(g_J(h,u)\) be a small MLP residual refinement. Train with top-\(K\) routing, while allowing new branch IDs to be added when the router repeatedly assigns an overflow bucket. 2. Pseudocode: compute logits and select \(J_t\); evaluate \(h_{t+1}=g_{J_t}(h_t,u_t)\); estimate the largest singular value of the branch Jacobian with one or two power iterations; add \(\lambda\max(0,\widehat{\sigma}_{\max}(Dg_{J_t})-s)^2\) to the loss; optionally add a distortion penalty comparing Jacobians at two nearby hidden states, \(\|Dg_J(h+\delta)-Dg_J(h)\|_F/(\|Dg_J(h)\|_F+\epsilon)\le L\|\delta\|^\alpha\). 3. Computed versus estimated: the contraction composition bound and \(s^n\) decay are computed from the imposed target \(s\); Jacobian norms, branch usage probabilities, and distortion constants are estimated on minibatch states. The symbolic-shift interpretation is exact only for separated, consistently invertible branches, so measure branch overlap and report it. 4. First cheap experiment: use a 1D or 2D noisy switching system and compare a GRU, ordinary top-\(K\) MoE-RNN, and this router on long-horizon prediction. Initialize two copies of a hidden state with perturbation \(\delta_0\), force the same routing sequence, and measure \(\delta_n/\delta_0\). The quantitative prediction is an exponential envelope \(\delta_n/\delta_0\le s^n\); fitting the slope of \(\log\delta_n\) against \(n\) should give a value no larger than \(\log s\), within roughly 20% when Jacobian estimates are accurate. Sweep the target contraction: exploding hidden-state sensitivity should appear when the measured maximum branch Jacobian crosses 1, while overly small \(s\) should reduce branch-conditioned expressivity. Compare top-\(K\) truncations by held-out rare-mode likelihood and test whether adding branches improves coverage without violating the contraction bound.