# Contracting Countable-Branch Router

- ID: 1636
- Canonical URL: https://synthcore.org/idea/1636/contracting-countable-branch-router
- API JSON: https://synthcore.org/api/idea/1636.json
- API Markdown: https://synthcore.org/api/idea/1636.md
- Verification status: unverified
- Source: [arXiv:2608.00398](https://arxiv.org/abs/2608.00398)
- Category: architecture
- Solves: stability, scalability, sample-efficiency
- ML areas: moe-routing, rnn, world-model, regularization
- Math tags: dynamical-systems, geometry, probability, information-theory
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Construct a routed neural state update from a collection of branch maps whose inverse-style refinement operators are uniformly contractive. The discrete routing sequence acts as an expandable symbolic code, while contraction makes the continuous state associated with a long routing history insensitive to initialization and earlier perturbations. Use a finite active top-\(K\) set during training, but retain an expandable branch table so the model can represent increasingly complex or rare modes.

## Mathematical statement

Let \(\mathcal{J}\) be a finite or countable branch index set and let \(\psi_J:[-1,1]\to J\) be the inverse of an expanding branch \(\varphi_J:J\to[-1,1]\). The paper assumes uniform contraction: for some \(0<s<1\), \(|\psi_J(x)-\psi_J(y)|\le s|x-y|\) for every branch \(J\). Therefore any length-\(n\) composition satisfies \(|(\psi_{J_1}\circ\cdots\circ\psi_{J_n})(x)-(\psi_{J_1}\circ\cdots\circ\psi_{J_n})(y)|\le s^n|x-y|\). A symbolic sequence \((J_1,J_2,\ldots)\in\mathbb{N}^{\mathbb{N}}\) determines a unique limiting point when the nested branch intervals are used, and the branch shift corresponds to the expanding map on the invariant set. In the neural adaptation, \(x\) is a scalar or vector latent state, \(g_J\) is a learned branch refinement map, and \(\|Dg_J(x)\|_2\le s\) is the directly enforced contraction certificate. The derivative distortion condition requires branch derivatives to vary Hölder-continuously relative to their minimum magnitude; this can be approximated by a Jacobian-variation penalty.

## Key formulas

- $$\left|\psi_J(x)-\psi_J(y)\right|\le s\left|x-y\right|,\qquad 0<s<1,$$
- $$\left|\psi_{J_1}\circ\cdots\circ\psi_{J_n}(x)-\psi_{J_1}\circ\cdots\circ\psi_{J_n}(y)\right|\le s^n|x-y|,$$
- $$\left|\psi_J'(x)-\psi_J'(y)\right|\le L\,\inf_{z\in[-1,1]}|\psi_J'(z)|\,|x-y|^{\alpha},$$
- $$\widehat{x}_{t+1}=g_{J_t}(\widehat{x}_t),\qquad J_t\sim\operatorname{TopK}(\operatorname{softmax}(r_\theta(\widehat{x}_t,u_t))),\qquad \|Dg_J(x)\|_2\le s<1.$$

## Implementation notes

1. Integration point: implement this as a recurrent latent block or mixture-of-experts router. Let \(h_t\in\mathbb{R}^d\) be the hidden state, let \(r_\theta(h_t,u_t)\) produce branch logits, and let each branch \(g_J(h,u)\) be a small MLP residual refinement. Train with top-\(K\) routing, while allowing new branch IDs to be added when the router repeatedly assigns an overflow bucket. 2. Pseudocode: compute logits and select \(J_t\); evaluate \(h_{t+1}=g_{J_t}(h_t,u_t)\); estimate the largest singular value of the branch Jacobian with one or two power iterations; add \(\lambda\max(0,\widehat{\sigma}_{\max}(Dg_{J_t})-s)^2\) to the loss; optionally add a distortion penalty comparing Jacobians at two nearby hidden states, \(\|Dg_J(h+\delta)-Dg_J(h)\|_F/(\|Dg_J(h)\|_F+\epsilon)\le L\|\delta\|^\alpha\). 3. Computed versus estimated: the contraction composition bound and \(s^n\) decay are computed from the imposed target \(s\); Jacobian norms, branch usage probabilities, and distortion constants are estimated on minibatch states. The symbolic-shift interpretation is exact only for separated, consistently invertible branches, so measure branch overlap and report it. 4. First cheap experiment: use a 1D or 2D noisy switching system and compare a GRU, ordinary top-\(K\) MoE-RNN, and this router on long-horizon prediction. Initialize two copies of a hidden state with perturbation \(\delta_0\), force the same routing sequence, and measure \(\delta_n/\delta_0\). The quantitative prediction is an exponential envelope \(\delta_n/\delta_0\le s^n\); fitting the slope of \(\log\delta_n\) against \(n\) should give a value no larger than \(\log s\), within roughly 20% when Jacobian estimates are accurate. Sweep the target contraction: exploding hidden-state sensitivity should appear when the measured maximum branch Jacobian crosses 1, while overly small \(s\) should reduce branch-conditioned expressivity. Compare top-\(K\) truncations by held-out rare-mode likelihood and test whether adding branches improves coverage without violating the contraction bound.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.