Reaction-Closed Sparse Routing

(1) Integrate this into the router of a sparse Mixture-of-Experts feed-forward layer or into channel gating in a dynamically sparse MLP. Let there be $n$ experts. Store a fixed dependency table containing rules $(I_j,P_j)$; for example, $I_j=\{2,5\}$ and $P_j=\{7\}$ means expert 7 may be activated only after experts 2 and 5 are active. The router computes ordinary logits $z\in\mathbb{R}^n$, but its binary support is constrained by the closure mechanism. (2) For each token, initialize $A$ with the top-$k$ router logits. Repeatedly compute $A_{new}=A\cup\bigcup_{j:I_j\subseteq A}P_j$ until $A_{new}=A$. Score and execute experts in the resulting support, then renormalize gate probabilities over that support. If the closure exceeds the compute budget $k_{max}$, retain the highest-logit members of the closure and mark omitted members as deferred candidates. During warmup, permit closure expansions but disable pruning. Afterwards, prune experts whose exponentially averaged gate mass is below threshold $\tau$, creating a trajectory-dependent downward transition. (3) Compute closure masks exactly from $(I_j,P_j)$; estimate router logits, gate masses, support churn, and pruning thresholds empirically. The dependency graph can be manually specified from layer hierarchy or sampled as a sparse DAG. (4) First experiment: train a 12-layer, 32-expert Switch-style Transformer on WikiText-103 or C4 with $k=2$. Compare ordinary top-k routing, load-balanced top-k routing, and reaction-closed routing at equal activated-expert FLOPs. Measure validation perplexity, permanently dead experts, adjacent-batch support Jaccard churn, token overflow rate, closure overhead, and wall-clock throughput. The hypothesis is fewer dead experts and lower support churn during warmup at equal perplexity or improved perplexity; failure is indicated by increased routing overhead or worse perplexity at fixed FLOPs. A cheaper pilot uses a two-layer sparse MLP on CIFAR-100 with a manually defined channel hierarchy.