Leading-Term Strand Router
Implementation & benchmark of arXiv:2608.27425 — Leading term strandings for webs
Source paper: Leading term strandings for webs arXiv:2608.27425 ⓘ · analyzed Aug 29, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Build a sparse neural mixing layer from colored directed strands rather than a dense all-to-all matrix. Feature channels are assigned ordered colors, local trivalent junctions conserve every color, and an edge width is the weighted sum of the colors carried by that edge; a differentiable penalty favors monotone, crossing-free routings that define a canonical leading term. This creates a structured routing prior that can be compared directly against dense attention and unconstrained sparse attention.
Formulas
Mathematical statement
A web is a directed trivalent graph whose edge weights satisfy, at every interior vertex v, $\sum_{i=1}^{3}\sigma_v(e_i)\ell_i\equiv0\pmod n$, where $\sigma_v(e)=1$ for an edge directed into v and $-1$ for an edge directed out, and $\ell_i\in\{1,\ldots,n-1\}$ is its weight. A strand system assigns binary incidence variables $\alpha_c^\vee(S(e))\in\{0,1\}$ indicating whether color $c\in\{1,\ldots,n-1\}$ traverses edge e. The paper proves color conservation at each trivalent vertex, $\sum_{i=1}^{3}\sigma_v(e_i)\alpha_c^\vee(S(e_i))=0$, and defines the edge weight by $\ell_i=\sum_{c=1}^{n-1}c\,\alpha_c^\vee(S(e_i))$, which implies the stronger integer balance $\sum_i\sigma_v(e_i)\ell_i=0$. For an ML layer, replace each strand by a feature-channel path and use a binary or soft incidence $a_{c,e}\in[0,1]$; define $w_e=\sum_c c a_{c,e}$. Enforce approximate conservation with $R_{\mathrm{cons}}=\sum_{v,c}(\sum_{e\ni v}\sigma_v(e)a_{c,e})^2$, and encourage a leading, monotone routing by penalizing color inversions between adjacent strands. The underlying leading-term principle is lexicographic selection of one tensor monomial among strandings, so a deterministic tie-breaker can select the routing with the smallest inversion count.
Implementation notes
1. Integration point: replace the value-mixing part of a small attention or graph-neural-network block. Let $X\in\mathbb R^{B\times T\times d}$ be token features and choose $C\le d$ ordered channel groups. A router predicts logits $r_{b,t,e,c}$ for a fixed sparse directed acyclic graph with source tokens, trivalent junctions, and destination tokens. Apply a sigmoid or straight-through Gumbel-sigmoid to obtain $a_{b,t,e,c}\in[0,1]$. The routed message on edge e is $m_e=a_e\odot W_e h_{\mathrm{src}(e)}$; destination nodes sum incoming messages and apply the usual output projection.
2. Pseudocode: initialize a fixed planar graph with maximum degree three; for each batch, compute $a=\operatorname{sigmoid}(r/\tau)$; set $w_e=\sum_{c=1}^{C}c a_{e,c}$; compute $R_{\mathrm{cons}}=\sum_{v,c}(\sum_{e\ni v}\sigma_v(e)a_{e,c})^2$; for every adjacent edge pair in the planar embedding, compute the inversion penalty in the displayed $R_{\mathrm{route}}$ formula; use loss $L=L_{\mathrm{task}}+\lambda_cR_{\mathrm{cons}}+\lambda_iR_{\mathrm{inv}}+\lambda_w\sum_e\operatorname{ReLU}(w_e-C)^2$. At inference, threshold $a_{e,c}$, remove edges with no active strand, and optionally retain only the lexicographically smallest routing among equal-score candidates.
3. The exact mathematical component is per-color flow conservation and the induced integer edge width. The planar graph, channel ordering, temperature schedule, penalty coefficients, and soft relaxation are engineering choices. Estimate routing sparsity directly as the fraction of inactive edges and measure conservation violation before thresholding and after thresholding.
4. First experiment: train a 6-layer, 8-head small Transformer on WikiText-2 or a synthetic long-range copy task. Compare dense attention, top-k attention with the same average number of edges, and this strand router at matched parameter count and FLOPs. Track validation perplexity, routing entropy, conservation residual, active-edge fraction, and loss versus wall-clock time. Success would be equal perplexity with at least 2x fewer active token-to-token edges, or faster loss descent and lower routing variance than unconstrained top-k attention; failure is worse perplexity at matched sparsity or a conservation penalty that collapses all strands to zero.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.