Unverified 2026

Pascal-simplex anti-collapse router

Implementation & benchmark of arXiv:2608.02707 — Discrete Unique Continuation on Simplex

Usefulness5/10

Difficulty6/10

Novelty9/10

Regularization Generalization Stability Moe routing Mlp Regularization Combinatorics Linear algebra Algebra

Source paper: Discrete Unique Continuation on Simplex arXiv:2608.02707 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an unconstrained collection of coefficients over degree-nR compositions by a signed simplex-indexed coefficient tensor satisfying the paper's local cancellation equations. Anchor the balanced coefficient and use the resulting discrete unique-continuation principle to prevent the learned tensor from collapsing onto a tiny set of compositions, while still allowing structured sparsity below the full simplex size. Apply the tensor to a signed residual feature mixture or to expert logits before a separate normalization step, since the coefficients are not required to be nonnegative.

Formulas

$$\Delta_{nR}^{(n)}=\left\{\alpha\in\mathbb{Z}_{\geq 0}^{n}:\alpha_{1}+\cdots+\alpha_{n}=nR\right\}$$

$$\sum_{i=1}^{n}g(\beta+e_{i})=0,\qquad\beta\in\Delta_{nR-1}^{(n)}$$

$$g(R,\ldots,R)\neq 0\quad\Longrightarrow\quad\left|\operatorname{supp}g\right|\geq c_{n}R^{\lceil n/2\rceil}$$

$$\mathcal{L}_{\mathrm{simplex}}(g)=\|Ag\|_{2}^{2}+\lambda_{\mathrm{anchor}}\big(g(R\mathbf{1}_{n})-\tau\big)^{2}+\lambda_{\mathrm{task}}\mathcal{L}_{\mathrm{task}}$$

Mathematical statement

For integers n>=2 and R>=1, define the composition simplex $\Delta_{nR}^{(n)}=\{\alpha\in\mathbb{Z}_{\ge 0}^n:\sum_i\alpha_i=nR\}$. Let $g:\Delta_{nR}^{(n)}\to\mathbb{R}$, and let $e_i$ be the i-th standard basis vector. The paper imposes the complete oriented-simplex relations $\sum_{i=1}^n g(\beta+e_i)=0$ for every $\beta\in\Delta_{nR-1}^{(n)}$. Its main theorem states that if the balanced coefficient is nonzero, $g(R\mathbf{1}_n)\neq 0$, then $|\operatorname{supp}g|\ge c_n R^{\lceil n/2\rceil}$, where $\operatorname{supp}g=\{\alpha:g(\alpha)\neq0\}$ and $c_n>0$ depends only on n. The construction uses factorial normalization to turn these discrete relations into one directional differential equation and a Pascal uncertainty principle; the transferable property is the lower bound on support under local linear cancellation. In the neural adaptation, $g$ is a learned signed coefficient vector, $Ag=0$ denotes all simplex relations, and $g(R\mathbf{1}_n)=\tau$ is an anchor with fixed nonzero $\tau$.

Implementation notes

(1) Integration point: create a small signed structured mixture inside an MoE router or MLP residual branch. Choose n=3 or 4 and R=2 or 3; enumerate every $\alpha\in\Delta_{nR}^{(n)}$, assign one trainable scalar $g_\alpha$, and map $\alpha$ to a feature/expert product such as $z_\alpha(x)=\prod_{i=1}^n h_i(x)^{\alpha_i}$ or to a learned expert embedding. The router output is $r(x)=\sum_\alpha g_\alpha z_\alpha(x)$, added as a residual to the ordinary MLP. Do not interpret g as probabilities; if expert probabilities are needed, use a separate softmax gate and let r be a signed correction. (2) Pseudocode: enumerate S=Delta(nR); construct rows of A by looping over beta in Delta(nR-1) and setting A[beta,beta+e_i]=1; initialize a random vector, project it with P=I-A^T(AA^T)^+A, then rescale so g[R,...,R]=tau; on each update set g <- P(g-lr*grad_g), followed by g <- g*(tau/g_balanced). Compute r(x)=sum_alpha g[alpha]*z_alpha(x), backpropagate the task loss, and optionally add a small penalty for numerical residual ||Ag||^2. (3) Computed from the paper: the exact linear relations, nullspace projection, balanced anchor, and predicted support threshold c_n R^{ceil(n/2)}. Estimated empirically: choose c_n from the smallest observed support over random nullspace vectors, and use thresholded support $|g_\alpha|>10^{-4}\|g\|_\infty$ to measure collapse. For scalability, replace the dense pseudoinverse by QR/SVD once per fixed n,R. (4) First experiment: train a 2-layer MLP on MNIST or CIFAR-10 with a 3-way, R=3 simplex residual branch, comparing unconstrained coefficients, L1 sparsity, and projected simplex coefficients at equal parameter count and FLOPs. Track validation accuracy, seed-to-seed variance, $\|Ag\|_2$, balanced coefficient magnitude, and effective support. The hypothesis is not necessarily higher peak accuracy; success is stable non-collapse with support above the predicted polynomial floor, lower variance, and equal or better accuracy than L1 at the same active-feature budget. An ablation removing the anchor must show that the support guarantee disappears.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.