# Pascal-simplex anti-collapse router

- ID: 1720
- Canonical URL: https://synthcore.org/idea/1720/pascal-simplex-anti-collapse-router
- API JSON: https://synthcore.org/api/idea/1720.json
- API Markdown: https://synthcore.org/api/idea/1720.md
- Verification status: unverified
- Source: [arXiv:2608.02707](https://arxiv.org/abs/2608.02707)
- Category: regularization
- Solves: generalization, stability
- ML areas: moe-routing, mlp, regularization
- Math tags: combinatorics, linear-algebra, algebra
- Ratings: usefulness 5/10; difficulty 6/10; novelty 9/10

## Idea description

Replace an unconstrained collection of coefficients over degree-nR compositions by a signed simplex-indexed coefficient tensor satisfying the paper's local cancellation equations. Anchor the balanced coefficient and use the resulting discrete unique-continuation principle to prevent the learned tensor from collapsing onto a tiny set of compositions, while still allowing structured sparsity below the full simplex size. Apply the tensor to a signed residual feature mixture or to expert logits before a separate normalization step, since the coefficients are not required to be nonnegative.

## Mathematical statement

For integers n>=2 and R>=1, define the composition simplex \(\Delta_{nR}^{(n)}=\{\alpha\in\mathbb{Z}_{\ge 0}^n:\sum_i\alpha_i=nR\}\). Let \(g:\Delta_{nR}^{(n)}\to\mathbb{R}\), and let \(e_i\) be the i-th standard basis vector. The paper imposes the complete oriented-simplex relations \(\sum_{i=1}^n g(\beta+e_i)=0\) for every \(\beta\in\Delta_{nR-1}^{(n)}\). Its main theorem states that if the balanced coefficient is nonzero, \(g(R\mathbf{1}_n)\neq 0\), then \(|\operatorname{supp}g|\ge c_n R^{\lceil n/2\rceil}\), where \(\operatorname{supp}g=\{\alpha:g(\alpha)\neq0\}\) and \(c_n>0\) depends only on n. The construction uses factorial normalization to turn these discrete relations into one directional differential equation and a Pascal uncertainty principle; the transferable property is the lower bound on support under local linear cancellation. In the neural adaptation, \(g\) is a learned signed coefficient vector, \(Ag=0\) denotes all simplex relations, and \(g(R\mathbf{1}_n)=\tau\) is an anchor with fixed nonzero \(\tau\).

## Key formulas

- $$\Delta_{nR}^{(n)}=\left\{\alpha\in\mathbb{Z}_{\geq 0}^{n}:\alpha_{1}+\cdots+\alpha_{n}=nR\right\}$$
- $$\sum_{i=1}^{n}g(\beta+e_{i})=0,\qquad\beta\in\Delta_{nR-1}^{(n)}$$
- $$g(R,\ldots,R)\neq 0\quad\Longrightarrow\quad\left|\operatorname{supp}g\right|\geq c_{n}R^{\lceil n/2\rceil}$$
- $$\mathcal{L}_{\mathrm{simplex}}(g)=\|Ag\|_{2}^{2}+\lambda_{\mathrm{anchor}}\big(g(R\mathbf{1}_{n})-\tau\big)^{2}+\lambda_{\mathrm{task}}\mathcal{L}_{\mathrm{task}}$$

## Implementation notes

(1) Integration point: create a small signed structured mixture inside an MoE router or MLP residual branch. Choose n=3 or 4 and R=2 or 3; enumerate every \(\alpha\in\Delta_{nR}^{(n)}\), assign one trainable scalar \(g_\alpha\), and map \(\alpha\) to a feature/expert product such as \(z_\alpha(x)=\prod_{i=1}^n h_i(x)^{\alpha_i}\) or to a learned expert embedding. The router output is \(r(x)=\sum_\alpha g_\alpha z_\alpha(x)\), added as a residual to the ordinary MLP. Do not interpret g as probabilities; if expert probabilities are needed, use a separate softmax gate and let r be a signed correction. (2) Pseudocode: enumerate S=Delta(nR); construct rows of A by looping over beta in Delta(nR-1) and setting A[beta,beta+e_i]=1; initialize a random vector, project it with P=I-A^T(AA^T)^+A, then rescale so g[R,...,R]=tau; on each update set g <- P(g-lr*grad_g), followed by g <- g*(tau/g_balanced). Compute r(x)=sum_alpha g[alpha]*z_alpha(x), backpropagate the task loss, and optionally add a small penalty for numerical residual ||Ag||^2. (3) Computed from the paper: the exact linear relations, nullspace projection, balanced anchor, and predicted support threshold c_n R^{ceil(n/2)}. Estimated empirically: choose c_n from the smallest observed support over random nullspace vectors, and use thresholded support \(|g_\alpha|>10^{-4}\|g\|_\infty\) to measure collapse. For scalability, replace the dense pseudoinverse by QR/SVD once per fixed n,R. (4) First experiment: train a 2-layer MLP on MNIST or CIFAR-10 with a 3-way, R=3 simplex residual branch, comparing unconstrained coefficients, L1 sparsity, and projected simplex coefficients at equal parameter count and FLOPs. Track validation accuracy, seed-to-seed variance, \(\|Ag\|_2\), balanced coefficient magnitude, and effective support. The hypothesis is not necessarily higher peak accuracy; success is stable non-collapse with support above the predicted polynomial floor, lower variance, and equal or better accuracy than L1 at the same active-feature budget. An ablation removing the anchor must show that the support guarantee disappears.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.