Folded cluster feature mixer
Implementation & benchmark of arXiv:2608.00963 — Cluster Algebras for Bosonic Plethysm
Source paper: Cluster Algebras for Bosonic Plethysm arXiv:2608.00963 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Build a neural feature-mixing block from a sparse exchange matrix and a fixed sequence of cluster-style mutations. Transpose-related channels share parameters through orbit tying, while positive multiplicative-additive updates are evaluated with log-sum-exp. The block is a concrete alternative to a dense MLP projection and can be tested at equal parameter count and compute.
Formulas
Mathematical statement
The paper defines an exchange matrix B=(b_uv) indexed by mutable variables. Its mutable principal part is skew-symmetrizable: d_u b_uv = -d_v b_vu, where d_u and d_v are positive transpose-orbit sizes. The paper uses mutation indices v_{q,u}=(i+u,q-u), with h=N-i, in the order (v_{h,0},...,v_{h,h};v_{h-1,0},...,v_{h-1,h-1};...;v_{1,0},v_{1,1}). For a positive cluster variable x_k and exchange-column coefficients b_uk, use the exchange update x'_k = (product_u x_u^[b_uk]_+ + product_u x_u^[-b_uk]_+) / x_k, where [r]_+=max(r,0). With z_u=log(x_u), the numerically stable neural version is z'_k = LSE(sum_u [b_uk]_+ z_u, sum_u [-b_uk]_+ z_u) - z_k, where LSE(a,c)=log(exp(a)+exp(c)). The sparse rows displayed in equations (89)-(90) indicate that each scheduled mutation interacts with only a small local neighborhood. The neural adaptation uses this locality for cheap mixing and uses the transpose involution to tie parameters or channel states across paired indices.
Implementation notes
1. Integration point: replace the first dense linear layer inside a Transformer feed-forward block or a small tabular/CIFAR MLP. Let the hidden width be n. Choose a sparse integer exchange matrix B with n mutable channels and optionally several frozen channels representing learned positive biases. Create transpose pairs using a permutation tau and enforce d_u B[u,v] = -d_v B[v,u] when constructing B. 2. Pseudocode: given hidden vector h, set z=log(softplus(h)+epsilon). For k in the fixed mutation schedule, compute p=sum_u max(B[u,k],0)*z[u] and q=sum_u max(-B[u,k],0)*z[u] over only nonzero sparse neighbors; update z[k]=logaddexp(p,q)-z[k]. Tie paired channels by either sharing their input projection and mutation coefficients or averaging their post-update states according to tau. After one or more sweeps, return y=h+alpha*W_out*softplus(z), followed by normalization. If dynamic quiver updates are desired, update B after mutating k: for i,j not equal to k, B[i,j]=B[i,j]+max(B[i,k],0)*max(B[k,j],0)-max(-B[i,k],0)*max(-B[k,j],0), then negate row k and column k. 3. Taken directly from the mathematics: the sparse signs, orbit pairing, symmetrizers, exchange update, and mutation order. Estimate empirically: alpha, epsilon, number of sweeps, frozen-channel initialization, and output projection. 4. First experiment: train a two-hidden-layer CIFAR-10 MLP and a 2-layer character-level language model. Compare the folded mixer with a dense MLP, a random sparse mixer, and a diagonal-plus-low-rank mixer at matched hidden width and training FLOPs. Measure validation accuracy or perplexity, mixer parameters, wall-clock time, gradient norm variance, activation range, and sensitivity to depth. A positive result is equal or better validation performance with at least 2x fewer mixer parameters, lower activation outlier frequency, and stable optimization without gradient clipping.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.