△ Mechanism confirmed, baseline not beaten 2026

Trace-Canonical Modular Blocks

Implementation & benchmark of arXiv:2607.08841 — Partially-Commutative Polynomial Optimization

Usefulness6/10

Difficulty4/10

Novelty6/10

Architecture Scalability Generalization Transformer Mlp Architecture Algebra Combinatorics Graph theory

Source paper: Partially-Commutative Polynomial Optimization arXiv:2607.08841 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Represent a modular neural block as a word over primitive operators and identify operators that commute because they act on disjoint tensor supports. Canonicalize every equivalent word into one representative, allowing architecture search, caching, and parameter sharing to operate on equivalence classes rather than redundant layer orderings.

Formulas

$$\mathcal W=\langle X\rangle,\qquad w=x_{i_1}x_{i_2}\cdots x_{i_k},\qquad |w|=k.$$

$$\mathcal W_{\mathcal G}=\mathcal W/\sim,\qquad u\sim v\ \Longleftrightarrow\ u\text{ can be transformed into }v\text{ by swaps }xy\leftrightarrow yx\text{ for }(x,y)\in E.$$

$$[w]=\{v\in\mathcal W:v\sim w\},\qquad f_{x_1}\circ f_{x_2}=f_{x_2}\circ f_{x_1}\ \text{whenever }(x_1,x_2)\in E.$$

$$\operatorname{canon}(w)=\min_{\prec}\{v:v\in[w]\},$$

Mathematical statement

The paper starts with the free monoid $\mathcal W=\langle X\rangle$, whose elements are words $w=x_{i_1}\cdots x_{i_k}$ over an alphabet $X$ of operators; multiplication is concatenation and $|w|$ is word length. A commutation graph $\mathcal G=(X,E)$ specifies which distinct letters commute: if $(x,y)\in E$, impose $xy=yx$. The partially-commutative monoid is the quotient $\mathcal W_{\mathcal G}=\mathcal W/\sim$, where $u\sim v$ when one can transform $u$ into $v$ by swapping adjacent commuting letters. The equivalence class is $[w]=\{v:v\sim w\}$, and all representatives have the same multiset of letters and degree. For neural operators $f_x$, define support $S_x$ as the tensor coordinates, channels, heads, or feature groups read and written by $f_x$. Declare $(x,y)\in E$ only when $S_x\cap S_y=\varnothing$, or when an analytically verified identity $f_x\circ f_y=f_y\circ f_x$ holds. Then swapping adjacent commuting operators preserves the network function exactly, whereas noncommuting pairs retain their order.

Implementation notes

Integrate this at the architecture-search and module-caching level for a small library of operators such as grouped linear maps, per-head attention, independent feature mixers, residual scalings, and channelwise nonlinearities. Give every primitive operator $x$ a support set $S_x$, for example a set of channel groups or attention heads. Build the commutation graph with an edge $(x,y)$ when the supports are disjoint; do not infer commutation merely from similar tensor shapes. For a candidate sequence $w=[x_1,\ldots,x_k]$, repeatedly scan adjacent pairs and swap $x_i,x_{i+1}$ if $(x_i,x_{i+1})\in E$ and the pair is inverted under a fixed total operator order $\prec$. Stop when no swap is possible; this produces a deterministic canonical representative $\operatorname{canon}(w)$. Use the canonical string as the key for weight sharing, memoized forward graphs, and NAS deduplication. During validation, compare outputs of every proposed swap numerically on random tensors; only allow exact or tolerance-certified swaps. The mathematical part is the equivalence relation and its correctness under verified commuting supports; the empirical part is measuring whether deduplication changes search coverage. First test on a 12-layer grouped-MLP or multi-head toy Transformer on CIFAR-100 or WikiText-2, comparing random architecture search with and without canonicalization at the same evaluation budget. Success means substantially fewer unique candidates, lower search wall-clock, and equal or better validation loss at fixed evaluations.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 9/10

Built a support-aware trace canonicalizer with deterministic adjacent commuting swaps, exact swap validation, and memoized forward evaluation. Commuting swaps had maximum output error 0.0, while noncommuting swaps changed outputs with median maximum difference 0.1464. On 500 deliberately redundant proposals, canonicalization reduced 500 strings to 50 classes with identical losses and a measured 3.79x forward-evaluation speedup.

Agent confidence: 9/10

Baseline: 500 strings evaluated; mean loss 1.49713; 5.6328 seconds
Idea: 50 canonical classes evaluated; mean loss 1.49713; 1.4845 seconds; 90% fewer evaluations; maximum loss disagreement 0.0; 3.794x measured speedup

Limitations: This is a small synthetic grouped-vector experiment, not CIFAR-100, WikiText-2, a Transformer, or a full NAS loop. Redundant proposals were intentionally generated from shared traces to test the claimed caching phenomenon; speed measurements are single-run timings and do not separately report canonicalization overhead.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2607.08841) Sep 4, 2026

No significant difference vs. the tuned baseline (−-0.0%, p=1, wins 0 / 8 paired seeds).

Benchmark: Sequence modeling
Model: modular_sequence_local
Paired seeds: 8
Baseline mean: 0.1317
Idea mean: 0.1317
Effect (Δ): 0 (−-0.0%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 1 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±0.0%
Mechanism: Confirmed ✓
Practical effect: No effect
Baseline tuning: swept over 3 configs

Limitations:

This benchmark tests task-level training rather than a full architecture-search wall-clock/cache evaluation; it does not measure NAS deduplication throughput or memory savings. The local modular model is not the built-in transformer_tiny, though it is a matched sequence-level modular architecture and both systems use identical training conditions. The mechanism signature evaluates one fixed proposed trace and one noncommuting alternative per trained seed.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 4, 2026 latest Failed ✗

Agent confidence: 10/10

Built a local trace-canonical modular sequence model and evaluated it against the identical noncanonical baseline on the structurally matched sequence forecast track. The mechanism signature was confirmed quantitatively: observed maximum commuting-swap error was 0.0, while the median noncommuting difference was 1.85068. However, baseline and idea had exactly identical test MSE (0.1316999), paired delta 0.0, and permutation p-value 1.0; canonicalization changes representation/cache keys, not the trained function or learning outcome here.

Baseline: sequence / modular_sequence_local; best lr=0.003; full 8-seed test MSE mean=0.1316999011, std=0.0170872853
Idea: best lr=0.003; full 8-seed test MSE mean=0.1316999011, std=0.0170872853; paired delta=0.0; permutation p=1.0; observed commuting max error=0.0; observed noncommuting median difference=1.85068297

Benchmark result