△ Mechanism confirmed, baseline not beaten 2026

Co-Prime Virtual-Aperture Attention

Implementation & benchmark of arXiv:2609.01979 — ISAC with Co-Prime Arrays: Virtual-Aperture Sensing and uplink downlink communications

Usefulness7/10

Difficulty5/10

Novelty7/10

Architecture Speedup Memory Scalability Attention Transformer Inference speedup Memory Combinatorics Linear algebra Number theory

Source paper: ISAC with Co-Prime Arrays: Virtual-Aperture Sensing and uplink downlink communications arXiv:2609.01979 ⓘ · analyzed Sep 3, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace dense or single-dilation sparse attention with two sequential sparse attention stages whose offsets form co-prime arithmetic progressions. The first stage mixes tokens separated by multiples of M2, the second by multiples of M1; their composition reaches virtual offsets mM2+nM1, providing many structured long-range interactions from only M1+M2-1 physical offset families. Use causal masking and residual connections so the module can replace a standard transformer attention block without changing sequence length.

Formulas

$$M_s=\left|\mathcal{P}_{\rm t}\cup\mathcal{P}_{\rm r}\right|=M_1+M_2-1.$$

$$\mathcal{V}_{\rm CPA}=\{(mM_2+nM_1)d\mid m=0,\ldots,M_1-1,\;n=0,\ldots,M_2-1\}.$$

$$\operatorname{Attn}_{D}(X)_i=\sum_{\delta\in D_i}\operatorname{softmax}_{\delta}\!\left(\frac{q_i k_{i+\delta}^{\top}}{\sqrt{d_h}}+b_{\delta}\right)v_{i+\delta},\qquad D_t=\{mM_2\}_{m=0}^{M_1-1},\quad D_r=\{nM_1\}_{n=0}^{M_2-1}.$$

$$H=\operatorname{Attn}_{D_r}\!\left(\operatorname{Attn}_{D_t}(X)\right),\qquad Y=X+W_oH,\quad \Delta_{\rm virtual}=mM_2+nM_1.$$

Mathematical statement

The paper defines two physical sensing position sets with coprime positive integers M1 and M2. Their shared origin gives Ms=M1+M2-1 distinct physical positions, while the cross-sum virtual array is V_CPA={(mM2+nM1)d | m=0,...,M1-1, n=0,...,M2-1}, where d is the base spacing. We transplant the cross-sum structure to token offsets by setting d=1 and using Dt={mM2} and Dr={nM1}. For a sequence x0,...,xL-1, let SparseAttn_D(X)i attend only to valid indices j=i+delta for delta in D, with learned relative bias b_delta and standard scaled dot-product softmax weights. The proposed block is H=SparseAttn_Dr(SparseAttn_Dt(X)), followed by a residual output projection. Its effective receptive-field offsets are sums delta_t+delta_r=mM2+nM1, the virtual positions from the paper. Coprimality gcd(M1,M2)=1 is the structural condition intended to reduce repeated offsets; boundary clipping and accidental duplicate offsets are handled by deduplicating the offset set or retaining multiple computational paths.

Implementation notes

Integrate this as an alternative to one standard self-attention block in a decoder-only transformer. Choose small coprime integers, for example M1=3 and M2=4, and define signed offsets for bidirectional models or nonnegative offsets for causal models. For each layer, construct Dt={0,M2,2M2,...,(M1-1)M2} and Dr={0,M1,2M1,...,(M2-1)M1}; for every batch, head, and token i, gather only positions i+delta that are inside the sequence and respect the causal mask. Apply ordinary scaled dot-product attention independently in stage t and stage r, using separate Q/K/V projections or shared projections, then add a residual output projection and the normal MLP. Compact pseudocode is: Dt=offsets(M2,M1); Dr=offsets(M1,M2); Z=sparse_attn(X,Dt); H=sparse_attn(Z,Dr); Y=X+Wo(H). The paper supplies the physical-count formula and cross-sum construction; no mathematical quantities need to be estimated during training. Enumerate all offset pairs (dt,dr) to measure the effective virtual-offset set and deduplicate sums, while separately counting multiply-adds, memory reads, and kernel launches. Start with a 6-layer, 256-dimensional character-level or WikiText-2 transformer at sequence length 1024. Compare dense attention, local-window attention with the same edge count, standard single-dilation attention, and the co-prime block. Report validation perplexity at equal training FLOPs, wall-clock throughput, peak activation memory, and the fraction of tokens receiving information from offsets above 128. The success signal is equal-or-lower perplexity with at least 2x lower attention cost, or substantially larger long-range coverage at the same edge budget. Ablate coprime pairs against non-coprime pairs such as (2,4), testing whether virtual-offset count and quality degrade as predicted.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a causal co-prime virtual-aperture attention MVP with M1=3 and M2=4, plus sparse and dense baselines. The math check reproduced 6 physical offset families, 12 unique virtual offsets, exact graph reachability, and fewer virtual offsets for the non-coprime (2,4) control. On a seeded lag-17 copy task, CPA reached 99.83% accuracy and loss 0.0201 at the same 7 attention edges per token as the sparse baseline, which reached 4.80% accuracy and loss 2.9467; this is a clear toy-task signal for long-range coverage, but not a speed win.

Agent confidence: 8/10

Baseline: single sparse: loss 2.9467, accuracy 0.0480, 3.195 s; dense: loss 2.7531, accuracy 0.1445, 0.788 s
Idea: CPA: loss 0.0201, accuracy 0.9983, 3.406 s; 7 attention edges/token, 12 virtual offsets, 50% of virtual offsets >8

Limitations: This is a tiny synthetic one-layer experiment with sequence length 24, one coprime pair, one non-coprime math control, and 180 optimization steps. It does not test language-model perplexity, multi-layer transformers, realistic data, equal FLOPs including projection costs, optimized sparse kernels, activation memory, or generalization beyond the fixed lag-17 task; measured wall-clock time was not improved.

How to run: python3 coprime_attention_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2609.01979) Sep 3, 2026

Ahead of the baseline (−4.3%) but not statistically significant (p=0.3787, wins 5 / 8 paired seeds).

Benchmark: Sequence modeling
Model: transformer_tiny
Paired seeds: 8
Baseline mean: 0.1462
Idea mean: 0.1398
Effect (Δ): -0.0063 (−4.3%; negative = idea better)
Wins: 5 / 8 paired seeds
p-value: 0.3787 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±11.1%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

The built-in sequence forecasting track was tested only; no language modeling, longer contexts, optimized sparse kernels, wall-clock throughput, activation-memory measurement, or additional coprime pairs were evaluated. The benchmark harness README was initially absent at the requested path but was subsequently located and used.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 3, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented Co-Prime Virtual-Aperture Attention on the structurally matched sequence forecasting track using an end-to-end trained two-stage sparse-attention transformer, with dense causal attention as the matched baseline. The math check held: 6 physical offsets and 12 virtual offsets for (3,4), versus 6 for the non-coprime (2,4) control. The idea improved mean MSE from 0.146195 to 0.139847, but the paired permutation test was not significant (p=0.3787), so this is not an established benchmark win; the trained-model gradient signature confirmed all predicted virtual offsets.

Baseline: Dense causal transformer, best lr=0.003, mean test MSE 0.146195, std 0.023272
Idea: Two-stage CPA sparse transformer, M1=3 M2=4, best lr=0.003, mean test MSE 0.139847, std 0.011513
Idea wins: 5
Mechanism signature confirmed: true
N pairs: 8
Paired delta mean: -0.0063477931544184685
Permutation p value: 0.3787

Benchmark result