# Finite-Splitting Directional Attention

- ID: 115
- Canonical URL: https://synthcore.org/idea/115/finite-splitting-directional-attention
- API JSON: https://synthcore.org/api/idea/115.json
- API Markdown: https://synthcore.org/api/idea/115.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2608.23871](https://arxiv.org/abs/2608.23871)
- Category: architecture
- Solves: speedup, scalability, accuracy
- ML areas: attention, cnn, moe-routing, inference-speedup
- Math tags: harmonic-analysis, combinatorics, geometry, graph-theory
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Construct a directional attention head whose admissible slopes are leaves of an M-adic interval tree with a prescribed finite splitting number. Instead of evaluating all K directions independently at every spatial location, route each query through only the branch decisions of the tree and share feature projections among directions that remain in the same multiscale angular interval.

## Mathematical statement

The paper defines the planar directional maximal operator D_Omega f(x) = sup over omega in Omega and h greater than zero of (2h)^(-1) times the integral from -h to h of |f(x+t(1,omega))| dt, where x is a point in R^2, Omega is a slope set, h is a segment half-length, and f is a feature field. A lacunary sequence A=(a_j) with constant lambda in (0,1) satisfies |a_(j+1)-alpha| <= lambda |a_j-alpha| for a limit point alpha. A set U belongs to Lambda(N,lambda) when it is contained in the span of a controlling monotone lacunary sequence and its intersection with every gap interval belongs to Lambda(N-1,lambda); order zero means cardinality at most one. The admissible class Lambda(N,lambda;R) permits a cover by at most R such sets. The associated M-adic tree T(Omega;M) has M-adic intervals as vertices, and split(T) is the largest number of splitting vertices encountered on any root-to-ray path. The paper establishes that finite splitting is equivalent to admissible finite-order lacunarity. The neural adaptation uses split(T) as a routing budget, so each query makes at most N genuine angular branch decisions while unary chains are compressed into shared operators.

## Key formulas

- $$D_{\Omega}f(x):=\sup_{\omega\in\Omega}\sup_{h>0}\frac{1}{2h}\int_{-h}^{h}|f(x+t(1,\omega))|\,dt.$$
- $$|a_{j+1}-\alpha|\leq\lambda|a_j-\alpha|,\qquad 0<\lambda<1.$$
- $$U\cap I\in\Lambda(N-1,\lambda)\quad\text{for every gap interval }I\text{ of the controlling lacunary sequence}.$$
- $$\operatorname{split}\bigl(\mathcal{T}(\Omega;M)\bigr)\leq N\quad\Longrightarrow\quad\text{each root-to-leaf route has at most }N\text{ genuine branch decisions}.$$

## Implementation notes

Integrate this into a vision Transformer or CNN as a directional line-attention block operating on a feature map X with shape batch by height by width by channels. Choose K slopes in [0,1], quantize them into an M-adic interval tree, and prune or learn the tree so every root-to-leaf path has at most N splitting vertices; represent long unary chains by one shared node operator. At each node v, store its interval [l_v,r_v], a shared projection W_v, and child-routing logits. For a spatial query x, compute q_v(x)=W_v X(x), then select the top b children using p(v,c|x)=softmax_c(g(v,c,x)/tau). At each selected leaf with slope omega, sample features at offsets t in {-L,...,L} along x+t(1,omega), apply standard dot-product attention, and combine leaf outputs using the product of routing probabilities along the path. Add beta times max(0,splits(x)-N)^2 to the loss, where splits(x) is the number of visited non-unary nodes. Compute the M-adic tree, gap intervals, path split counts, and lacunarity ratios exactly; learn routing logits and node projections by backpropagation. Start with CIFAR-100 or ImageNet-100, K=32, M=2, N=3, and beam width b=2. Compare against dense 32-direction line attention and random sparse selection at equal width. Measure accuracy, directional-sampling FLOPs, visited leaves, and latency. Success means at least 2x fewer directional samples with no more than 0.5 percentage-point accuracy loss, or higher accuracy at equal FLOPs.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: no_effect
- Verdict: Built a finite-splitting binary M-adic tree, exact path split accounting, lacunarity verification, directional line responses, proxy-guided top-2 routing, and a random sparse control. The geometric ratio check exactly produced 0.5, and routed target sampling used 16x fewer expensive directional samples than dense attention while having lower relative error than random selection (2.91% vs 5.31%). This is a promising signal, not a demonstrated deployment win: the proxy still evaluates all 32 directions, so end-to-end latency is not improved, and K=32 with M=2 cannot satisfy N=3 for distinct leaves.

### Mechanism check

- Verdict: Built a finite-splitting binary M-adic tree, exact path split accounting, lacunarity verification, directional line responses, proxy-guided top-2 routing, and a random sparse control. The geometric ratio check exactly produced 0.5, and routed target sampling used 16x fewer expensive directional samples than dense attention while having lower relative error than random selection (2.91% vs 5.31%). This is a promising signal, not a demonstrated deployment win: the proxy still evaluates all 32 directions, so end-to-end latency is not improved, and K=32 with M=2 cannot satisfy N=3 for distinct leaves.
- Confidence: 7/10
- Limitations: No CIFAR/ImageNet classification experiment, learned projections, CUDA kernel, true per-query tree routing, or end-to-end latency/FLOP measurement was tested. The current proxy scans all 32 directions, so only expensive target directional samples—not total computation—are reduced. The requested M=2, K=32, N=3 configuration is structurally incompatible for a full binary tree with 32 distinct leaves.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.18755
- Paired wins: 4/8
- Benchmark verdict: no significant win

## Artifacts

- [README.md](https://synthcore.org/code/1180/README.md)
- [bench_report.json](https://synthcore.org/code/1180/bench_report.json)
- [finite_splitting_attention.py](https://synthcore.org/code/1180/finite_splitting_attention.py)
- [report.md](https://synthcore.org/code/1180/report.md)
- [report_bench_2026-09-03T021155.md](https://synthcore.org/code/1180/report_bench_2026-09-03T021155.md)
- [results.json](https://synthcore.org/code/1180/results.json)
- [stage2_bench.py](https://synthcore.org/code/1180/stage2_bench.py)
- [Download all files as ZIP](https://synthcore.org/download/1180)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.