# Contractive 9-Way Hierarchical Positional Encoding

- ID: 1617
- Canonical URL: https://synthcore.org/idea/1617/contractive-9-way-hierarchical-positional-encoding
- API JSON: https://synthcore.org/api/idea/1617.json
- API Markdown: https://synthcore.org/api/idea/1617.md
- Verification status: unverified
- Source: [arXiv:2608.00022](https://arxiv.org/abs/2608.00022)
- Category: architecture
- Solves: scalability, memory, accuracy
- ML areas: embedding, transformer, graph-nn
- Math tags: geometry, metric-geometry, topology, dynamical-systems
- Ratings: usefulness 5/10; difficulty 4/10; novelty 4/10

## Idea description

Replace a large flat positional-embedding table with a recursively decoded nine-way address whose child transformations contract coordinates by exactly 1/3. Encode an input position using features attached to the address prefix at several depths, guaranteeing that increasing depth produces a geometrically localized representation and that an infinite valid address cannot ambiguously represent two distinct points. This is especially suitable for 2D vision tokens, maps, point clouds, or long-context layouts.

## Mathematical statement

The paper defines a child-cell decode on a compact convex triangle \(\Omega\subset b_{\mathrm{oct}}\) by \(p\mapsto \tfrac{1}{3}p+\mathrm{offset}(C_i)\), where \(p\in\Omega\) is a local 2D coordinate and \(C_i\), \(i\in\{0,\ldots,8\}\), is one of nine children. For an address prefix \((a_0,\ldots,a_L)\), \(S_L\) is the corresponding nested compact convex cell. The paper proves \(S_0\supset S_1\supset\cdots\) and \(\mathrm{diam}(S_L)\le d_0(1/3)^L\to0\), where \(d_0\) is the diameter of the root cell. Cantor's intersection theorem then gives \(\bigcap_{L=0}^{\infty}S_L=\{p^*\}\), so every infinite valid address determines one unique limit point. In the neural adaptation, use the same affine recursion for normalized coordinates, cache one learned feature per prefix or per level, and concatenate or sum features from levels \(0\) through \(L\). The contraction gives a known spatial resolution bound \(d_0 3^{-L}\), while shared child offsets reduce the number of independent positional parameters.

## Key formulas

- $$p\;\mapsto\;\tfrac{1}{3}\,p+\mathrm{offset}(C_i)$$
- $$S_{0}\supset S_{1}\supset S_{2}\supset\cdots,\qquad\mathrm{diam}(S_{L})\;\leq\;d_{0}\left(\tfrac{1}{3}\right)^{L}\;\to\;0$$
- $$\bigcap_{L=0}^{\infty}S_{L}\;=\;\{p^{*}\}$$
- $$h(x)=\mathrm{MLP}\!\left(\left[\,E_0[a_0],E_1[a_0,a_1],\ldots,E_L[a_0,\ldots,a_L],\;x-\hat{x}_L\,\right]\right),\qquad \hat{x}_{\ell+1}=\tfrac13\hat{x}_{\ell}+o_{a_{\ell+1}}$$

## Implementation notes

Integrate this at the positional-encoding or input-embedding stage for a 2D image, point-cloud, map, or document-layout transformer. Normalize every coordinate \(x\) into a root triangle or square split into a small fixed number of triangles; for an MVP, use one triangular chart and nine child offsets \(o_i\) arranged on a 3-by-3 refinement pattern. First precompute a deterministic address function: set \(p_0=x\), and for each level \(\ell\), choose the child index \(a_\ell\) containing the normalized point, then update the local coordinate with the inverse affine map \(p_{\ell+1}=3(p_\ell-o_{a_\ell})\). Store either a learned embedding table for each prefix, a hash table keyed by the integer address, or shared level-wise child embeddings. At inference, reconstruct the cell representative with \(\hat{x}_0=0\), \(\hat{x}_{\ell+1}=\tfrac13\hat{x}_\ell+o_{a_{\ell+1}}\), and optionally append the residual \(x-\hat{x}_L\) to an MLP. The neural encoding can be \(z(x)=\sum_{\ell=0}^{L}W_\ell E_\ell[a_{0:\ell}]\), followed by the normal token projection. The paper supplies the exact contraction and resolution guarantee; estimate only child membership, hash collisions, and embedding widths empirically. Start with CIFAR-100 or a 256-by-256 NeRF-style coordinate regression task, comparing sinusoidal/Fourier features and a flat learned grid at equal parameter count. Measure validation accuracy or coordinate MSE, memory, and throughput while increasing resolution. Success is equal-or-better accuracy with fewer positional parameters, smoother extrapolation to unseen resolutions, and error decreasing approximately as the maximum cell diameter \(d_0 3^{-L}\) shrinks.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.