Unverified 2026

Signed-base fractal positional features

Implementation & benchmark of arXiv:2607.29327 — Fractal random variables defined by probability distributions of digits of their $G_2$-representation having two bases with different signs

Usefulness6/10

Difficulty4/10

Novelty7/10

Architecture Accuracy Sample efficiency Transformer Attention Embedding Probability Dynamical systems Measure theory

Source paper: Fractal random variables defined by probability distributions of digits of their $G_2$-representation having two bases with different signs arXiv:2607.29327 ⓘ · analyzed Aug 31, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Augment standard Transformer positional embeddings with coordinates generated by the paper's signed-base digit expansion. Previous binary digits determine the sign and scale of later contributions, while a two-state Markov chain controls correlations between digits. This supplies multiscale positional structure using a small number of transition and base parameters.

Formulas

$$x=\delta_{a_1}+\sum_{k=2}^{\infty}\left(\delta_{a_k}\prod_{j=1}^{k-1}g_{a_j}\right),\qquad \delta_{a_k}=a_k g_{1-a_k},\qquad g_1=g_0-1$$

$$x=\frac{a_1}{2}+\sum_{k=2}^{\infty}\frac{a_k(-1)^{\sigma_k}}{2^k},\qquad \sigma_k=\sum_{i=1}^{k-1}a_i\qquad (g_0=1/2)$$

$$P(a_{k+1}=v\mid a_k=u)=p_{uv},\qquad \sum_{v\in\{0,1\}}p_{uv}=1$$

$$\phi_{m,r}(t)=\left[\sin(2\pi b_m x_{m,r}(t)),\cos(2\pi b_m x_{m,r}(t))\right]$$

Mathematical statement

The paper sets g_0 in [1/2,1) and g_1=g_0-1, and maps binary digits a_1,a_2,... to x=sum over k of a_k g_(1-a_k) times the product over i<k of g_(a_i). Here a_k is the kth binary digit, g_(1-a_k) is its additive coefficient, and the product is the signed contraction induced by previous digits. Theorem 1 states that every x in [0,g_0] has at least one such representation. At g_0=1/2, the expansion becomes x=a_1/2+sum over k>=2 of a_k(-1)^(sigma_k)/2^k, where sigma_k=sum over i<k of a_i; thus every one digit flips the sign of all later scales. A Markov digit source uses p_uv=P(a_(k+1)=v given a_k=u), for u,v in {0,1}. The transferable asset is an explicit bounded multiscale coordinate with controllable digit correlations.

Implementation notes

Integrate this at the Transformer input, either by replacing learned absolute position embeddings or by adding a residual fractal positional branch. For each feature channel m, choose g0[m] in [0.5,0.95], depth K, initial digit probability pi[m], transition matrix P[m], and frequency b[m]. Generate one digit sequence for every position t and channel m, compute the truncated recurrence, then project sine/cosine features into d_model dimensions. Pseudocode: initialize a[1] from Bernoulli(pi); set x=a[1]*g_(1-a[1]) and prod=g_(a[1]); for k=2,...,K sample a[k] from softmax(P[a[k-1],:]), set x += a[k]*g_(1-a[k])*prod, and set prod *= g_(a[k]); concatenate sin(2*pi*b*x) and cos(2*pi*b*x) across m and apply a learned linear projection. The recurrence, bounded coordinate, and Markov parameterization come from the paper; P, pi, g0, frequencies, and projection weights are tuned or learned. Start with fixed deterministic digits for reproducibility, then test resampling as augmentation. Train a six-layer Transformer on WikiText-2 at sequence lengths 128 and 512, comparing learned and sinusoidal positions at equal FLOPs. Test iid digits against p00=p11=0.9. Success means lower validation perplexity, faster loss descent, or better length extrapolation without added attention cost.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.