# Quadratic-Chirp Positional Rotation

- ID: 2797
- Canonical URL: https://synthcore.org/idea/2797/quadratic-chirp-positional-rotation
- API JSON: https://synthcore.org/api/idea/2797.json
- API Markdown: https://synthcore.org/api/idea/2797.md
- Verification status: unverified
- Source: [arXiv:2608.28917](https://arxiv.org/abs/2608.28917)
- Category: architecture
- Solves: accuracy, generalization, stability
- ML areas: attention, transformer, embedding
- Math tags: harmonic-analysis, dynamical-systems, spectral-theory, number-theory
- Ratings: usefulness 6/10; difficulty 4/10; novelty 6/10

## Idea description

Replace the linear phase progression in a positional encoding or rotary attention mechanism with a deterministic quadratic phase. The resulting position signal is generated by an irrational rotation with linearly changing increments, and the paper proves that its infinite diffraction measure is purely absolutely continuous, suggesting disorder-like spectral coverage without random sampling.

## Mathematical statement

The central sequence is $V(n)=\lambda\cos\big(2\pi(x_{1}+nx_{2}+\frac{n(n-1)}{2}\alpha)\big)$, where $n\in\mathbb Z$ is position, $\lambda\neq 0$ is amplitude, and $x_{1},x_{2}\in\mathbb R$ are phase offsets. The coefficient $\alpha\notin\mathbb Q$ is the quadratic phase increment. The phase is $\phi_n=x_{1}+nx_{2}+n(n-1)\alpha/2$, so its first difference is $\phi_{n+1}-\phi_n=x_{2}+n\alpha$ and its second difference is the constant irrational rotation $\alpha$. The paper's main theorem states that the diffraction measure associated with this deterministic sequence is purely absolutely continuous for all such $\lambda,\alpha,x_1,x_2$; informally, its spectral mass has no Bragg point component despite the sequence being deterministic. For a neural implementation, use the complex positional rotation $r_n=\exp(2\pi i\phi_n)$ and apply it independently to 2D feature pairs. For positions $m,n$, the relative rotation is $r_m\overline r_n=\exp(2\pi i[(m-n)x_2+(m(m-1)-n(n-1))\alpha/2])$, whose quadratic term prevents a single fixed frequency from dominating long contexts. The theorem concerns the infinite diffraction object; finite-model validation must estimate spectral flatness empirically rather than assume exact finite-length absolute continuity.

## Key formulas

- $$V(n)=\lambda\cos\Big(2\pi\Big(x_{1}+nx_{2}+\frac{n(n-1)}{2}\alpha\Big)\Big),\qquad \lambda\neq 0,\ \alpha\notin\mathbb Q.$$
- $$\phi_n=x_1+n x_2+\frac{n(n-1)}{2}\alpha,\qquad \phi_{n+1}-\phi_n=x_2+n\alpha,\qquad \phi_{n+2}-2\phi_{n+1}+\phi_n=\alpha.$$
- $$r_n=e^{2\pi i\phi_n},\qquad r_m\overline{r_n}=e^{2\pi i\left((m-n)x_2+\frac{m(m-1)-n(n-1)}{2}\alpha\right)}.$$
- $$\widehat V_L(k)=\sum_{n=0}^{L-1}V(n)e^{-2\pi i kn/L},\qquad \mathrm{SF}(V;L)=\frac{\exp\left(\frac{1}{L}\sum_{k=0}^{L-1}\log(|\widehat V_L(k)|^2+\varepsilon)\right)}{\frac{1}{L}\sum_{k=0}^{L-1}(|\widehat V_L(k)|^2+\varepsilon)}.$$

## Implementation notes

1. Integration point: modify the positional rotation inside a RoPE-style self-attention layer, before forming query-key dot products. Split each query and key head vector into adjacent 2D pairs. For pair index j, use a base scale beta_j, set alpha_j to an irrational value such as the golden-ratio conjugate times beta_j modulo 1, and optionally use a small learned or fixed x2_j. 2. Pseudocode: compute positions n=0,...,L-1; for every pair j compute phi[n,j] = x1[j] + n*x2[j] + 0.5*n*(n-1)*alpha[j]; c=cos(2*pi*phi), s=sin(2*pi*phi); rotate each pair (u,v) to (c*u-s*v, s*u+c*v) for Q and K; then compute standard attention logits Q_rot @ K_rot.T / sqrt(d). Cache phi or use the recurrence phi[n+1,j]=phi[n,j]+x2[j]+n*alpha[j] to avoid repeated quadratic arithmetic. 3. Math computed exactly: the phase recurrence and relative rotation come directly from the quadratic sequence. Estimate the finite spectral proxy by FFT of each scalar channel or of the induced positional coordinates, reporting spectral flatness SF and maximum normalized periodogram peak; these are diagnostics, not consequences guaranteed at finite L. 4. First experiment: train a 6-layer, 8-head Transformer on synthetic modular addition and copy/reversal tasks with context lengths 128 and 512, comparing standard RoPE, learned absolute embeddings, and quadratic-chirp RoPE at equal parameter count and FLOPs. Evaluate training loss, length extrapolation, attention-distance distributions, and SF on lengths 128, 512, and 2048. Success is lower extrapolation error or fewer long-range aliasing failures without degraded in-range perplexity; reject the method if attention becomes excessively diffuse or optimization becomes unstable. A second cheap test is character-level enwik8 with train length 512 and test length 2048, measuring bits-per-byte and peak-memory overhead.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.