Odd-Harmonic Spherical Positional Encoding

(1) Integrate this at the positional-embedding stage of a small decoder-only transformer. For each token position n, map it to a phase t_n = 2*pi*n/P when the task has known period P. For ordinary sequence positions, use t_n = 2*pi*n/P_max with a large P_max, or concatenate this encoding with the existing positional encoding. Construct k+1 harmonic pairs at odd frequencies 1, 3, ..., 2k+1, then project the resulting vector from dimension 2(k+1) to the model dimension d_model using a learned matrix W_p and add it to token embeddings before the first attention block. Start with fixed a_j = 1. An optional learned version uses unconstrained parameters b_j, sets a_j = softplus(b_j), and recomputes Z = sum_j a_j^2 on every forward pass. (2) Pseudocode: for each position n, set t = 2*pi*n/P; initialize z as an empty vector; for j from 0 through k, set omega = 2*j+1 and append a[j]*cos(omega*t) and a[j]*sin(omega*t); divide z by sqrt(sum_j a[j]^2); set h[n] = token_embedding[n] + W_p times z. (3) Compute the embedding and normalization exactly from the displayed formulas. Log the empirical positional kernel and verify that its value before projection equals K_a(t_n,t_m). Estimate no theorem constants; all quantities are directly computable. (4) First experiment: train a 2-layer, 4-head transformer on modular addition, copy-with-period-P, and synthetic periodic sequence-prediction tasks. Compare learned absolute positions, standard sinusoidal positions with frequencies 1, 2, 4, ..., and this odd-harmonic spherical encoding for k in {1, 3, 7, 15}. Keep projected positional dimension and parameter count comparable. Evaluate validation accuracy at training lengths and substantially longer extrapolation lengths, attention entropy versus phase difference, positional-vector norm variation, gradient-norm variance, and wall-clock overhead. The falsifiable hypothesis is improved periodic extrapolation and more stable positional norms, especially when the test period is represented by the encoding.