Unverified 2026

Ramanujan Signed Ring Mixer

Implementation & benchmark of arXiv:2607.18334 — Signed circulants at the Ramanujan bound

Usefulness6/10
Difficulty3/10
Novelty6/10

Source paper: Signed circulants at the Ramanujan bound arXiv:2607.18334 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an unsigned two-hop cyclic mixer by the paper's alternating signed circulant. The sign pattern preserves one-step and two-step interactions while reducing the exact spectral radius from 4 to 2√2, allowing a larger raw mixing coefficient under the same operator-norm stability constraint.

Formulas

$$(A_\sigma)_{i,i+1}=(A_\sigma)_{i+1,i}=1,\qquad (A_\sigma)_{i,i+2}=(A_\sigma)_{i+2,i}=(-1)^i,\qquad i\in\mathbb Z_n.$$
$$\\operatorname{spec}(A_\\sigma)=\\left\\{\\pm 2\\sqrt{\\cos^2\\theta_k+\\cos^2 2\\theta_k}:\\ \theta_k=\\frac{2\\pi k}{n}\\right\\},\\qquad \\rho(A_\\sigma)=2\\sqrt2.$$
$$\lambda_k^+=2\cos\theta_k+2\cos 2\theta_k,\qquad \rho(A_+)=4.$$
$$H^{(\ell+1)}=\phi\!\left(c_\ell A_\sigma H^{(\ell)}W_\ell+b_\ell\right),\qquad c_\ell=\frac{\gamma}{2\sqrt2\,\|W_\ell\|_2},\quad 0<\gamma\leq 1.$$

Mathematical statement

For even n≥10, let vertices be i∈Z_n. Define the symmetric signed adjacency matrix A_sigma by (A_sigma)_{i,i+1}=(A_sigma)_{i+1,i}=1 and (A_sigma)_{i,i+2}=(A_sigma)_{i+2,i}=(-1)^i; this is well-defined because i and i+2 have the same parity. With Fourier angles theta_k=2πk/n, the paper states that the spectrum is {±2√(cos²(theta_k)+cos²(2 theta_k))}, and therefore rho(A_sigma)=2√2. The unsigned circulant with +1 on both step-1 and step-2 edges has eigenvalues lambda_k^+=2 cos(theta_k)+2 cos(2 theta_k) and spectral radius 4. Because A_sigma is real symmetric, its spectral norm equals its spectral radius. For a 1-Lipschitz activation phi, the map H↦phi(c A_sigma H W) has Lipschitz upper bound c rho(A_sigma) ||W||_2. Choosing c=gamma/(2√2 ||W||_2), with 0<gamma≤1, makes the linear part gamma-Lipschitz; the unsigned counterpart requires c=gamma/(4 ||W||_2).

Implementation notes

Integrate this as a fixed sequence mixer or graph-convolution layer for tensors H with shape [batch, n, d], where n is even and d is the feature width. Construct A_sigma H without storing a dense matrix: for each position i, compute M_i=H_{i-1}+H_{i+1}+(-1)^i(H_{i-2}+H_{i+2}), using circular indexing. In tensor code, create a parity vector p_i=(-1)^i and calculate M=roll(H,+1)+roll(H,-1)+p*(roll(H,+2)+roll(H,-2)); then calculate Y=activation(c*M@W+b). At every training step, estimate the spectral norm of W with one to three power-iteration steps, or use a cached estimate updated once per optimizer step. Set c=gamma/(2√2*||W||_2), initially gamma=0.9, so the fixed graph factor is the exact paper value rho(A_sigma)=2√2 and only the learned channel norm is estimated. Compare against the unsigned operator M_plus=roll(H,+1)+roll(H,-1)+roll(H,+2)+roll(H,-2), normalized with c=gamma/(4*||W||_2), and against a standard residual 1D convolution with the same parameter count. First test 12- and 24-layer models on ring-node label propagation, cyclic denoising, and a small sequence-copy task. Record loss versus optimizer steps and FLOPs, gradient norms by depth, activation variance, and validation accuracy. The hypothesis is that the signed mixer permits larger unnormalized mixing or deeper stable stacks; success is faster loss descent at equal FLOPs, fewer exploding or vanishing gradients, or higher accuracy at equal depth.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.