# Di-Cayley two-stream spectral layer

- ID: 231
- Canonical URL: https://synthcore.org/idea/231/di-cayley-two-stream-spectral-layer
- API JSON: https://synthcore.org/api/idea/231.json
- API Markdown: https://synthcore.org/api/idea/231.md
- Verification status: unverified
- Source: [arXiv:2608.27231](https://arxiv.org/abs/2608.27231)
- Category: architecture
- Solves: speedup, scalability, accuracy
- ML areas: graph-nn, transformer, attention, embedding
- Math tags: representation-theory, harmonic-analysis, linear-algebra, group-theory
- Ratings: usefulness 6/10; difficulty 5/10; novelty 5/10

## Idea description

Represent tokens or nodes as two feature streams indexed by a finite group G, and replace dense pairwise mixing by a learned di-Cayley operator with within-stream and cross-stream connections. In the Fourier or irreducible-representation basis, each group frequency is processed independently by a small block matrix, giving exact translation or group equivariance and O(|G| log |G|) mixing for cyclic groups.

## Mathematical statement

A Cayley graph on a group G with connection set S has an arc from u to v when vu^{-1} is in S; its adjacency operator is group convolution by the indicator of S. The paper computes spectra using irreducible representations ρ of G. For a two-part graph with within-part connection set S and cross-part connection set T, the local spectral block is M_ρ = [[ρ(S), ρ(T)], [ρ(T)*, ρ(S)]], where ρ(S) = sum over s in S of ρ(s), ρ(T) = sum over t in T of ρ(t), and * denotes conjugate transpose. Each eigenvalue of M_ρ is repeated according to the representation dimension d_ρ. For an abelian group such as G = Z_n, every irreducible representation is the scalar character χ_k(g) = exp(2π i k g/n), so M_k is a 2 by 2 matrix. The neural adaptation replaces fixed connection sums by learned channel matrices L_k, R_k, and T_k, using the two-stream block update [H_L'_k; H_R'_k] = σ([[L_k,T_k],[T_k*,R_k]][H_L_k;H_R_k]). Because group convolution diagonalizes into these representation blocks, the resulting linear layer commutes with simultaneous group shifts.

## Key formulas

- $$u\sim v\quad\Longleftrightarrow\quad vu^{-1}\in S$$
- $$M_{\rho}=\begin{pmatrix}\rho(S)&\rho(T)\\\rho(T)^{*}&\rho(S)\end{pmatrix},\qquad \rho(S)=\sum_{s\in S}\rho(s),\quad \rho(T)=\sum_{t\in T}\rho(t)$$
- $$\chi_k(g)=\omega^{k\cdot g}=\exp\!\left(\frac{2\pi i}{n}kg\right),\qquad \omega=\exp\!\left(\frac{2\pi i}{n}\right),\quad G=\mathbb Z_n$$
- $$\begin{bmatrix}H^{L\prime}_k\\H^{R\prime}_k\end{bmatrix}=\sigma\!\left(\begin{bmatrix}L_k&T_k\\T_k^{*}&R_k\end{bmatrix}\begin{bmatrix}H^L_k\\H^R_k\end{bmatrix}\right),\qquad H^L_k=\operatorname{FFT}(H^L)_k,\quad H^R_k=\operatorname{FFT}(H^R)_k$$

## Implementation notes

(1) Integration point: insert this layer where a model performs spatial token mixing, graph message passing, or attention over positions arranged on a cycle, torus, periodic grid, or another known finite-group orbit. Maintain two tensors H_L and H_R with shape [batch, |G|, channels], representing two node types, parity classes, orientations, or alternating token streams. The MVP uses G = Z_n and replaces an n by n attention or graph-mixing matrix.

(2) Pseudocode:
```text
input HL, HR: [batch, n, d]
ZL = FFT(HL, axis=position)
ZR = FFT(HR, axis=position)
for k in 0,...,n-1:
    qL = L[k] @ ZL[:, k, :] + T[k] @ ZR[:, k, :]
    qR = conj(T[k]).T @ ZL[:, k, :] + R[k] @ ZR[:, k, :]
    ZL[:, k, :], ZR[:, k, :] = activation(qL), activation(qR)
HL2 = real(IFFT(ZL, axis=position))
HR2 = real(IFFT(ZR, axis=position))
return (HL, HR) + alpha * (HL2, HR2)
```
Initialize L[k] and R[k] near identity and T[k] near zero. To guarantee real outputs, tie conjugate frequencies: L[n-k] = conjugate(L[k]), R[n-k] = conjugate(R[k]), and T[n-k] = conjugate(T[k]).

(3) Computed directly from the paper's mathematics: the two-stream block form, the Fourier character basis, and independent processing of representation modes. Learned empirically: L[k], R[k], T[k], the retained frequency set, and residual scale alpha. A fixed-connection baseline can use L[k] = sum over s in S_l of χ_k(s), R[k] = sum over s in S_r of χ_k(s), and T[k] = sum over t in S_m of χ_k(t), before allowing these coefficients to become channel matrices. For nonabelian groups, replace FFT by a group Fourier transform and use one block per irreducible representation.

(4) First cheap experiment: train a small sequence model on length-128 cyclic classification tasks, such as deciding whether two marked symbols have a specified modular separation. Compare a standard Transformer encoder, a dense graph-convolution baseline, and this spectral layer at matched parameter count. Measure validation accuracy, loss versus wall-clock time, peak memory, and equivariance error E = ||f(shift_a H) - shift_a f(H)||_2 / ||f(H)||_2. Then test a periodic-grid image or molecular-ring benchmark. The expected signal is equivariance error near numerical precision, O(n log n) position mixing instead of O(n^2), lower memory, and equal or better accuracy on tasks respecting cyclic structure. On unstructured data, the method should not beat attention and may underfit.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.