# Fourier Turing Recurrent Layer

- ID: 1438
- Canonical URL: https://synthcore.org/idea/1438/fourier-turing-recurrent-layer
- API JSON: https://synthcore.org/api/idea/1438.json
- API Markdown: https://synthcore.org/api/idea/1438.md
- Verification status: unverified
- Source: [arXiv:2607.26331](https://arxiv.org/abs/2607.26331)
- Category: architecture
- Solves: accuracy, stability, sample-efficiency
- ML areas: cnn, ssm, training-dynamics, initialization
- Math tags: dynamical-systems, spectral-theory, linear-algebra, stochastic-processes
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Replace one spatial convolution block by a recurrent Fourier-domain layer that couples every mode k to its opposite mode -k and gives the strongest amplification to a nonzero selected wave number k*. The layer crosses a controlled Turing-like instability at k* and uses cubic saturation to produce bounded structured features instead of unbounded activation growth.

## Mathematical statement

The paper defines, for a homogeneous Gaussian momentum pair (k,-k), an effective damping \(\kappa_k\), squeezing or pair-coupling amplitude \(g_k\), and rotation frequency \(\Omega_\lambda\). It introduces \(R_k=\sqrt{\kappa_k^2+4\Omega_\lambda^2}\) and \(\eta_k=2|g_k|/R_k\); the pair is stable exactly when \(\eta_k<1\), with instability or threshold at \(\eta_k=1\). We transfer this mode-wise threshold to a real-valued feature map on a periodic \(H\times W\) grid: \(z_k\) is the complex Fourier coefficient of one feature channel, \(\overline{z_{-k}}\) is its conjugate opposite-mode partner, \(\kappa_k>0\) is damping, \(\Omega_k\) is phase rotation, and \(g_k\) is a learnable nonnegative coupling. The adapted dynamics use the same linear pair coupling and add \(-\beta_k|z_k|^2z_k\), with \(\beta_k>0\), as a saturating nonlinear term. Initialize \(\eta_k<1\) except for a narrow annulus around a learnable nonzero \(k_*\), where \(\eta_k\) is slightly above one; the cubic term then bounds the resulting pattern amplitude. The paper's covariance theorem, \(\sup_{0\le t\le T}\|V_N(t)-V(t)\|=O(N^{-1/2})\), motivates monitoring feature covariance with increasing channel count, but this guarantee is not assumed to hold for neural features.

## Key formulas

- $$R_k:=\sqrt{\kappa_k^2+4\Omega_\lambda^2},\qquad \eta_k:=\frac{2|g_k|}{R_k}.$$
- $$\eta_k<1\ \Longleftrightarrow\ \text{the opposite-momentum pair }(k,-k)\text{ is stable},\qquad \widetilde{\nu}_{-}(k,-k)=\frac{1}{2(1+\eta_k)},\qquad E_{\rm LN}(k,-k)=\log_2(1+\eta_k).$$
- $$\frac{d z_k}{dt}=-(\kappa_k+i\Omega_k)z_k+g_k\,\overline{z_{-k}}-\beta_k|z_k|^2z_k+u_k(t),\qquad \frac{d z_{-k}}{dt}=-(\kappa_k+i\Omega_k)z_{-k}+g_k\,\overline{z_k}-\beta_k|z_{-k}|^2z_{-k}+u_{-k}(t).$$
- $$\mathcal L_{\rm Turing}=\lambda_\eta\sum_k\bigl[\max(0,\eta_k-1-\delta)\bigr]^2+\lambda_{\rm off}\sum_{k\notin\mathcal A(k_*)}\eta_k^2,\qquad \eta_k=\frac{2g_k}{\sqrt{\kappa_k^2+4\Omega_k^2}}.$$

## Implementation notes

Integrate this as a replacement for a depthwise spatial convolution in a small CNN or U-Net encoder. The exact tensors are the feature map \(x\in\mathbb R^{B\times C\times H\times W}\), its 2-D FFT \(z=\mathrm{FFT2}(x)\), and paired frequency bins \(k\) and \(-k\). For each recurrent substep, first compute \(z=\mathrm{FFT2}(x)\). For every retained frequency pair, compute positive parameters \(\kappa_k=\mathrm{softplus}(a_k)\), \(g_k=\mathrm{softplus}(b_k)\), \(\Omega_k=c_k\), and \(\beta_k=\mathrm{softplus}(d_k)\). Calculate \(\eta_k=2g_k/\sqrt{\kappa_k^2+4\Omega_k^2}\), then update \(z_k\leftarrow z_k+\Delta t[-(\kappa_k+i\Omega_k)z_k+g_k\overline{z_{-k}}-\beta_k|z_k|^2z_k+u_k]\), applying the analogous update to \(-k\). Perform an inverse FFT and add a pointwise residual connection. Parameterize \(g_k\) as a Gaussian bump centered at a learnable radius \(r_*>0\), making the preferred wavelength nonzero: \(g_k=g_0\exp[-(|k|-r_*)^2/(2\sigma^2)]\). The exact mathematical quantities computed are \(R_k\) and \(\eta_k\); \(r_*\), \(\sigma\), and the gain scale are learned. Add the displayed stability loss, keeping nonselected modes below threshold while allowing the selected annulus to satisfy \(\eta_k\approx1+\delta\). Start with CIFAR-10 at 32x32 resolution, compare against a same-parameter depthwise convolution and a Fourier Neural Operator block, and use 4-8 Euler substeps. Measure validation accuracy, robustness to translations and noise, activation spectral concentration around \(r_*\), and exploding or vanishing activation frequency. A successful result is higher accuracy or corruption robustness at equal FLOPs, stable recurrent rollout over 8 substeps, and a clear nonzero Fourier peak that moves when the learned \(r_*\) is varied.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.