# Laguerre Memory Convolution

- ID: 2306
- Canonical URL: https://synthcore.org/idea/2306/laguerre-memory-convolution
- API JSON: https://synthcore.org/api/idea/2306.json
- API Markdown: https://synthcore.org/api/idea/2306.md
- Verification status: unverified
- Source: [arXiv:2608.14769](https://arxiv.org/abs/2608.14769)
- Category: architecture
- Solves: memory, speedup, stability
- ML areas: ssm, rnn, inference-speedup, memory
- Math tags: harmonic-analysis, linear-algebra, numerical-analysis, dynamical-systems
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Replace the length-L learned convolution kernel in a causal sequence layer with K Laguerre basis functions, where K is much smaller than L and the basis parameter controls the decay time scale. The layer retains a long receptive field but learns only K coefficients, while FFT or a fixed state-space realization evaluates the resulting convolution efficiently. This is especially appropriate for audio, sensor streams, and long-context regression where the desired impulse response is smooth or exponentially decaying.

## Mathematical statement

The paper starts from the finite causal convolution model $y=\mathcal{T}h$, where $h=(h_0,\ldots,h_{N-M})$ is the impulse-response vector, $y=(y_0,\ldots,y_{N-1})$ is the observed output, and $\mathcal{T}$ is the Toeplitz matrix formed from the input sequence $u=(u_0,\ldots,u_{M-1})$. It embeds this system into the square circulant equation $\widetilde{\mathcal{T}}\widetilde h=y$, where $\widetilde h$ is zero-padded to length $N$; circulant multiplication is diagonalized by the DFT and can therefore be evaluated with FFTs. The transferable coordinate change is a Laguerre-Fourier expansion of the impulse response. For a directly implementable neural adaptation, use the discrete Laguerre transfer basis indexed by $k=0,\ldots,K-1$, with decay parameter $a\in(0,1)$: $L_k(z)=\sqrt{1-a^2}(z^{-1}-a)^k/(1-a z^{-1})^{k+1}$. Let $\ell_k[n]$ be the inverse-DTFT coefficients of $L_k(e^{i\omega})$ and parameterize the causal kernel as $h[n]=\sum_{k=0}^{K-1}c_k\ell_k[n]$, where $c_k$ are learned coefficients. The network output is $y=\mathcal{T}(u)h$. The useful properties are structured long decay, basis compression, and FFT evaluation through $\widehat y_j=\widehat u_j\widehat h_j$.

## Key formulas

- $$H(\omega^k)=\frac{Y(\omega^k)}{U(\omega^k)}$$
- $$\mathcal{T}(u)h=y,\qquad \mathcal{T}(u)\in\mathbb{C}^{N\times(N-M+1)},\quad h=(h_0,\ldots,h_{N-M})^\top,\quad y=(y_0,\ldots,y_{N-1})^\top$$
- $$\widetilde{\mathcal{T}}\widetilde h=y,\qquad \widetilde h=(h_0,\ldots,h_{N-M},0,\ldots,0)^\top\in\mathbb{C}^{N}$$
- $$L_k(z)=\sqrt{1-a^2}\,\frac{(z^{-1}-a)^k}{(1-a z^{-1})^{k+1}},\qquad h[n]=\sum_{k=0}^{K-1}c_k\ell_k[n],\qquad \widehat y_j=\widehat u_j\sum_{k=0}^{K-1}c_k\widehat\ell_k(\omega_j)$$

## Implementation notes

Integrate this at the long convolution operation of a causal SSM/RNN or temporal convolution block, not at attention logits. Inputs are a batch of sequences $u\in\mathbb{R}^{B\times T\times d}$, a truncation length $L\leq T$, a basis size $K\ll L$, and a fixed or learned decay parameter $a\in(0,1)$. Precompute a basis matrix $B\in\mathbb{R}^{L\times K}$ by generating the first $L$ impulse-response samples $\ell_k[n]$ of the rational transfer functions $L_k(z)$; generate each response by filtering a discrete impulse through the stable denominator $(1-a z^{-1})^{k+1}$, and optionally orthonormalize the finite-window columns because truncation can make them non-orthogonal. The trainable parameters are coefficient vectors $c\in\mathbb{R}^{K\times d}$, or $C\in\mathbb{R}^{K\times d_{in}\times d_{out}}$ for channel mixing. At each forward pass form the kernel $h=Bc$ and compute causal convolution by zero-padding $u$ and $h$ to at least $T+L-1$, applying real FFTs, multiplying spectra, inverse FFTing, and cropping the first $T$ positions. Pseudocode is: `B = laguerre_basis(L,K,a); h = B @ c; U = rfft(pad(u)); H = rfft(pad(h)); y = irfft(U * H)[:T]`. Estimate $a$ by a small grid such as $\{0.8,0.9,0.95,0.98,0.995\}$, or learn $a=\sigma(\alpha)$ with clipping below one. Compute $\kappa(B)=\sigma_{max}(B)/\sigma_{min}(B)$ periodically and add a penalty on excessive conditioning if needed. The first cheap experiment should use a synthetic damped-system prediction task or 1D audio forecasting with $T=4096$, comparing a length-1024 direct convolution and a standard SSM baseline against $K\in\{16,32,64\}$. Measure validation error, parameter count, peak memory, and wall-clock throughput. Success means at least 4x fewer kernel parameters with no more than 1% relative error increase, or lower error at equal parameter count; also test sequences twice as long as those used in training and check that the learned impulse response remains bounded and does not develop unstable long-lag oscillations.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.