# Spectral Phase-Lifted Features

- ID: 2127
- Canonical URL: https://synthcore.org/idea/2127/spectral-phase-lifted-features
- API JSON: https://synthcore.org/api/idea/2127.json
- API Markdown: https://synthcore.org/api/idea/2127.md
- Verification status: unverified
- Source: [arXiv:2608.11437](https://arxiv.org/abs/2608.11437)
- Category: architecture
- Solves: accuracy, stability
- ML areas: ssm, rnn, mlp, embedding
- Math tags: functional-analysis, spectral-theory, linear-algebra, approximation-theory
- Ratings: usefulness 5/10; difficulty 7/10; novelty 8/10

## Idea description

Augment a hidden representation with positively homogeneous interaction features built from approximate eigenmodes of a linear layer. Fractional products of mode magnitudes and phases provide nonlinear channels whose transformation laws are inherited from the spectrum of the underlying operator, potentially representing oscillatory or multiplicative dynamics more compactly than a generic MLP.

## Mathematical statement

The paper embeds a Banach space E into a free complex Banach lattice through the map \(\delta_E:E\to \FBL_{\mathbb C}[E]\subseteq C_{ph}(E',\mathbb C)\), with \(\delta_E(z)(z')=\langle z',z\rangle\). Here \(E'\) is the dual space, \(C_{ph}\) is the space of continuous positively homogeneous functions, and \(\langle z',z\rangle\) is the dual pairing. The explicit functional-calculus example is \(f_\gamma(z)=|z_1|^\gamma|z_2|^{1-\gamma}\), for \(\gamma\in[0,1]\); it is positively homogeneous of degree one and is nonzero exactly when both coordinate or eigenmode inputs are nonzero. Theorem 7.11 states that if \(T:E\to E\) has an eigenvalue of modulus \(r>0\), and \(U=\{\prod_{k=1}^n \lambda_k/|\lambda_k|: n\in\mathbb N_0,\ \lambda_k\in(\sigma_{\mathrm{pnt}}(T)\setminus\{0\})\cap r\mathbb T\}\), then \(ru\in\sigma_{\mathrm{pnt}}(\overline T)\) for every \(u\in U\). Here \(\sigma_{\mathrm{pnt}}(T)\) is the point spectrum, \(\mathbb T\) is the unit circle, and \(\overline T\) is the lattice-homomorphic extension. The neural adaptation uses \(\phi_{ij,\gamma}(h)=|a_i(h)|^\gamma|a_j(h)|^{1-\gamma}\exp(i[\gamma\arg a_i(h)+(1-\gamma)\arg a_j(h)])\), where \(a_i(h)\) and \(a_j(h)\) are approximate complex eigenmode coordinates. If \(a_k(Th)=\lambda_k a_k(h)\), then the lifted feature scales approximately by \(|\lambda_i|^\gamma|\lambda_j|^{1-\gamma}e^{i(\gamma\arg\lambda_i+(1-\gamma)\arg\lambda_j)}\).

## Key formulas

- $$\delta_E:E\hookrightarrow\FBL_{\mathbb C}[E]\subseteq C_{ph}(E',\mathbb C),\qquad \delta_E(z)(z')=\langle z',z\rangle.$$
- $$f_\gamma(z)=|z_1|^\gamma|z_2|^{1-\gamma},\qquad \gamma\in[0,1].$$
- $$U=\left\{\prod_{k=1}^{n}\frac{\lambda_k}{|\lambda_k|}\ \middle|\ n\in\mathbb N_0,\ \lambda_1,\dots,\lambda_n\in(\sigma_{\operatorname{pnt}}(T)\setminus\{0\})\cap r\mathbb T\right\},\qquad ru\in\sigma_{\operatorname{pnt}}(\overline T).$$
- $$\phi_{ij,\gamma}(h)=\bigl(|a_i(h)|+\varepsilon\bigr)^\gamma\bigl(|a_j(h)|+\varepsilon\bigr)^{1-\gamma}\exp\!\left(i\left[\gamma\arg(a_i(h))+(1-\gamma)\arg(a_j(h))\right]\right).$$

## Implementation notes

(1) Integrate the lift after a complex-valued linear state-transition layer or, in a real model, after a real matrix layer represented by paired real and imaginary channels. Let the layer be \(h_{t+1}=Wh_t+b_t\), with hidden width \(d\). Initially use one small recurrent or SSM-style block rather than every transformer layer, so the dynamical effect is measurable. (2) Estimate the top \(K\) eigenpairs of \(W\) using Arnoldi iteration, or use a learned analysis map \(A\) and periodically fit \(WA\approx A\Lambda\). For each hidden state compute mode coordinates \(a=A^\dagger h\). Select pairs \((i,j)\) among modes with similar radii \(|\lambda_i|\) and \(|\lambda_j|\), and concatenate real and imaginary parts of \(\phi_{ij,\gamma}(h)\) for \(\gamma\in\{0.25,0.5,0.75\}\). (3) Use this pseudocode: `h = W(h_prev)+b; a = pinv(A)@h; new=[]; for (i,j) in pairs: mag=(abs(a[i])+eps)**gamma*(abs(a[j])+eps)**(1-gamma); phase=gamma*angle(a[i])+(1-gamma)*angle(a[j]); new += [mag*cos(phase), mag*sin(phase)]; h_out = h + P(concat(h,new))`, where \(P\) is a learned projection back to width \(d\). Stop gradients through eigenvectors for the first experiment and recompute them every 100-500 optimizer steps. (4) The paper supplies the positive-homogeneity and multiplicative-phase construction; eigendecomposition, pair selection, normalization, and projection are empirical choices. Add scale normalization \(\tilde\phi=\phi/(\operatorname{RMS}(\phi)+\varepsilon)\) and optionally gate the lift with a learned sigmoid. First test on copying, selective-memory, and sinusoidal long-range prediction tasks using a 2-layer SSM or GRU with 128 hidden units, comparing the lifted model against the same-width baseline and a width-matched MLP feature expansion. Measure validation loss versus sequence length, gradient-norm variance, and performance at equal parameter count. The hypothesis is lower error on oscillatory or multiplicative-memory tasks and slower degradation as sequence length grows; a negative result is no improvement or unstable phase behavior after eigenmode tracking.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.