Unverified 2026

Polynomial Spectral Mode Controller

Implementation & benchmark of arXiv:2607.20218 — A Direct Polynomial Approach to Spectral Decomposition

Usefulness5/10

Difficulty6/10

Novelty6/10

Architecture Stability Accuracy Inference speedup Rnn Ssm Training dynamics Inference speedup Algebra Linear algebra Approximation theory Dynamical systems

Source paper: A Direct Polynomial Approach to Spectral Decomposition arXiv:2607.20218 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace an eigendecomposition-based spectral controller in a small recurrent or state-space transition layer with explicit polynomial projectors. Each hidden state is split into invariant modes, and each mode receives a separately constrained recurrent multiplier, enabling direct suppression of unstable modes or selective retention of long-memory modes using only matrix-polynomial evaluations.

Formulas

$$h_i(t)=1-\left(1-\prod_{j\neq i}\left(\frac{t-\lambda_j}{\lambda_i-\lambda_j}\right)^{m_j}\right)^{m_i}$$

$$h_i(t)\equiv\delta_{ij}\ \bmod{(t-\lambda_j)^{m_j}}$$

$$P_i=h_i(A),\qquad N_i=(A-\lambda_iI)P_i,\qquad A=\sum_{i=1}^{r}(\lambda_iP_i+N_i)$$

$$x_{t+1}=\sum_{i=1}^{r}g_iP_i x_t+Bu_t,\qquad g_i=\rho\tanh(\widehat g_i),\quad 0<\rho<1$$

Mathematical statement

For distinct eigenvalues $\lambda_1,\ldots,\lambda_r$ with minimal-polynomial multiplicities $m_1,\ldots,m_r$, define $q_i(t)=\prod_{j\ne i}((t-\lambda_j)/(\lambda_i-\lambda_j))^{m_j}$ and $h_i(t)=1-(1-q_i(t))^{m_i}$. The paper states the Hermite congruences $h_i(t)\equiv \delta_{ij}\pmod{(t-\lambda_j)^{m_j}}$, meaning that $h_i$ equals one to the required derivative order at $\lambda_i$ and zero to the required order at every other eigenvalue. For a matrix $A$ with minimal polynomial $\mu_A(t)=\prod_i(t-\lambda_i)^{m_i}$, the exact spectral projector is $P_i=h_i(A)$, and the nilpotent component is $N_i=(A-\lambda_i I)P_i$. These satisfy $A=\sum_i(\lambda_iP_i+N_i)$. In the first implementation, restrict $A$ to a real symmetric matrix, so $m_i=1$, $N_i=0$, and the formula simplifies to $P_i=\prod_{j\ne i}(A-\lambda_jI)/(\lambda_i-\lambda_j)$. The controller applies mode gains $g_i$ through $A_g=\sum_i g_iP_i$, with $|g_i|<1$ imposed for stable recurrent dynamics.

Implementation notes

Integrate this into a small SSM or RNN transition matrix, with the projectors applied to the hidden state immediately before the recurrent update. Start with hidden dimension 32 or 64 and a real symmetric learnable matrix $A$; use the parameterization $A=(M+M^\top)/2$ and periodically estimate its distinct eigenvalues with a symmetric eigensolver. Cluster eigenvalues separated by less than a tolerance $\varepsilon$, and use one representative $\lambda_i$ per cluster; initially set every cluster multiplicity to one. At each refresh, construct the polynomial coefficients of $h_i(t)=\prod_{j\ne i}(t-\lambda_j)/(\lambda_i-\lambda_j)$, and evaluate $P_i x$ by Horner's rule rather than materializing dense matrix powers. The training update is: inputs $x_t,u_t$; compute $z_i=P_i x_t$ for all modes; compute $g_i=\rho\tanh(\widehat g_i)$; return $x_{t+1}=\sum_i g_i z_i+Bu_t$. For a non-symmetric extension, retain the full Hermite formula, estimate $\lambda_i,m_i$ from a Schur decomposition, and compute $N_i x=(A-\lambda_iI)P_i x$, but do not use this in the first MVP because nearly repeated eigenvalues can make denominators unstable. The mathematical quantities are the projector polynomial and the exact decomposition identity; eigenvalues, cluster assignments, and the refresh interval are empirical design choices. Test first on adding-task or sequential MNIST with a 32-state vanilla Elman RNN and a diagonal/state-space baseline. Compare equal hidden size and equal training FLOPs, measuring gradient explosion, validation accuracy, and long-sequence memory. Success is a materially lower spectral radius and fewer exploding trajectories without sacrificing short-sequence accuracy, or improved accuracy at sequence lengths four times longer than the vanilla RNN. Also benchmark inference against a version that explicitly eigendecomposes whenever mode gains are changed; the polynomial version should avoid per-sequence eigendecompositions.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.