# Truncation-Corrected Local Pseudospectral Regularizer

- ID: 1592
- Canonical URL: https://synthcore.org/idea/1592/truncation-corrected-local-pseudospectral-regularizer
- API JSON: https://synthcore.org/api/idea/1592.json
- API Markdown: https://synthcore.org/api/idea/1592.md
- Verification status: unverified
- Source: [arXiv:2607.29354](https://arxiv.org/abs/2607.29354)
- Category: regularization
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, regularization, training-dynamics
- Math tags: spectral-theory, linear-algebra, functional-analysis
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Replace an expensive global resolvent calculation for a recurrent or state-space transition operator by measurements on overlapping finite patches. Penalize patches whose shifted operator has small minimum gain, while adding the paper's explicit O(1/n) truncation penalty so that increasing the patch size produces a predictable tightening of the stability certificate. This targets non-normal transient amplification that is invisible to ordinary eigenvalue or spectral-radius regularization.

## Mathematical statement

The paper considers an operator A on a discrete group and finite local patches A_{n,k}, where n is the patch size and k indexes the translated patch. For a global vector x and its restriction x_{n,k} to patch k, the central estimate is ||A_{n,k} x_{n,k}|| / ||x_{n,k}|| <= ||A x|| / ||x|| + epsilon_n, with truncation penalty epsilon_n approximately equal to c/n; the proportionality constant depends on the truncation method. For a neural transition matrix W in R^{d x d}, define the shifted operator B(z) = zI - W, where z is a complex candidate perturbation location, and let B_{n,k}(z) = zI_n - W_{n,k} be the corresponding local patch. The local minimum gain is m_{n,k}(z) = sigma_min(B_{n,k}(z)) = min_{v != 0} ||B_{n,k}(z)v||_2 / ||v||_2. If the patch is invertible, its resolvent norm is ||B_{n,k}(z)^{-1}||_2 = 1 / m_{n,k}(z). Use the corrected margin m_{n,k}(z) - epsilon_hat_n, with epsilon_hat_n = c_hat/n, inside a differentiable penalty. The correction prevents a small patch from being treated as globally safe merely because boundary truncation hides an unstable direction.

## Key formulas

- $$\frac{\|A_{n,k}x_{n,k}\|}{\|x_{n,k}\|}\ \leq\ \frac{\|Ax\|}{\|x\|}+\varepsilon_n,\qquad \varepsilon_n\sim\frac{1}{n}.$$
- $$B(z)=zI-W,\qquad B_{n,k}(z)=zI_n-W_{n,k},\qquad m_{n,k}(z)=\sigma_{\min}(B_{n,k}(z))=\min_{v\neq0}\frac{\|B_{n,k}(z)v\|_2}{\|v\|_2}.$$
- $$\mathcal{R}_{\mathrm{local}}(W)=\frac{1}{|\mathcal{Z}|K}\sum_{z\in\mathcal{Z}}\sum_{k=1}^{K}\operatorname{softplus}\!\left(\tau-\left[m_{n,k}(z)-\widehat{\varepsilon}_n\right]\right),\qquad \widehat{\varepsilon}_n=\frac{\widehat{c}}{n}.$$

## Implementation notes

Integrate this during training into the transition operator of an RNN, linear state-space model, or structured sequence layer, rather than into the input and output projections. Assume the layer applies h_{t+1} = phi(W h_t + U x_t), and use the same recurrent matrix W that is repeatedly applied across time. First choose a patch size n, such as 16 or 32, and construct K overlapping coordinate windows over the hidden dimensions. For convolutional, banded, or block-circulant W, use translated group patches; for a dense W, use contiguous principal submatrices or fixed random coordinate patches. At each regularization step, sample a small set Z of complex shifts, for example z = r exp(i theta), with r in {0.9, 1.0, 1.1} and eight equally spaced angles. For every z and patch k, form B_{n,k} = z I_n - W_{n,k}. Compute its smallest singular value using an exact differentiable SVD for the first MVP, or approximate it with two to five inverse-power iterations on B_{n,k}^* B_{n,k}; add lambda times the local regularizer to the task loss. Estimate c_hat empirically by measuring the statistic at patch sizes 8, 16, and 32, fitting an upper envelope of the observed discrepancy to c/n, and multiplying the fitted constant by 1.5 for safety. The paper supplies the 1/n scaling; the implementation estimates the constant and selects the patch family. Test first on permuted sequential MNIST with a one-layer tanh RNN of hidden size 128. Compare an unconstrained baseline and spectral-radius clipping at equal FLOPs. Record long-sequence gradient norms, loss after 50 and 200 updates, maximum hidden-state Jacobian amplification, and validation accuracy. Success means fewer exploding or vanishing-gradient runs and better long-horizon accuracy at comparable parameter count. Also verify that increasing n produces a tighter, more stable resolvent envelope.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.