# Pole-Certified SSM Initialization

- ID: 407
- Canonical URL: https://synthcore.org/idea/407/pole-certified-ssm-initialization
- API JSON: https://synthcore.org/api/idea/407.json
- API Markdown: https://synthcore.org/api/idea/407.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.04568](https://arxiv.org/abs/2607.04568)
- Category: architecture
- Solves: accuracy, stability, sample-efficiency
- ML areas: ssm, initialization, training-dynamics
- Math tags: complex-analysis, linear-algebra, approximation-theory, dynamical-systems
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Extract a small set of stable exponential modes from an observed neural sequence and use them to initialize a diagonal or block-diagonal state-space model. Hankel-pencil eigenvalues propose the modes, while persistence across shifts and contour margins reject modes caused by noise or a short-lived background.

## Mathematical statement

The paper assumes a meromorphic continuation F(z)=h(z)+sum_{j=1}^N r_j/(z-p_j), with h holomorphic beyond the target region, distinct simple poles p_j, and nonzero residues r_j. Expanding each pole term at the origin gives c_n=b_n+sum_{j=1}^N a_j lambda_j^n, where c_n is the observed Taylor or Fourier coefficient, b_n is the rapidly decaying holomorphic-background coefficient, lambda_j=p_j^{-1} is the exponential base, and a_j=-r_j/p_j is its amplitude. For a pure N-mode sequence, the Hankel matrices H_0=[c_{s+i+j}]_{i,j=0}^{N-1} and H_1=[c_{s+i+j+1}]_{i,j=0}^{N-1} satisfy a generalized eigenvalue relation whose eigenvalues are lambda_j. The annihilating polynomial q(lambda)=prod_j(lambda-lambda_j)=lambda^N+q_{N-1}lambda^{N-1}+...+q_0 obeys sum_{k=0}^N q_k c_{n+k}=0. In pole coordinates p=lambda^{-1}, Q(p)=p^N q(p^{-1})=prod_j(1-lambda_j p). The argument principle counts zeros inside a contour Gamma by Z_Gamma=(1/(2 pi i)) integral_Gamma Q'(p)/Q(p) dp. The boundary margin m_Gamma=min_{p in Gamma}|Q(p)| measures robustness: if perturbations from noise and background are small relative to this margin, the local zero count is stable.

## Key formulas

- $$F(z)=h(z)+\sum_{j=1}^{N}\frac{r_j}{z-p_j},\qquad h\in\mathcal{O}(\mathbb{D}_{\rho_h}),\qquad 1<|p_j|<R<\rho_h.$$
- $$c_n=b_n+\sum_{j=1}^{N}a_j\lambda_j^n,\qquad \lambda_j=p_j^{-1},\qquad a_j=-r_j/p_j.$$
- $$H_1v=\lambda H_0v,\qquad H_0=[c_{s+i+j}]_{i,j=0}^{N-1},\qquad H_1=[c_{s+i+j+1}]_{i,j=0}^{N-1},$$
- $$Z_{\Gamma}=\frac{1}{2\pi i}\oint_{\Gamma}\frac{Q'(p)}{Q(p)}\,dp,\qquad Q(p)=p^Nq(p^{-1})=\prod_{j=1}^{N}(1-\lambda_jp),\qquad m_{\Gamma}=\min_{p\in\Gamma}|Q(p)|.$$

## Implementation notes

(1) Integration point: add a small diagonal or 2-by-2 real-block state-space branch to an existing sequence model, such as an S4-style layer, gated residual block, or recurrent residual stream. Collect a scalar or low-dimensional probe sequence y_0,...,y_{T-1} from a frozen teacher model, sensor stream, or intermediate-token feature over time. Initialize the SSM state matrix A from certified bases lambda_j instead of random or logarithmic initialization. (2) Pseudocode: choose order N and shifts S. For every shift s, build H0[i,j]=y[s+i+j] and H1[i,j]=y[s+i+j+1]. Solve H1 v=lambda H0 v using SVD truncation of H0. Retain eigenvalues with modulus below 1+epsilon and acceptable pencil condition number. Cluster eigenvalues across shifts, fit q(lambda)=product(lambda-lambda_j), and map each lambda_j to a real scalar mode or a 2-by-2 real block for complex-conjugate pairs. Set A to these blocks and fit input/output vectors B,C by least squares on y_t approximately C A^t B. (3) Compute the Hankel pencil, generalized eigenvalues, annihilating polynomial, and contour winding count directly. Estimate background and noise from the residual after fitting retained modes. For each candidate annulus contour in p-space, evaluate Q on 128-512 boundary points, compute discrete phase winding, and record m_Gamma; reject candidates whose count changes across shifts or whose margin is below 5-10 times the residual scale. (4) First experiment: use a 1-2 layer GRU or diagonal SSM on synthetic sums of damped sinusoids plus white noise, then test a character-level Penn Treebank task or long-range-copy benchmark. Compare random HiPPO/log-spaced initialization, learned initialization, and pole-certified initialization at equal parameter count. Success means faster early loss descent, fewer exploding or vanishing gradients, and improved extrapolation at sequence lengths 2-4 times the training length; target at least 10% lower validation loss or 2x longer stable rollout at equal FLOPs.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a Hankel-pencil pole estimator with shift persistence, conjugate-pair rejection, contour winding, margin evaluation, and a deterministic extrapolation comparison. The noiseless generalized-eigenvalue identity was verified to about 8.3e-14, and the contour winding count matched the expected zero count with margin 0.0988. However, noisy certification almost always rejected all modes, so no reliable initialization benefit was observed; the proposed effect is not supported by this MVP.

### Mechanism check

- Verdict: Built a Hankel-pencil pole estimator with shift persistence, conjugate-pair rejection, contour winding, margin evaluation, and a deterministic extrapolation comparison. The noiseless generalized-eigenvalue identity was verified to about 8.3e-14, and the contour winding count matched the expected zero count with margin 0.0988. However, noisy certification almost always rejected all modes, so no reliable initialization benefit was observed; the proposed effect is not supported by this MVP.
- Confidence: 9/10
- Limitations: This is a small synthetic extrapolation test, not a trained GRU/SSM or language/long-range benchmark. The estimator uses fixed order and simple proximity-based persistence rather than a fully tuned SVD/conditioning/contour certification procedure; no GPU, learned dynamics, FLOP-matched training, or multi-dimensional probes were tested.

## Artifacts

- [experiment.py](https://synthcore.org/code/1290/experiment.py)
- [report.md](https://synthcore.org/code/1290/report.md)
- [results.json](https://synthcore.org/code/1290/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1290)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.