Pole-Certified SSM Initialization
Implementation & benchmark of arXiv:2607.04568 — Determinant Characteristics and Argument-Principle Certification for Visible Poles in Meromorphic Continuation
Source paper: Determinant Characteristics and Argument-Principle Certification for Visible Poles in Meromorphic Continuation arXiv:2607.04568 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Extract a small set of stable exponential modes from an observed neural sequence and use them to initialize a diagonal or block-diagonal state-space model. Hankel-pencil eigenvalues propose the modes, while persistence across shifts and contour margins reject modes caused by noise or a short-lived background.
Formulas
Mathematical statement
The paper assumes a meromorphic continuation F(z)=h(z)+sum_{j=1}^N r_j/(z-p_j), with h holomorphic beyond the target region, distinct simple poles p_j, and nonzero residues r_j. Expanding each pole term at the origin gives c_n=b_n+sum_{j=1}^N a_j lambda_j^n, where c_n is the observed Taylor or Fourier coefficient, b_n is the rapidly decaying holomorphic-background coefficient, lambda_j=p_j^{-1} is the exponential base, and a_j=-r_j/p_j is its amplitude. For a pure N-mode sequence, the Hankel matrices H_0=[c_{s+i+j}]_{i,j=0}^{N-1} and H_1=[c_{s+i+j+1}]_{i,j=0}^{N-1} satisfy a generalized eigenvalue relation whose eigenvalues are lambda_j. The annihilating polynomial q(lambda)=prod_j(lambda-lambda_j)=lambda^N+q_{N-1}lambda^{N-1}+...+q_0 obeys sum_{k=0}^N q_k c_{n+k}=0. In pole coordinates p=lambda^{-1}, Q(p)=p^N q(p^{-1})=prod_j(1-lambda_j p). The argument principle counts zeros inside a contour Gamma by Z_Gamma=(1/(2 pi i)) integral_Gamma Q'(p)/Q(p) dp. The boundary margin m_Gamma=min_{p in Gamma}|Q(p)| measures robustness: if perturbations from noise and background are small relative to this margin, the local zero count is stable.
Implementation notes
(1) Integration point: add a small diagonal or 2-by-2 real-block state-space branch to an existing sequence model, such as an S4-style layer, gated residual block, or recurrent residual stream. Collect a scalar or low-dimensional probe sequence y_0,...,y_{T-1} from a frozen teacher model, sensor stream, or intermediate-token feature over time. Initialize the SSM state matrix A from certified bases lambda_j instead of random or logarithmic initialization. (2) Pseudocode: choose order N and shifts S. For every shift s, build H0[i,j]=y[s+i+j] and H1[i,j]=y[s+i+j+1]. Solve H1 v=lambda H0 v using SVD truncation of H0. Retain eigenvalues with modulus below 1+epsilon and acceptable pencil condition number. Cluster eigenvalues across shifts, fit q(lambda)=product(lambda-lambda_j), and map each lambda_j to a real scalar mode or a 2-by-2 real block for complex-conjugate pairs. Set A to these blocks and fit input/output vectors B,C by least squares on y_t approximately C A^t B. (3) Compute the Hankel pencil, generalized eigenvalues, annihilating polynomial, and contour winding count directly. Estimate background and noise from the residual after fitting retained modes. For each candidate annulus contour in p-space, evaluate Q on 128-512 boundary points, compute discrete phase winding, and record m_Gamma; reject candidates whose count changes across shifts or whose margin is below 5-10 times the residual scale. (4) First experiment: use a 1-2 layer GRU or diagonal SSM on synthetic sums of damped sinusoids plus white noise, then test a character-level Penn Treebank task or long-range-copy benchmark. Compare random HiPPO/log-spaced initialization, learned initialization, and pole-certified initialization at equal parameter count. Success means faster early loss descent, fewer exploding or vanishing gradients, and improved extrapolation at sequence lengths 2-4 times the training length; target at least 10% lower validation loss or 2x longer stable rollout at equal FLOPs.
Verification
Stage 1 · Toy mechanism gate: Failed ✗
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built a Hankel-pencil pole estimator with shift persistence, conjugate-pair rejection, contour winding, margin evaluation, and a deterministic extrapolation comparison. The noiseless generalized-eigenvalue identity was verified to about 8.3e-14, and the contour winding count matched the expected zero count with margin 0.0988. However, noisy certification almost always rejected all modes, so no reliable initialization benefit was observed; the proposed effect is not supported by this MVP.
- Agent confidence
- 9/10
- Baseline
- Mean extrapolation MSE: 1.07e-4, 1.15e-3, and 9.30e-3 at noise 0.01, 0.035, and 0.1 respectively.
- Idea
- Mean certified pairs: 0.2, 0.0, and 0.0 at noise 0.01, 0.035, and 0.1 respectively; MSE was infinite when no modes were certified, and 1.058e-4 in the sole successful low-noise trial.
Limitations: This is a small synthetic extrapolation test, not a trained GRU/SSM or language/long-range benchmark. The estimator uses fixed order and simple proximity-based persistence rather than a fully tuned SVD/conditioning/contour certification procedure; no GPU, learned dynamics, FLOP-matched training, or multi-dimensional probes were tested.
How to run: python3 experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- experiment.py 3.0 KB View Raw
- report.md 1.4 KB View
- results.json 3.2 KB View Raw