Unverified 2026

Zeta spectral monitor for recurrent training

Implementation & benchmark of arXiv:2609.01399 — The Quadratic Easy Coefficients Conjecture via Finite-Type Shifts and Zeta Functions

Usefulness6/10

Difficulty5/10

Novelty6/10

Regularization Stability Generalization Accuracy Rnn Ssm Optimizer Regularization Dynamical systems Spectral theory Linear algebra Combinatorics Optimization

Source paper: The Quadratic Easy Coefficients Conjecture via Finite-Type Shifts and Zeta Functions arXiv:2609.01399 ⓘ · analyzed Sep 2, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the determinant and trace-power identities of the rules matrix as a spectral diagnostic for recurrent or state-space training. Penalize unstable or excessively resonant modes through a truncated log-zeta objective, while retaining selected eigenvalues near the unit circle when long memory is desired. This gives a falsifiable transition criterion based on closed-walk growth rather than only gradient norms.

Formulas

$$\zeta_R(z)=\frac{1}{\det(I-zR)}=\prod_j(1-z\lambda_j)^{-1},\qquad \log\zeta_R(z)=\sum_{n=1}^{\infty}\frac{z^n\operatorname{tr}(R^n)}{n}.$$

$$L_{\mathrm{zeta}}(\theta)=-\sum_{n=1}^{K}\frac{\alpha^n}{n}\operatorname{tr}(R(\theta)^n),\qquad 0<\alpha<\frac{1}{\rho(R(\theta))}.$$

$$\widehat{\operatorname{tr}(R^n)}=\frac{1}{J}\sum_{j=1}^{J}v_j^{\top}R^nv_j,\qquad \mathbb{E}[v_jv_j^{\top}]=I.$$

$$\rho(R)<1\ \Rightarrow\ \|R^n\|\leq C(\rho(R)+\varepsilon)^n\ \text{for sufficiently large }n.$$

Mathematical statement

Let R(theta) be a recurrent transition matrix, reduced Jacobian, or learned finite-state transfer matrix parameterized by neural parameters theta. The paper identifies the symbolic-dynamics zeta function with zeta_R(z)=det(I-zR)^(-1). Its power-series coefficients are tr(R^n), aggregating all length-n closed walks, and its singularities occur at z=1/lambda_j for eigenvalues lambda_j of R. For a contraction radius alpha satisfying 0<alpha<1/rho(R), use the truncated objective L_zeta=-sum_{n=1}^K alpha^n tr(R^n)/n, or monitor G(alpha)=sum_{n=1}^K alpha^n|tr(R^n)|/n. Hutchinson random vectors estimate traces without explicitly forming dense powers. The relevant stability boundary is rho(R)=1, and the truncated zeta should increase sharply as alpha rho(R) approaches one from below.

Implementation notes

Integration point: apply this monitor to the recurrent Jacobian or transition matrix at every optimizer step. For an explicit linear state-space layer, use its state matrix A directly. For an RNN, estimate a local Jacobian R=partial h_{k+1}/partial h_k on a minibatch and several sequence positions; this is not the full parameter Hessian. Choose K=4 to 8 and J=2 to 8 Rademacher vectors. For each vector, set u_{j,0}=v_j and recursively compute u_{j,n}=Ru_{j,n-1}; estimate each trace by J^(-1) sum_j v_j^T u_{j,n}. Add lambda L_zeta to the task loss, and separately log power growth and a few Lanczos eigenvalue estimates. If the estimated spectral radius exceeds a target r_*<1, reduce recurrent gain or apply spectral normalization. If memory is required, constrain only unstable modes while allowing selected modes near the unit circle. The determinant/zeta identity and trace expansion are imported from the paper; Hutchinson estimation, Jacobian selection, and the penalty are adaptations. First experiment: compare GRU, orthogonal RNN, and zeta-regularized RNN on Copy Memory and sequential-MNIST with equal parameter counts. Sweep recurrent gain across its instability boundary. The predicted signature is a sharp increase in G(alpha) as alpha rho(R) approaches one from below, and exploding long-horizon gradients when measured rho(R)>1. The estimated boundary should agree with power iteration within 10-20 percent.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.