Caratheodory-kernel passivity regularizer
Implementation & benchmark of arXiv:2607.05334 — Operator model and a trace formula for pairs of unitary operators
Source paper: Operator model and a trace formula for pairs of unitary operators arXiv:2607.05334 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Regularize a learned state-space transfer function so its matrix response has positive real part on sampled points in the unit disk and its associated reproducing-kernel Gram matrix is positive semidefinite. This provides a frequency-domain stability signal that complements rollout-based penalties and spectral-radius clipping.
Formulas
Mathematical statement
The paper assumes that Phi(lambda) is an n-by-n analytic matrix function on the unit disk D with positive real part, meaning Herm(Phi(lambda)) = (Phi(lambda)+Phi(lambda)^*)/2 is positive semidefinite. Its Herglotz representation is Phi(lambda) = iA + integral over the unit circle of (e^(it)+lambda)/(e^(it)-lambda) dmu(e^(it)), where A satisfies A+A^*=0 and mu is a positive matrix-valued measure. The associated reproducing kernel is L_Phi(lambda,w) = [Phi(lambda)+Phi(w)^*]/[2(1-lambda conjugate(w))], and finite Gram matrices formed from this kernel are positive semidefinite. For a neural state-space model with transfer response R_theta(lambda) = C(I-lambda A_theta)^(-1)B, define the adapted response Phi_theta(lambda) = I + R_theta(lambda). Penalize violations of Herm(Phi_theta(lambda)) being positive semidefinite and violations of the kernel Gram matrix K being positive semidefinite. The eigenvalue margins are directly computable from finite matrices.
Implementation notes
Apply the regularizer to a linear SSM or to a frozen-point linearization of an RNN. The exact integration point is the training loss, after constructing the transition matrix A_theta and input/output matrices B and C; the forward recurrence remains unchanged. Choose M complex sample points lambda_j = rho_j exp(i omega_j), with rho_j between 0.8 and 0.99 and omega_j uniformly covering [0, 2 pi). For each point, solve the linear system (I - lambda_j A_theta) X_j = B rather than explicitly forming an inverse, then compute Phi_j = I + C X_j and H_j = (Phi_j + Phi_j^*)/2. Penalize the negative eigenvalue margin using relu(epsilon - lambda_min(H_j)) squared. For a second test, select m at most 16 sample points, form K[i,j] = (Phi_i + Phi_j^*)/(2(1 - lambda_i conjugate(lambda_j))), symmetrize K as (K+K^*)/2, and add relu(epsilon - lambda_min(K)) squared. Pseudocode: sample lambdas; solve each resolvent system; calculate Phi and H; calculate minimum Hermitian eigenvalues; build K; add alpha times the total penalty to the task loss. The Herglotz representation and kernel formula come from the paper; the frequency grid, epsilon, penalty coefficient alpha, and identity offset are empirical design choices. Begin with a 64-state SSM on copy, adding, and ListOps tasks. Compare no regularizer, spectral-radius clipping, and Caratheodory-kernel regularization with matched optimization settings. Track the minimum sampled eigenvalue, rollout error at horizons 2x and 4x training length, gradient norms, validation loss, and wall-clock overhead. Success means improved long-horizon extrapolation and a nonnegative or near-nonnegative response margin without degrading short-context accuracy by more than a small tolerance.
Verification
Stage 1 · Toy mechanism gate: Failed ✗
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built a differentiable Caratheodory-kernel regularizer using resolvent solves, sampled passivity margins, and a symmetrized kernel Gram matrix, plus a reproducible linear-SSM comparison. The numerical check confirmed that a non-passive scalar response has negative H and kernel eigenvalue margins and that optimization can repair them. However, the learned toy SSM was already strongly passive without regularization, so CK produced essentially unchanged long-horizon MSE (2.958e-5 baseline vs 3.080e-5 idea) while adding about 2.58x runtime overhead; therefore no practical win was observed.
- Agent confidence
- 9/10
- Baseline
- train MSE 6.912e-6; long-horizon MSE 2.958e-5; H minimum 1.0480; kernel minimum 1.8868; runtime 9.93 s
- Idea
- train MSE 6.719e-6; long-horizon MSE 3.080e-5; H minimum 1.0481; kernel minimum 1.8867; runtime 25.64 s; overhead 2.58x
- Math check
- good response H min 2.0 and kernel min 3.553; bad response H min -0.4 and kernel min -7.198; gradient repair reached H min 64.56 and kernel min 114.70
Limitations: Only a small scalar-output, 8-state linear SSM and three fixed seeds were tested. The comparison omitted spectral-radius clipping and the proposed copy, adding, and ListOps tasks; the training fixture did not begin from substantially non-passive systems, so it cannot establish whether CK helps in genuinely unstable or non-passive regimes.
How to run: python3 experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- experiment.py 5.2 KB View Raw
- report.md 1.6 KB View
- results.json 2.6 KB View Raw