# Caratheodory-kernel passivity regularizer

- ID: 429
- Canonical URL: https://synthcore.org/idea/429/caratheodory-kernel-passivity-regularizer
- API JSON: https://synthcore.org/api/idea/429.json
- API Markdown: https://synthcore.org/api/idea/429.md
- Verification status: mechanism_failed
- Source: [arXiv:2607.05334](https://arxiv.org/abs/2607.05334)
- Category: regularization
- Solves: stability, generalization, accuracy
- ML areas: ssm, rnn, regularization, training-dynamics
- Math tags: functional-analysis, harmonic-analysis, linear-algebra, operator-theory
- Ratings: usefulness 6/10; difficulty 6/10; novelty 7/10

## Idea description

Regularize a learned state-space transfer function so its matrix response has positive real part on sampled points in the unit disk and its associated reproducing-kernel Gram matrix is positive semidefinite. This provides a frequency-domain stability signal that complements rollout-based penalties and spectral-radius clipping.

## Mathematical statement

The paper assumes that Phi(lambda) is an n-by-n analytic matrix function on the unit disk D with positive real part, meaning Herm(Phi(lambda)) = (Phi(lambda)+Phi(lambda)^*)/2 is positive semidefinite. Its Herglotz representation is Phi(lambda) = iA + integral over the unit circle of (e^(it)+lambda)/(e^(it)-lambda) dmu(e^(it)), where A satisfies A+A^*=0 and mu is a positive matrix-valued measure. The associated reproducing kernel is L_Phi(lambda,w) = [Phi(lambda)+Phi(w)^*]/[2(1-lambda conjugate(w))], and finite Gram matrices formed from this kernel are positive semidefinite. For a neural state-space model with transfer response R_theta(lambda) = C(I-lambda A_theta)^(-1)B, define the adapted response Phi_theta(lambda) = I + R_theta(lambda). Penalize violations of Herm(Phi_theta(lambda)) being positive semidefinite and violations of the kernel Gram matrix K being positive semidefinite. The eigenvalue margins are directly computable from finite matrices.

## Key formulas

- $$\Phi(\lambda)=iA+\int_{0}^{2\pi}\frac{e^{it}+\lambda}{e^{it}-\lambda}\,d\mu(e^{it}),\qquad A+A^{*}=0.$$
- $$L_{\Phi}(\lambda,w)=\frac{\Phi(\lambda)+\Phi(w)^{*}}{2(1-\lambda\overline{w})}.$$
- $$\Phi_{\theta}(\lambda)=I+C(I-\lambda A_{\theta})^{-1}B,\qquad H_{\theta}(\lambda)=\frac{\Phi_{\theta}(\lambda)+\Phi_{\theta}(\lambda)^{*}}{2}\succeq 0.$$
- $$\mathcal{L}_{\mathrm{CK}}=\frac{1}{M}\sum_{j=1}^{M}\left[\max\left(0,\varepsilon-\lambda_{\min}(H_{\theta}(\lambda_j))\right)\right]^2+\frac{1}{M^2}\left[\max\left(0,\varepsilon-\lambda_{\min}(K)\right)\right]^2,\quad K_{ij}=L_{\Phi_{\theta}}(\lambda_i,\lambda_j).$$

## Implementation notes

Apply the regularizer to a linear SSM or to a frozen-point linearization of an RNN. The exact integration point is the training loss, after constructing the transition matrix A_theta and input/output matrices B and C; the forward recurrence remains unchanged. Choose M complex sample points lambda_j = rho_j exp(i omega_j), with rho_j between 0.8 and 0.99 and omega_j uniformly covering [0, 2 pi). For each point, solve the linear system (I - lambda_j A_theta) X_j = B rather than explicitly forming an inverse, then compute Phi_j = I + C X_j and H_j = (Phi_j + Phi_j^*)/2. Penalize the negative eigenvalue margin using relu(epsilon - lambda_min(H_j)) squared. For a second test, select m at most 16 sample points, form K[i,j] = (Phi_i + Phi_j^*)/(2(1 - lambda_i conjugate(lambda_j))), symmetrize K as (K+K^*)/2, and add relu(epsilon - lambda_min(K)) squared. Pseudocode: sample lambdas; solve each resolvent system; calculate Phi and H; calculate minimum Hermitian eigenvalues; build K; add alpha times the total penalty to the task loss. The Herglotz representation and kernel formula come from the paper; the frequency grid, epsilon, penalty coefficient alpha, and identity offset are empirical design choices. Begin with a 64-state SSM on copy, adding, and ListOps tasks. Compare no regularizer, spectral-radius clipping, and Caratheodory-kernel regularization with matched optimization settings. Track the minimum sampled eigenvalue, rollout error at horizons 2x and 4x training length, gradient norms, validation loss, and wall-clock overhead. Success means improved long-horizon extrapolation and a nonnegative or near-nonnegative response margin without degrading short-context accuracy by more than a small tolerance.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a differentiable Caratheodory-kernel regularizer using resolvent solves, sampled passivity margins, and a symmetrized kernel Gram matrix, plus a reproducible linear-SSM comparison. The numerical check confirmed that a non-passive scalar response has negative H and kernel eigenvalue margins and that optimization can repair them. However, the learned toy SSM was already strongly passive without regularization, so CK produced essentially unchanged long-horizon MSE (2.958e-5 baseline vs 3.080e-5 idea) while adding about 2.58x runtime overhead; therefore no practical win was observed.

### Mechanism check

- Verdict: Built a differentiable Caratheodory-kernel regularizer using resolvent solves, sampled passivity margins, and a symmetrized kernel Gram matrix, plus a reproducible linear-SSM comparison. The numerical check confirmed that a non-passive scalar response has negative H and kernel eigenvalue margins and that optimization can repair them. However, the learned toy SSM was already strongly passive without regularization, so CK produced essentially unchanged long-horizon MSE (2.958e-5 baseline vs 3.080e-5 idea) while adding about 2.58x runtime overhead; therefore no practical win was observed.
- Confidence: 9/10
- Limitations: Only a small scalar-output, 8-state linear SSM and three fixed seeds were tested. The comparison omitted spectral-radius clipping and the proposed copy, adding, and ListOps tasks; the training fixture did not begin from substantially non-passive systems, so it cannot establish whether CK helps in genuinely unstable or non-passive regimes.

## Artifacts

- [experiment.py](https://synthcore.org/code/1299/experiment.py)
- [report.md](https://synthcore.org/code/1299/report.md)
- [results.json](https://synthcore.org/code/1299/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1299)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.