{
 "artifacts": null,
 "category": "architecture",
 "description": "Replace an unconstrained token mixer or feed-forward residual map with a feature-conditioned rational operator whose transfer matrix is positive real on the positive orthant. Apply it through a resolvent, rather than an additive residual, so that the accretivity certificate yields a nonexpansive implicit update and suppresses activation explosions.",
 "formulas_latex": [
  "$$H(\\zeta)=D+C\\bigl(P(\\zeta)+A\\bigr)^{-1}B,\\qquad P(\\zeta)=\\zeta_{1}P_{1}+\\zeta_{2}P_{2}.$$",
  "$$\\begin{bmatrix}A+A^{*}\u0026C^{*}-B\\\\ C-B^{*}\u0026D+D^{*}\\end{bmatrix}\\geq 0.$$",
  "$$\\operatorname{Re}H(\\zeta)=\\frac{H(\\zeta)+H(\\zeta)^{*}}{2}\\geq0,\\qquad \\left\\|(I_d+\\eta H(\\zeta))^{-1}\\right\\|_2\\leq1.$$",
  "$$z_j(x)=\\epsilon+\\operatorname{softplus}(w_j^{\\top}q(x)),\\qquad y=(I_d+\\eta[D+C(z_1P_1+z_2P_2+A)^{-1}B])^{-1}x.$$"
 ],
 "id": 3119,
 "implementation": "Integrate this at the token-mixing or FFN-residual point of a small Transformer or sequence model. Let $x\\in\\mathbb{R}^d$ be one token vector, let $q(x)$ be either $x$ or a cheap pooled context vector, and compute two strictly positive scalar coordinates $z_j=\\epsilon+\\operatorname{softplus}(w_j^\\top q)$. Use a shared state size $r\\ll d$ across all tokens. Initialize $P_1$ as a diagonal projection onto the first $r/2$ state coordinates and set $P_2=I_r-P_1$; optionally replace these by $P_1=Q\\operatorname{diag}(1_{r/2},0)Q^\\top$ and $P_2=I-P_1$, with $Q$ parameterized by a Cayley transform. For each token, form $M=A+z_1P_1+z_2P_2$, solve $MU=B x$, and compute $h(x)=D x+C U$. Then solve $(I_d+\\eta H)x'=x$ for the output $x'$, using a Cholesky solve when the symmetric part is explicitly symmetrized, or a stable LU solve otherwise; add a standard gated projection around this block if dimensionality changes. During training enforce the paper's KYP certificate by forming $K=\\begin{bmatrix}A+A^\\top\u0026C^\\top-B\\\\C-B^\\top\u0026D+D^\\top\\end{bmatrix}$ and adding $\\lambda\\sum_i\\operatorname{softplus}(\\delta-\\lambda_i(K))^2$ to the loss, where $\\lambda_i(K)$ are eigenvalues and $\\delta\u003e0$ is a desired margin. A stronger variant projects $K$ onto the PSD cone after every optimizer step and reconstructs the block variables. The paper's mathematical computation is the rational realization and KYP constraint; $z_j$, the KYP margin, and $\\eta$ are learned or tuned. First test on CIFAR-10 with a 6-layer width-128 Transformer and on the Adding or Copying synthetic long-range sequence task, comparing against a standard FFN/token mixer with equal parameter count. Measure loss versus optimizer steps, maximum activation norm, gradient norm, failed linear solves, and validation accuracy. Success is a substantially lower tail of activation and gradient norms with no loss of accuracy, or faster loss descent at equal FLOPs; also report the overhead from the $r\\times r$ solve and test whether $r=16$ matches a much larger unconstrained mixer.",
 "math_summary": "The paper realizes a matrix-valued Herglotz function as $H(\\zeta)=D+C(P(\\zeta)+A)^{-1}B$, where $\\zeta=(\\zeta_1,\\zeta_2)\\in\\mathbb{C}_+^2$, $P(\\zeta)=\\zeta_1P_1+\\zeta_2P_2$, $A\\in\\mathbb{C}^{r\\times r}$ is the state matrix, $B\\in\\mathbb{C}^{r\\times d}$, $C\\in\\mathbb{C}^{d\\times r}$, $D\\in\\mathbb{C}^{d\\times d}$, and $P_1,P_2\\in\\mathbb{C}^{r\\times r}$ are orthogonal projections satisfying $P_j=P_j^*$, $P_j^2=P_j$, and $P_1+P_2=I_r$. The paper's KYP condition is $\\begin{bmatrix}A+A^*\u0026C^*-B\\\\C-B^*\u0026D+D^*\\end{bmatrix}\\geq0$, where $X^*$ denotes conjugate transpose and $X\\geq0$ means positive semidefinite. Under this condition, $H$ has nonnegative Hermitian part on the bihalfplane: $\\operatorname{Re}H(\\zeta):=(H(\\zeta)+H(\\zeta)^*)/2\\geq0$. For real neural tensors use real matrices and evaluate only $\\zeta_j\\geq\\epsilon\u003e0$. The adapted layer is $y=(I_d+\\eta H(\\zeta(x)))^{-1}x$ with $\\eta\u003e0$; accretivity implies the resolvent bound $\\|(I_d+\\eta H(\\zeta))^{-1}\\|_2\\leq1$, giving a direct stability target.",
 "math_tags": [
  "linear-algebra",
  "control-theory",
  "dynamical-systems"
 ],
 "ml_areas": [
  "ssm",
  "transformer",
  "training"
 ],
 "paper": {
  "arxiv_id": "2609.03054",
  "arxiv_url": "https://arxiv.org/abs/2609.03054",
  "summary_what_math_gives_to_ml": "The paper gives finite-dimensional state-space realizations for multivariate rational maps whose positivity or contractivity is certified by explicit matrix inequalities. The transferable asset is the positive-real/Herglotz realization: a rational feature mixer can be parameterized so that its symmetric part is positive semidefinite over an entire positive-input domain, rather than merely being stable at sampled training points. A particularly promising neural adaptation is to use such a map inside an implicit resolvent layer, whose accretivity gives a built-in nonexpansive stability mechanism. This could provide a compact alternative to unconstrained token mixers or state-space blocks, especially when feature-dependent rational filtering is desired.",
  "title": "The classes of bivariate Schur and Herglotz matrix-valued rational functions: realizations, symmetrizations, and related determinantal representations",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 7,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "accuracy",
  "scalability"
 ],
 "title": "Positive-real rational resolvent mixer",
 "url": "https://synthcore.org/idea/3119/positive-real-rational-resolvent-mixer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}