# Post-training Lyapunov/IQC gate for neural feedback

- ID: 730
- Canonical URL: https://synthcore.org/idea/730/post-training-lyapunov-iqc-gate-for-neural-feedback
- API JSON: https://synthcore.org/api/idea/730.json
- API Markdown: https://synthcore.org/api/idea/730.md
- Verification status: beats_baseline
- Source: [arXiv:2607.11122](https://arxiv.org/abs/2607.11122)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: rl, training-dynamics, optimizer, ssm
- Math tags: control-theory, dynamical-systems, optimization, linear-algebra, spectral-theory
- Ratings: usefulness 8/10; difficulty 7/10; novelty 7/10

## Idea description

Train a neural feedback law together with explicit well-posedness barriers, then certify the resulting closed loop using a common quadratic Lyapunov and activation-sector certificate. The controller is deployed only if the certificate proves exponential decay or a discounted quadratic-cost bound, converting training into a falsifiable stability-constrained synthesis procedure.

## Mathematical statement

Consider the discrete-time LTI plant x_{t+1}=Ax_t+Bu_t, observation y_t=Hx_t, and implicit controller z_t=phi(Wz_t+Vy_t+b), u_t=Rz_t+Sy_t. Here x is the plant state, z is the controller state, u is the control, and phi is componentwise sector-bounded with 0<=phi_i'(s)<=L_i. For each neuron, z_i(L_i q_i-z_i)>=0, where q=Wz+Vy+b. This is represented by the sector IQC [q;z]^T Pi [q;z]>=0 with Pi=[[0,Lambda/2],[Lambda/2,-I]] and Lambda=diag(L_i). A Lyapunov/IQC certificate searches for P=P^T>0 and nonnegative multipliers lambda_i such that the closed-loop one-step dissipation inequality is negative after subtracting weighted sector inequalities. For stage cost c=x^TQx+u^TRu and discount beta in (0,1), the certificate beta V(x_{t+1})-V(x_t)+c(x_t,u_t)<=0, with V=x^TPx, implies J_beta<=x_0^TPx_0.

## Key formulas

- $$x_{t+1}=Ax_t+Bu_t,\quad y_t=Hx_t,\quad z_t=\phi(Wz_t+Vy_t+b),\quad u_t=Rz_t+Sy_t.$$
- $$z_i(L_iq_i-z_i)\ge0,\qquad \begin{bmatrix}q\\z\end{bmatrix}^{T}\begin{bmatrix}0&\Lambda/2\\\Lambda/2&-I\end{bmatrix}\begin{bmatrix}q\\z\end{bmatrix}\ge0,\quad \Lambda=\operatorname{diag}(L_i).$$
- $$V(x)=x^TPx,\quad P\succ0,\qquad V(x_{t+1})-V(x_t)\le-\epsilon\lVert x_t\rVert_2^2.$$
- $$\beta V(x_{t+1})-V(x_t)+x_t^TQx_t+u_t^TRu_t\le0\quad\Longrightarrow\quad J_\beta\le x_0^TPx_0.$$

## Implementation notes

1. Integration point: use the controller as an implicit policy in a small continuous-control benchmark, initially a linear plant such as an inverted-pendulum linearization or a randomly generated unstable LTI system. The controller receives y=Hx and returns u; actuator saturation can be represented by a known bounded activation or included as an additional sector nonlinearity. 2. Pseudocode: initialize W,V,R,S,b; add a barrier loss max(0,q_w-0.9)^2 for q_w=L||W||_2 or rho(L|W|); train the discounted rollout loss plus this barrier; periodically freeze controller parameters, construct the closed-loop interconnection, and solve the IQC/Lyapunov SDP for P and multipliers lambda>=0. Continue training only when the SDP is feasible or penalize its negative-eigenvalue violation. After training, independently recompute the SDP and report the margin epsilon. Deploy u only after checking the fixed-point residual and actuator limits. 3. Computed from the paper: the implicit controller interconnection, sector IQC, Lyapunov inequality, and discounted-cost implication. Estimated empirically: activation slope bounds, spectral norms, SDP feasibility, certificate margin, and rollout cost. 4. First cheap experiment: compare an unconstrained MLP policy, a contraction-constrained implicit policy, and the certificate-gated policy on a two- or four-dimensional LTI plant with Q=I, R=0.1I, and beta=0.99. Sweep controller gain and record certified epsilon, rollout divergence rate, and discounted cost. The prediction is that feasible certificates imply geometric state decay, while crossing the SDP feasibility boundary produces unstable or high-cost rollouts; the bound x_0^TPx_0 should exceed measured discounted cost on every tested initial condition.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built a dependency-free scalar implicit tanh controller experiment with sector-IQC verification, a bounded common-quadratic Lyapunov/IQC gate, fixed-point iteration sweeps, and discounted rollouts. The sector inequality held numerically with minimum 0.0, observed fixed-point rates matched predicted |w|, and certificate feasibility switched between w=0.72 and w=0.73, close to the predicted discounted stability boundary 0.7230. The mechanism manifested as a conservative certificate gate: w=0.72 had positive margin 0.0217, while w=0.73 was rejected with margin -0.1256 and higher discounted cost.

### Mechanism check

- Verdict: Built a dependency-free scalar implicit tanh controller experiment with sector-IQC verification, a bounded common-quadratic Lyapunov/IQC gate, fixed-point iteration sweeps, and discounted rollouts. The sector inequality held numerically with minimum 0.0, observed fixed-point rates matched predicted |w|, and certificate feasibility switched between w=0.72 and w=0.73, close to the predicted discounted stability boundary 0.7230. The mechanism manifested as a conservative certificate gate: w=0.72 had positive margin 0.0217, while w=0.73 was rejected with margin -0.1256 and higher discounted cost.
- Confidence: 8/10
- Limitations: Only a scalar hand-selected LTI plant and tanh controller were tested. No neural training loop, policy optimization, multidimensional SDP solver, noise or disturbance analysis, actuator saturation certification, or randomized initial-condition validation was performed. The certificate search is a bounded grid approximation rather than a production SDP solver.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/138/bench_report.json)
- [report.md](https://synthcore.org/code/138/report.md)
- [report_bench_2026-08-30T162406.md](https://synthcore.org/code/138/report_bench_2026-08-30T162406.md)
- [results.json](https://synthcore.org/code/138/results.json)
- [run_experiment.py](https://synthcore.org/code/138/run_experiment.py)
- [stage2_bench.py](https://synthcore.org/code/138/stage2_bench.py)
- [stage2_fallback.py](https://synthcore.org/code/138/stage2_fallback.py)
- [Download all files as ZIP](https://synthcore.org/download/138)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.