{
 "artifacts": [
  {
   "name": "experiment.py",
   "url": "https://synthcore.org/code/1066/experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1066/report.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1066/results.json"
  }
 ],
 "category": "dynamics",
 "description": "Train a neural policy together with a positive neural Lyapunov function so that the learned closed-loop transition decreases the function at every sampled state in a prescribed operating region. This converts policy learning from an unconstrained reward problem into a constrained dissipativity problem and provides an inference-time monitor that can reject or damp actions when the certificate is violated.",
 "download_zip": "https://synthcore.org/download/1066",
 "formulas_latex": [
  "$$V_{\\psi}(x^{\\star})=0,\\qquad V_{\\psi}(x)\\geq c_{1}\\|x-x^{\\star}\\|_{2}^{2},\\qquad c_{1}\u003e0,$$",
  "$$\\Delta V_{\\theta,\\phi,\\psi}(x)=V_{\\psi}\\!\\left(\\hat f_{\\phi}(x,\\pi_{\\theta}(x))\\right)-V_{\\psi}(x)\\leq -c_{2}\\|x-x^{\\star}\\|_{2}^{2},\\qquad c_{2}\u003e0,$$",
  "$$\\mathcal{L}=\\mathcal{L}_{\\mathrm{task}}+\\lambda_{V}\\mathbb{E}_{x\\sim\\mathcal{D}}[\\operatorname{softplus}(\\varepsilon_{V}-V_{\\psi}(x))]+\\lambda_{\\Delta}\\mathbb{E}_{x\\sim\\mathcal{D}}[\\operatorname{softplus}(\\Delta V_{\\theta,\\phi,\\psi}(x)+c_{2}\\|x-x^{\\star}\\|_{2}^{2})],$$",
  "$$V_{t+1}\\leq\\left(1-\\frac{c_{2}}{c_{1}}\\right)V_{t}\\quad\\Longrightarrow\\quad V_{t}\\leq\\left(1-\\frac{c_{2}}{c_{1}}\\right)^{t}V_{0},\\qquad 0\u003cc_{2}\u003cc_{1}.$$"
 ],
 "id": 2823,
 "implementation": "1. Integration point: use this in model-based RL, recurrent world-model rollout training, or any neural state-transition module where stability of generated trajectories matters. Maintain three networks: a dynamics model $\\hat f_{\\phi}$, policy $\\pi_{\\theta}$, and Lyapunov network $V_{\\psi}$. Parameterize $V_{\\psi}(x)=\\|g_{\\psi}(x)-g_{\\psi}(x^{\\star})\\|_{2}^{2}+\\epsilon\\|x-x^{\\star}\\|_{2}^{2}$ with $\\epsilon\u003e0$ to make positivity automatic. 2. Pseudocode: sample states $x$ from replay data and perturbations around the equilibrium; compute $u=\\pi_{\\theta}(x)$, $x^{+}=\\hat f_{\\phi}(x,u)$, and $dV=V_{\\psi}(x^{+})-V_{\\psi}(x)$; add the two hinge terms in the displayed loss; update $\\theta,\\psi$ jointly and update $\\phi$ using the transition prediction loss. Every $K$ steps, freeze the networks and test $V(x)\u003e0$ and $dV+c_{2}\\|x-x^{\\star}\\|^{2}\\leq0$ on a dense grid or interval boxes. 3. Computed quantities are the one-step model prediction, Lyapunov values, and violation margins. Model error and the largest verified region must be estimated from held-out transitions or interval bounds. 4. First cheap experiment: train a two-layer policy and Lyapunov network on a two-dimensional nonlinear benchmark such as $x_{t+1}=x_t+0.05[-x_{1}^{3}+u,\\ x_{2}+0.2x_{1}]$, comparing PPO or supervised control against the certificate-regularized version. Sweep $\\lambda_{\\Delta}$ and estimate the largest radius $r$ for which the decrease condition holds. The quantitative prediction is that certified trajectories satisfy $V_t/V_0\\leq(1-c_{2}/c_{1})^t$ up to model error, and empirical divergence begins near the radius or parameter value where the measured minimum decrease margin crosses zero.",
 "math_summary": "Let $x\\in\\mathbb{R}^{n}$ be the state, $u=\\pi_{\\theta}(x)$ the neural controller with parameters $\\theta$, and $\\hat f_{\\phi}(x,u)$ the identified one-step dynamics model with parameters $\\phi$. Let $V_{\\psi}(x)$ be a neural Lyapunov candidate with parameters $\\psi$, equilibrium $x^{\\star}$, and region $\\mathcal{X}$. Enforce $V_{\\psi}(x^{\\star})=0$, $V_{\\psi}(x)\\geq c_{1}\\|x-x^{\\star}\\|_{2}^{2}$, and $V_{\\psi}(\\hat f_{\\phi}(x,\\pi_{\\theta}(x))) - V_{\\psi}(x)\\leq -c_{2}\\|x-x^{\\star}\\|_{2}^{2}$ for all $x\\in\\mathcal{X}$, where $c_{1},c_{2}\u003e0$. The decrease inequality implies geometric decay of the sampled closed-loop energy when the dynamics are discrete-time. In practice, optimize hinge penalties for violations and periodically verify the inequalities over boxes or a sampled state set; the certified subset is the set on which all inequalities hold.",
 "math_tags": [
  "control-theory",
  "dynamical-systems",
  "optimization",
  "linear-algebra"
 ],
 "ml_areas": [
  "rl",
  "training-dynamics",
  "world-model",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2608.29303",
  "arxiv_url": "https://arxiv.org/abs/2608.29303",
  "summary_what_math_gives_to_ml": "The paper offers a constructive pipeline for learning feedback policies for unknown nonlinear dynamics: identify a model from offline data, jointly synthesize a neural controller and neural Lyapunov function, enforce inputs with hard saturation, and certify the result using SMT plus local Lyapunov analysis. The transferable asset is a closed-loop certificate that turns policy learning into a falsifiable decrease-condition problem. A strong neural-network transfer is to train policies or recurrent state updates together with an energy function whose one-step change is negative on a specified region, while explicitly accounting for model-identification error. The most useful experimental signature is a certified region of attraction and a sharp failure boundary when the Lyapunov decrease margin becomes nonpositive.",
  "title": "Learning neural controllers for nonlinear systems from data",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 8
 },
 "solves": [
  "stability",
  "accuracy",
  "generalization"
 ],
 "title": "Lyapunov-Certified Policy Training",
 "url": "https://synthcore.org/idea/2823/lyapunov-certified-policy-training",
 "verification": {
  "peer_reviewed": false,
  "stage1_mechanism_check": {
   "worked": false,
   "confidence": 9,
   "verdict": "Built a PyTorch MVP with an equilibrium-preserving policy, positive Lyapunov network, hinge certificate loss, grid verification, radius sweep, and rollout monitoring in experiment.py. The algebraic recurrence V_{t+1}\u003c=qV_t was verified synthetically, and increasing certificate weight reduced sampled violation from 0.934 (lambda=0) to 0.00030 (lambda=4), but the claimed contraction factor q=0.6875 was not observed: certified one-step V ratios were roughly 0.90-0.99. This exposes a limitation in the stated math: V\u003e=c1||x||^2 alone cannot convert decrease proportional to ||x||^2 into geometric decay in V; an upper bound V\u003c=c3||x||^2 is also required. Therefore the practical penalty effect appeared, but the authoritative Lyapunov contraction claim was not confirmed.",
   "metrics": {
    "baseline": "lambda_delta=0: task MSE 7.49e-06, mean hinge violation 0.9340, certified fraction 72.9%, certified radius 0.0, rollout V30/V0=1.208",
    "idea": "lambda_delta=4: task MSE 4.32e-03, mean hinge violation 2.96e-04, certified fraction 99.57%, certified radius 0.35 in the main grid, certified one-step ratio median 0.897 and max 0.978; rollout V30/V0=0.00071"
   },
   "how_to_run": "python3 experiment.py",
   "files": [
    "experiment.py",
    "results.json",
    "run4.log"
   ],
   "limitations": "Only a tiny 2D deterministic benchmark was tested; no learned dynamics model, transition-model error, held-out generalization, interval certification, PPO/RL training, action rejection monitor, or multiple random seeds was evaluated. The radius sweep has grid-resolution and near-equilibrium numerical sensitivity."
  },
  "status": "mechanism_failed",
  "status_label": "Mechanism failed",
  "updated_at": "2026-09-01T23:36:34",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}