{
 "artifacts": [
  {
   "name": "experiment.py",
   "url": "https://synthcore.org/code/1218/experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1218/report.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1218/results.json"
  },
  {
   "name": "sweep.py",
   "url": "https://synthcore.org/code/1218/sweep.py"
  },
  {
   "name": "sweep_results.json",
   "url": "https://synthcore.org/code/1218/sweep_results.json"
  }
 ],
 "category": "dynamics",
 "description": "Replace an unconstrained continuous-depth neural update with a constrained implicit Runge–Kutta step whose internal stages and final state are required to remain in a convex feasible set. For box-constrained latent states, this prevents exploding or negative states while retaining the high-order structure of Radau or Gauss integration and avoiding the order-destroying behavior of post-step clipping.",
 "download_zip": "https://synthcore.org/download/1218",
 "formulas_latex": [
  "$$(y^{\\prime},v)=\\langle F_t(y),v\\rangle\\qquad\\forall v\\in\\mathcal V,$$",
  "$$Y_i=y_n+k\\sum_{j=1}^{s}a_{ij}f_\\theta(t_n+c_jk,Y_j),\\qquad y_{n+1}=y_n+k\\sum_{i=1}^{s}b_i f_\\theta(t_n+c_ik,Y_i),$$",
  "$$\\min_{Y_1,\\ldots,Y_s,z\\in\\mathcal K}\\;\\frac12\\sum_{i=1}^{s}\\left\\|Y_i-y_n-k\\sum_{j=1}^{s}a_{ij}f_\\theta(t_n+c_jk,Y_j)\\right\\|_2^2+\\frac{\\rho}{2}\\left\\|z-y_n-k\\sum_{i=1}^{s}b_if_\\theta(t_n+c_ik,Y_i)\\right\\|_2^2,$$",
  "$$\\Pi_{[\\ell,u]^d}(x)_r=\\min\\{u_r,\\max\\{\\ell_r,x_r\\}\\}.$$"
 ],
 "id": 3027,
 "implementation": "Integrate this at the latent-state update of a continuous-depth residual block; do not change the neural vector field $f_\\theta$. Choose a box $\\mathcal K=[\\ell,u]^d$, such as $[-4,4]^d$ for a latent state or $[0,1]^d$ for probability-like states. Use a two-stage implicit midpoint/Gauss method for the first MVP, or a three-stage Radau IIA method for a stiff benchmark. For each step, initialize every stage with $Y_i^{(0)}=y_n$ and solve the constrained nonlinear least-squares problem above using projected Gauss–Newton, projected L-BFGS, or Adam with an explicit box projection after every iteration. The pseudocode is: compute $r_i=Y_i-y_n-k\\sum_j a_{ij}f_\\theta(t_n+c_jk,Y_j)$ and $r_z=z-y_n-k\\sum_i b_if_\\theta(t_n+c_ik,Y_i)$; update all $Y_i,z$ to reduce $L_{RK}=\\frac12\\sum_i\\|r_i\\|_2^2+\\frac\\rho2\\|r_z\\|_2^2$; replace every variable by $\\Pi_{[\\ell,u]^d}$; stop when the residual norm reaches tolerance; set $y_{n+1}=z$. The exact coefficients $(A,b,c)$ are taken from the selected standard RK scheme, while the paper-derived component is the simultaneous feasible stage solve. Record residual norms, maximum constraint violation, number of inner iterations, and neural-vector-field evaluations. The first cheap experiment should use a small continuous-depth MLP on MNIST or a synthetic stiff classification system, comparing unconstrained RK, unconstrained RK followed by clipping, and the constrained solver at matched step sizes and function-evaluation budgets. Success means zero violations, fewer exploding trajectories, and equal or better validation accuracy at comparable wall-clock cost.",
 "math_summary": "The paper starts from the evolution equation $(y^{\\prime},v)=\\langle F_t(y),v\\rangle$ with a solution constrained to a closed convex set $\\mathcal K$. For a neural ODE, write $y'(t)=f_\\theta(t,y(t))$, where $y(t)\\in\\mathbb R^d$ and $f_\\theta$ is the neural vector field. Let $y_n$ be the current state, $k$ the time step, and let an $s$-stage implicit Runge–Kutta method have coefficients $A=(a_{ij})$, weights $b=(b_i)$, and nodes $c_i$. The unconstrained equations are $Y_i=y_n+k\\sum_{j=1}^s a_{ij}f_\\theta(t_n+c_jk,Y_j)$ and $y_{n+1}=y_n+k\\sum_{i=1}^s b_if_\\theta(t_n+c_ik,Y_i)$. The transferred monolithic construction solves a constrained optimization problem over all stages and the final state: minimize the squared Runge–Kutta residual while imposing $Y_i\\in\\mathcal K$ and $y_{n+1}\\in\\mathcal K$. Here $Y_i$ are stage states, $y_{n+1}$ is the next latent state, and $\\mathcal K=[\\ell,u]^d$ is a user-chosen box or another closed convex set. Convexity makes projection and feasible optimization well-defined; when the exact solution remains away from the boundary, the underlying implicit method retains its formal order.",
 "math_tags": [
  "convex-analysis",
  "dynamical-systems",
  "numerical-analysis"
 ],
 "ml_areas": [
  "training",
  "training-dynamics",
  "mlp"
 ],
 "paper": {
  "arxiv_id": "2609.01915",
  "arxiv_url": "https://arxiv.org/abs/2609.01915",
  "summary_what_math_gives_to_ml": "The paper develops a systematic way to make high-order time discretizations preserve a known convex feasible set, rather than relying on an unconstrained numerical trajectory followed by ad hoc clipping. Its most transferable asset is the combination of convex-set projection with implicit Runge–Kutta or multistep updates, which can preserve positivity or bounded latent states in continuous-depth neural networks. A second useful construction is the Bernstein basis: coefficient-wise bounds imply pointwise bounds through the convex-hull property, giving a bounded implicit-neural-representation output layer without dense pointwise constraint checking. The strongest experiments should compare projected and monolithic constrained integrators against clipping and unconstrained solvers at equal function evaluations, measuring both constraint violations and task accuracy.",
  "title": "Bounds-Constrained Finite Element Approximation of Time-Dependent Partial Differential Equations",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 6,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "accuracy"
 ],
 "title": "Feasible High-Order Neural ODE Solver",
 "url": "https://synthcore.org/idea/3027/feasible-high-order-neural-ode-solver",
 "verification": {
  "peer_reviewed": false,
  "stage1_mechanism_check": {
   "worked": false,
   "confidence": 8,
   "verdict": "Built a two-stage Gauss-Legendre implicit solver and the paper-style box-constrained nonlinear least-squares variant. The unconstrained method numerically achieved fourth-order convergence, with observed orders 4.016, 4.004, 4.001, and 4.000, while the constrained method guaranteed zero stage and final-state violations. However, on the stiff scalar benchmark it often had larger residuals and worse error at coarse step sizes, with many inner iterations; clipping matched unconstrained final-state accuracy in this test. Feasibility was demonstrated, but the claimed accuracy or cost win was not.",
   "metrics": {
    "baseline": "Gauss implicit RK achieved fourth-order convergence; for lambda=40 and n=2, final error was 0.3013 with maximum stage violation 0.1077; for n=10, final error was 7.25e-12.",
    "idea": "The constrained Gauss solve had zero constraint violation; for lambda=40 and n=2, final error was 0.00224 with mean residual 0.2501 and 16 inner iterations; for n=10, final error was zero with mean residual 0.00547 and 45 inner iterations. For lambda=10 and n=2, constrained error was 0.0632 versus baseline error 0.0109."
   },
   "how_to_run": "python3 experiment.py \u0026\u0026 /home/maxwelhelp/main/bin/python3 sweep.py",
   "files": [
    "experiment.py",
    "sweep.py",
    "results.json",
    "sweep_results.json"
   ],
   "limitations": "Only a one-dimensional synthetic stiff system was tested; no neural vector field, MNIST or classification task, Radau method, multidimensional boxes, GPU timing, FLOP-matched training, or comparison against a projected Newton/Gauss-Newton solver was evaluated. The constrained optimizer uses bounded L-BFGS-B and can terminate at nonzero residual near active constraints."
  },
  "status": "mechanism_failed",
  "status_label": "Mechanism failed",
  "updated_at": "2026-09-03T12:16:43",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": false,
    "tested": true
   }
  }
 }
}