{
 "artifacts": null,
 "category": "regularization",
 "description": "For each training example, locally linearize the network and attach a certified residual radius obtained from verifier affine relaxations and a path-based Hessian remainder. Penalize the worst-case class-margin violation under this residual instead of using a global Lipschitz penalty. This should encourage locally affine, robust representations while retaining much tighter certificates than global spectral-norm bounds.",
 "formulas_latex": [
  "$$r_j(\\delta)=f_{\\theta,j}(x_0+\\delta)-f_{\\theta,j}(x_0)-J_{0,j:}\\delta=\\int_0^1(1-t)\\,\\delta^\\top H_j(x_0+t\\delta)\\delta\\,dt,$$",
  "$$|r_j(\\delta)|\\le \\frac{1}{2}\\max_{t\\in[0,1]}\\left|\\delta^\\top H_j(x_0+t\\delta)\\delta\\right|\\le \\rho_j:=\\frac{1}{2}\\sum_{a=1}^{d}\\sum_{b=1}^{d}\\widehat H_{j,ab}\\epsilon_a\\epsilon_b,$$",
  "$$m_{y,k}=f_{\\theta,y}(x_0)-f_{\\theta,k}(x_0)-\\left\\|(J_{0,y:}-J_{0,k:})\\odot\\epsilon\\right\\|_1-(\\rho_y+\\rho_k),\\qquad \\mathcal{L}_{\\mathrm{cert}}=\\sum_{k\\ne y}\\operatorname{softplus}(-m_{y,k}/\\tau).$$"
 ],
 "id": 326,
 "implementation": "(1) Integration point: add this term to the training loss of an MLP, CNN, or transformer classifier immediately after computing logits. Use x_0 as the clean minibatch input, and certify a small input box with half-width vector epsilon; initially use uniform epsilon equal to 1/255 for images or 0.01 times the per-feature standard deviation for tabular data. Compute logits f_theta(x_0) and the Jacobian J_0 with forward-mode or reverse-mode automatic differentiation.\n\n(2) Pseudocode:\n```\nlogits = f_theta(x0)\nJ = jacobian(f_theta, x0)\nfor each output j:\n    rho_hess[j] = 0.5 * sum_ab(Hhat[j,a,b] * eps[a] * eps[b])\n    rho_ver[j] = verifier_residual_radius(f_theta, x0, eps, j)\n    rho[j] = min(rho_hess[j], rho_ver[j])\nfor k != y:\n    grad_margin = abs((J[y]-J[k]) * eps).sum()\n    margin[k] = logits[y]-logits[k]-grad_margin-rho[y]-rho[k]\nL = cross_entropy(logits, y) + lambda_cert * sum(softplus(-margin/tau))\nbackprop(L)\n```\nThe verifier residual should be computed by propagating affine lower and upper bounds through each layer, then subtracting the local affine prediction f_theta(x_0) + J_0 delta; retain a stop-gradient copy of bound slopes if full verifier differentiation is unstable.\n\n(3) What comes from the mathematics: the integral Taylor remainder, the quadratic Hessian bound, and the certified margin inequality. Estimate Hhat cheaply by evaluating Hessian-vector products at M points x_0 + t epsilon sign vectors, or use interval/autodiff bounds for small M; estimate verifier bounds with existing CROWN-style affine propagation. The bound is conservative when M is finite, so log the empirical maximum remainder over random perturbations and the certificate slack.\n\n(4) First experiment: train a 2-layer ReLU MLP and a small ResNet on MNIST or CIFAR-10 with ordinary cross-entropy, global spectral-norm regularization, and this loss. Use 1000 held-out examples and PGD at the same epsilon. Success means higher certified accuracy and PGD accuracy at equal clean accuracy, with a smaller empirical-to-certified margin gap. Also compare wall-clock overhead and test whether using verifier residuals permits a larger epsilon than a global Lipschitz baseline.",
 "math_summary": "The core construction is the second-order path remainder for a vector-valued network f_theta: R^d -\u003e R^K. At center x_0, let delta = x - x_0, J_0 = grad f_theta(x_0), and H_j(x) = Hessian of f_theta,j(x) for output coordinate j. Taylor's integral identity gives r_j(delta) = f_theta,j(x_0 + delta) - f_theta,j(x_0) - J_0,j: delta = integral from 0 to 1 of (1-t) delta^T H_j(x_0+t delta) delta dt, hence |r_j(delta)| \u003c= one half times the maximum over t in [0,1] of |delta^T H_j(x_0+t delta) delta|. For a box perturbation |delta_i| \u003c= epsilon_i, a directly implementable conservative bound is rho_j = one half sum over a,b of Hhat_jab epsilon_a epsilon_b, where Hhat_jab \u003e= sup over t and delta of |partial_ab f_theta,j(x_0+t delta)|. The paper's NN-specific improvement is to first compute an affine verifier relaxation l_j(x) \u003c= f_theta,j(x) \u003c= u_j(x), then apply a local Jacobian correction around x_0; operationally this replaces the loose rho_j by a residual half-width estimated from the verifier. For class y and competitor k, the certified lower margin is m_yk = f_y(x_0) - f_k(x_0) - ||(J_0,y: - J_0,k:) elementwise-multiplied by epsilon||_1 - (rho_y + rho_k), where the elementwise product accounts for the input box. Minimize the softplus of negative certified margins divided by temperature tau.",
 "math_tags": [
  "convex-analysis",
  "linear-algebra",
  "optimization",
  "numerical-analysis",
  "geometry"
 ],
 "ml_areas": [
  "loss",
  "regularization",
  "training",
  "mlp"
 ],
 "paper": {
  "arxiv_id": "2607.01203",
  "arxiv_url": "https://arxiv.org/abs/2607.01203",
  "summary_what_math_gives_to_ml": "The paper develops differentiable, tight bounds on the error made when replacing nonlinear or neural dynamics by a local affine model. The transferable asset is the combination of path-based second-order Taylor bounds, verifier-generated affine relaxations, and local Jacobian corrections: it produces a computable residual set rather than relying on a loose global Lipschitz constant. In a neural classifier or policy, the same residual set can become a certified local robustness loss, an adaptive perturbation radius, or a certificate-guided data augmentation rule. The most practical first transfer is a differentiable margin regularizer that penalizes samples whose affine prediction plus certified residual cannot preserve the correct class or action.",
  "title": "GPU-Parallel Linearization Error Bounds for Real-Time Robust Optimal Control of Nonlinear and Neural Network Dynamics",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 5,
  "usefulness": 7
 },
 "solves": [
  "accuracy",
  "stability",
  "generalization"
 ],
 "title": "Path-Certified Local Robustness Loss",
 "url": "https://synthcore.org/idea/326/path-certified-local-robustness-loss",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}