{
 "artifacts": null,
 "category": "optimization",
 "description": "Replace a hard nonnegative slack or ReLU output by the barrier-derived map \\(x_s(w)=\\frac{w+\\sqrt{w^2+4s}}{2}\\). Unlike an arbitrary smooth activation, this output is the unique positive solution of \\(x(x-w)=s\\), so the network can explicitly monitor complementarity and anneal \\(s\\) toward the true inequality-constrained solution. Use it in a constrained output head or in hidden layers whose activations represent nonnegative resource, probability, or routing slack variables.",
 "formulas_latex": [
  "$$\\varphi_{+,s}(w)=\\frac{w}{2}+\\sqrt{\\frac{w^{2}}{4}+s}=\\frac{w+\\sqrt{w^2+4s}}{2},\\qquad s\u003e0.$$",
  "$$x=\\varphi_{+,s}(w)\u003e0\\quad\\Longleftrightarrow\\quad x^2-wx-s=0\\quad\\Longleftrightarrow\\quad x-w-\\frac{s}{x}=0.$$",
  "$$F_s(x;w)=\\frac12(x-w)^2-s\\log x,\\quad x\u003e0,\\qquad \\partial_xF_s=x-w-\\frac{s}{x}=0.$$",
  "$$\\frac{\\partial\\varphi_{+,s}}{\\partial w}=\\frac12+\\frac{w}{2\\sqrt{w^2+4s}},\\qquad 0\\leq\\varphi_{+,s}(w)-[w]_+\\leq\\sqrt{s}.$$"
 ],
 "id": 2892,
 "implementation": "(1) Integration point: add a constrained head after an MLP, transformer projection, or router logits. Let the unconstrained tensor be \\(w=f_\\theta(z)\\); apply the elementwise central-path layer \\(x=\\varphi_{+,s}(w)\\) wherever the model requires a strictly positive slack, nonnegative allocation, positive intensity, or nonnegative mixture weight. For a simplex output, use \\(x_i=\\varphi_{+,s}(w_i)\\) followed by normalization \\(p_i=x_i/(\\sum_jx_j+\\epsilon)\\). Do not use this layer where exact zeros are essential during early training unless an explicit threshold is added only at inference. (2) Pseudocode: `w = net(z); x = 0.5*(w + sqrt(w*w + 4*s)); rc = x*(x-w)-s; task_loss = L(x,y); loss = task_loss + lambda_c*mean(rc*rc); loss.backward(); update(theta); s = max(s_min, rho*s)` with \\(0\u003c\\rho\u003c1\\). The analytic derivative in backpropagation is \\(0.5+0.5*w/sqrt(w*w+4*s)\\). Start with \\(s_0\\) equal to \\(0.01\\) times the median squared magnitude of \\(w\\), then reduce it only when the task loss and gradient norm are stable; compare geometric annealing with fixed \\(s\\). (3) Computed from the mathematics: the forward map, derivative, positivity guarantee, and complementarity residual formula. Estimated empirically: the initial scale, annealing rate, task-dependent tradeoff \\(\\lambda_c\\), and whether the uniform \\(\\sqrt{s}\\) approximation error is visible on the learned output distribution. Monitor minimum \\(x\\), mean and maximum \\(|r_c|\\), gradient variance, and fraction of outputs below \\(2\\sqrt{s}\\). (4) First experiment: train a small MLP on nonnegative regression, such as predicting Poisson rates from MNIST or synthetic \\(y=\\exp(a^Tz)+\\) noise, comparing ReLU, softplus, exponential, and this layer at matched parameter count and FLOPs. Then test a constrained mixture head on CIFAR-10 with cross-entropy after normalization. The expected signal is fewer NaNs and lower gradient variance than exponential outputs, lower final constraint violation than an unconstrained linear head with penalty, and no worse validation loss than softplus. A useful falsification is that annealing \\(s\\) fails to improve task loss or produces worse calibration at equal compute.",
 "math_summary": "The paper defines the smooth positive part \\(\\varphi_{+,s}(w)=\\frac{w}{2}+\\sqrt{\\frac{w^{2}}{4}+s}=\\frac{w+\\sqrt{w^2+4s}}{2}\\), for \\(s\u003e0\\), and states that it converges uniformly to \\([w]_+=\\max(w,0)\\) as \\(s\\to0\\). Let \\(x=\\varphi_{+,s}(w)\\). Squaring the defining expression gives \\(x^2-wx-s=0\\), equivalently \\(x-w-s/x=0\\), with \\(x\u003e0\\); this is the perturbed complementarity or central-path equation. It is the stationarity equation of the scalar barrier objective \\(F_s(x;w)=\\frac12(x-w)^2-s\\log x\\) on \\(x\u003e0\\). The derivative used in backpropagation is \\(\\frac{\\partial x}{\\partial w}=\\frac12+\\frac{w}{2\\sqrt{w^2+4s}}\\in(0,1)\\). The smoothing error is uniformly bounded by \\(\\sqrt{s}\\), since \\(0\\le \\varphi_{+,s}(w)-[w]_+\\le\\sqrt{s}\\). In a neural inequality layer, \\(w\\) is the unconstrained network proposal, \\(x\\) is the strictly positive feasible slack, and \\(s\\) is the barrier parameter. The complementarity residual is \\(r_c=x(x-w)-s\\), which should be near zero up to floating-point error.",
 "math_tags": [
  "convex-analysis",
  "optimization",
  "dynamical-systems"
 ],
 "ml_areas": [
  "mlp",
  "optimizer",
  "training",
  "loss"
 ],
 "paper": {
  "arxiv_id": "2608.30470",
  "arxiv_url": "https://arxiv.org/abs/2608.30470",
  "summary_what_math_gives_to_ml": "The paper turns a nonsmooth positive-part contact law into the exact central-path map \\(\\varphi_{+,s}(w)=\\frac{w+\\sqrt{w^2+4s}}{2}\\), obtained by eliminating a strictly positive slack variable under a logarithmic barrier. Its transferable asset is not merely smoothing: the map is the unique positive solution of a perturbed complementarity equation, has an explicit derivative, and supplies a principled continuation parameter rather than an arbitrary activation heuristic. This can be tested as a differentiable inequality layer for neural networks, where outputs or residual slacks must remain nonnegative and the barrier parameter is annealed according to constraint and optimization scales. The idea is moderately novel because softplus-like smooth ReLUs are known, but the explicit primal-dual central-path interpretation and complementarity residual provide useful diagnostics and scheduling rules.",
  "title": "A Barrier-Regularized Symmetric Nitsche Method for the Signorini Problem",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 3,
  "novelty": 4,
  "usefulness": 5
 },
 "solves": [
  "stability",
  "accuracy",
  "generalization"
 ],
 "title": "Central-Path ReLU Inequality Layer",
 "url": "https://synthcore.org/idea/2892/central-path-relu-inequality-layer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}