Pointwise Clarke-Tangent Training
Implementation & benchmark of arXiv:2607.03195 — The Clarke tangent and normal cones to decomposable sets in Lebesgue spaces
Source paper: The Clarke tangent and normal cones to decomposable sets in Lebesgue spaces arXiv:2607.03195 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Train a neural function under hard pointwise constraints by projecting its desired output-space update into the Clarke tangent cone of the admissible set at every sampled input. Fit the resulting feasible measurable direction with a parameter update instead of repeatedly allowing the network to violate constraints and repairing it with a penalty.
Formulas
Mathematical statement
The paper defines the decomposable selection set $\mathsf{Sel}_{p}(S)=\{x\in L^{p}(T,Z):x(t)\in S(t)\ \text{a.e.}\}$, where $(T,\Sigma,\mu)$ is a complete $\sigma$-finite measure space, $Z$ is a separable Banach space, $S:T\rightrightarrows Z$ is measurable with nonempty closed values, and $p\in[1,\infty)$. Its central result is $\widehat{\mathbf T}_{\mathsf{Sel}_{p}(S)}(x)=\{v\in L^{p}(T,Z):v(t)\in\widehat{\mathbf T}_{S(t)}(x(t))\ \text{a.e.}\}$. Here $x$ is the current feasible function, $v$ is a global function-space direction, $\widehat{\mathbf T}_{C}(u)$ is the regular/Clarke tangent cone to a closed set $C$ at $u$, and $\widehat{\mathbf T}_{S(t)}(x(t))$ is the pointwise cone. A usable distance characterization is $v\in\widehat{\mathbf T}_{C}(u)$ when $\lim_{h\downarrow0}\operatorname{dist}(u+h v,C)/h=0$. Thus, for finite $p$, local pointwise tangent membership is exactly the first-order feasibility condition for the entire selection-valued neural function. In a minibatch, replace the almost-everywhere condition by tangent membership at every sampled point and approximate the $L^p$ norm by a batch average.
Implementation notes
(1) Integration point: apply the method at the output of a neural operator, dynamics model, or constrained prediction head. For each minibatch input $t_i$, compute $y_i=f_\theta(t_i)$ and define a closed admissible set $S(t_i)$. Start with boxes, simplices, nonnegative physical quantities, or norm-bounded controls. Compute the ordinary output-space descent direction $g_i=-\nabla_{y_i}\ell_i$ while holding the network parameters fixed. (2) Pseudocode: y=f(theta,t); loss=task_loss(y,target); g=-grad_y(loss); for i in batch: C=S(t[i]); v[i]=project_to_tangent_cone(g[i], C, y[i]); J[i]=jvp(f,theta,t[i]); delta_theta=solve_ridge(J_stack,v_stack,lambda); theta += eta*delta_theta. For a box $C=[a,b]^d$, use $v_j=\min(g_j,0)$ at $y_j=a_j$, $v_j=\max(g_j,0)$ at $y_j=b_j$, and $v_j=g_j$ in the interior. For a simplex, impose $\mathbf 1^Tv=0$ and require $v_j\ge 0$ on active zero coordinates, solving a small quadratic program. For a general set, approximate tangent projection by minimizing $\|v-g\|^2+\rho\,\operatorname{dist}(y+\varepsilon v,C)^2$ for small decreasing $\varepsilon$. (3) The theorem supplies the pointwise-to-global tangent principle; Jacobian-vector products, cone projections, ridge regularization, and step-size selection are empirical approximations. Use JVPs or implicit least-squares iterations rather than materializing the Jacobian, and line-search or clip $\eta$ because tangent feasibility is only first order. (4) First experiment: train a small MLP on synthetic regression with simplex-valued or box-constrained outputs, comparing Adam with a violation penalty, ordinary projected outputs, and tangent training at equal FLOPs. Record task loss, fraction of violated coordinates, maximum violation, and loss decrease per optimizer step. Success means lower violation at equal loss or faster loss descent without penalty-induced oscillations.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.