Accessibility regularizer for power-law experts
Implementation & benchmark of arXiv:2608.17714 — Newton Support Functions and Metric Completion of Singular Conformal Metrics at Corners
Source paper: Newton Support Functions and Metric Completion of Singular Conformal Metrics at Corners arXiv:2608.17714 ⓘ · analyzed Sep 1, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
When a neural field learns power-law exponents, penalize exponent configurations whose Newton support violates the paper's finite-distance accessibility condition. This discourages combinations of exponents that create excessively strong joint singularities while preserving anisotropic scaling when it is supported by the data.
Formulas
Mathematical statement
For exponent pairs \((a_j,b_j)\), the paper's support function is \(\mathcal{H}(p,q)=\max_j(pa_j+qb_j)\). A projective direction \([(p,q)]\), with \(p,q>0\), is accessible exactly when \(\mathcal{H}(p,q)<2\min\{p,q\}\). For nonnegative exponent data, the paper further states that the corner is at finite metric distance exactly when \(\max_j(a_j+b_j)<2\). The latter follows by evaluating the balanced direction \(p=q\) and is convenient for implementation. We use a soft violation penalty rather than a hard constraint: \(R_{\mathrm{acc}}=\operatorname{softplus}(\tau(\max_j(a_j+b_j)-2))/\tau\), where \(\tau\) controls sharpness. To test anisotropic rather than only balanced behavior, sample positive directions \((p_k,q_k)\) and add \(\frac{1}{K}\sum_k\operatorname{softplus}(\tau[\mathcal{H}(p_k,q_k)-2\min(p_k,q_k)])/\tau\). This is a transferable stability prior, not a theorem about arbitrary neural networks: it is justified when the network explicitly represents singular coordinate scalings.
Implementation notes
Apply the regularizer to the exponent parameters of the Newton-envelope layer, not to ordinary unconstrained MLP weights. Parameterize a_j=softplus(alpha_j) and b_j=softplus(beta_j) so the intended nonnegative-exponent assumption holds. During each training step, draw K fixed or freshly sampled positive directions p_k,q_k from log-uniform values in [0.25,4], optionally normalize each pair by p_k+q_k because the condition is homogeneous. Compute h_k=max_j(p_k*a_j+q_k*b_j), v_k=h_k-2*min(p_k,q_k), and add lambda_acc/K times softplus(tau*v_k)/tau to the task loss. Include the balanced shortcut v_bal=max_j(a_j+b_j)-2 for a cheap always-on penalty. Start with lambda_acc in {1e-4,1e-3,1e-2} and tau=10; anneal lambda from zero over the first 10% of training so the model first discovers useful exponents. The exact support maximum and accessibility threshold come from the paper; K, direction sampling, lambda, and softplus smoothing are engineering choices. First test on a synthetic near-corner regression task with train points at moderate radii and test points closer to x=y=0, comparing no regularization, balanced-only regularization, and directional regularization. Track extrapolation error, maximum predicted magnitude, gradient norm, and learned exponents. Success means improved near-corner extrapolation and fewer exploding gradients at equal in-domain error; reject the method if it simply biases the model toward underfitting or materially worsens ordinary-domain accuracy.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.