# Inverse-inequality resolution control

- ID: 1906
- Canonical URL: https://synthcore.org/idea/1906/inverse-inequality-resolution-control
- API JSON: https://synthcore.org/api/idea/1906.json
- API Markdown: https://synthcore.org/api/idea/1906.md
- Verification status: unverified
- Source: [arXiv:2608.06687](https://arxiv.org/abs/2608.06687)
- Category: stability
- Solves: stability, accuracy, speedup
- ML areas: mlp, training, loss, initialization
- Math tags: approximation-theory, functional-analysis, geometry, pde, numerical-analysis
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Use the network-space inverse inequality to choose derivative order, collocation resolution, and feature separation jointly instead of enforcing arbitrarily high-order residuals on an under-resolved network. This creates an anti-aliasing rule: a network whose parameters are separated by \(\underline h\) cannot represent high Sobolev frequencies without a factor \(\underline h^{-(r-s)}\), so derivative penalties above the resolvable order should be disabled or accompanied by refinement.

## Mathematical statement

For \(v_n\in L_n^k(\Theta_n)\), where \(L_n^k(\Theta_n)\) is the linearized ReLU-\(k\) network space, and \(\underline h\) is the antipodal separation distance of \(\Theta_n\), the paper proves \(\|v_n\|_{\mathcal{H}^r(\mathbb{S}^d)}\lesssim\underline h^{-(r-s)}\|v_n\|_{\mathcal{H}^s(\mathbb{S}^d)}\) for \(0\le s<r<k+1/2\). Here \(\mathcal{H}^s\) is the Sobolev norm of order \(s\), \(k\) is the ReLU smoothness/order parameter, and the hidden constant is independent of \(\underline h\). Since quasi-uniform sets with \(n\) points typically satisfy \(\underline h\asymp n^{-1/d}\), the bound becomes \(\|v_n\|_{H^r}\lesssim n^{(r-s)/d}\|v_n\|_{H^s}\). In a neural PDE solver, interpret \(s\) as the derivative order reliably measured by the collocation mesh and \(r\) as the derivative order appearing in a loss or stability estimate. The growth factor flags when high-order derivative losses will amplify coefficient noise and conditioning.

## Key formulas

- $$\|v_{n}\|_{\mathcal{H}^{r}(\mathbb{S}^{d})}\lesssim\underline{h}^{-(r-s)}\|v_{n}\|_{\mathcal{H}^{s}(\mathbb{S}^{d})},\qquad 0\le s<r<k+\tfrac{1}{2}.$$
- $$\underline h\asymp n^{-1/d}\quad\Longrightarrow\quad\|v_n\|_{\mathcal{H}^{r}}\lesssim n^{(r-s)/d}\|v_n\|_{\mathcal{H}^{s}}.$$
- $$\lambda_r=\min\left(1,\frac{\tau}{\underline h^{-(r-s)}E_s}\right),\qquad L=L_s+\lambda_rL_r.$$

## Implementation notes

1. Integration point: modify the derivative-residual loss and collocation scheduler in a PINN, neural operator, or coordinate MLP. The relevant quantities are feature-parameter separation \(\underline h\), derivative order \(r\) used by the PDE residual, and a lower-order diagnostic norm of order \(s\). 2. Pseudocode: initialize feature parameters \(\Theta\); compute \(\underline h=\min_{i\ne j}\|\theta_i-\theta_j\|\) after identifying antipodal duplicates; choose the largest candidate derivative order \(r<k+1/2\) satisfying \(\underline h^{-(r-s)}\le C_{\max}\). At each training phase, estimate lower-order residual norm \(E_s\) on a validation point set, compute the bound proxy \(B_r=\underline h^{-(r-s)}E_s\), and either use derivative loss \(L_r\) only when \(B_r\) is below a threshold or refine by adding well-separated features and collocation points. A practical loss is \(L=L_s+\lambda_rL_r\) with \(\lambda_r=\min(1,\tau/B_r)\). 3. The theorem supplies the exponent \(r-s\) and admissible range \(r<k+1/2\); the constant, actual Sobolev norms, and domain transfer must be estimated empirically. For nonspherical inputs, replace \(\underline h\) by minimum hidden-weight separation or a measured feature-kernel length scale. 4. First experiment: train width-128 and width-256 coordinate ReLU MLPs on a 2D Poisson or biharmonic problem with first-, second-, and fourth-order residual penalties. Compare fixed derivative losses against the adaptive rule at equal FLOPs. Track gradient-norm variance, residual-Jacobian condition number, dense-grid PDE residual, and final solution error. Success is that the adaptive rule prevents exploding derivative gradients and achieves equal or lower dense-grid error without increasing the collocation budget. A falsifying result is no reduction in instability or no correlation between predicted factor \(\underline h^{-(r-s)}\) and measured derivative amplification.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.