Training ideas

Curricula, sampling schedules, pseudo-labeling and multi-task mixing, with the standard training recipe as the control.

✗ Failed on benchmark 2026

Localized Petrov–Galerkin Neural Residuals

Replace the pointwise strong-form PINN loss with a vector of localized weak residuals generated by fixed compactly supported polynomial test functions. Use a neural network or KAN as the trial function, integrate by parts once, and evaluate each test residual with Gauss–Legendre quadrature; this lowers the required derivative order and prevents a few high-curvature collocation points from dominating training.

Useful8/10
Difficulty5/10
Novelty5/10
Paper: PG-KINN: A Physics-Informed Petrov-Galerkin Kolmogorov-Arnold Network for Solving Forward and Inverse PDEs arXiv:2607.20378
✗ Failed on benchmark 2026

Wittrick–Williams Mode Enumerator

Use a spectral eigenvalue-counting function to bracket each target mode before neural optimization. The network then solves only within an interval containing exactly one eigenfrequency, preventing optimization from repeatedly collapsing to the lowest mode or jumping between modes.

Useful7/10
Difficulty5/10
Novelty8/10
Paper: A Framework Integrating the Dynamic Stiffness Matrix with Physics-Informed Neural Networks for Solving Eigenvalue Problems and Analysing Dynamic Response arXiv:2608.28683
△ Mechanism confirmed, baseline not beaten 2026

Adjoint-Weak Fractional Residuals

Replace pointwise fractional derivatives of noisy trajectories in a neural PDE or neural dynamics loss with weak projections in which the fractional operator acts on smooth test functions. The network is trained to match integral residuals over local space-time windows, making the residual insensitive to high-frequency measurement noise while retaining sensitivity to the underlying fractional dynamics.

Useful7/10
Difficulty4/10
Novelty5/10
Paper: Robust data-driven discovery of fractional differential equations via weak formulations and Pareto-based subset selection arXiv:2608.12879
✗ Failed on benchmark 2026

Patch-Consensus Weak Residual Training

Train a neural PDE surrogate using weak residuals on randomly sampled local patches rather than pointwise derivative residuals. On every patch, identify which candidate differential-operator terms are consistently supported, then aggregate supports across many patches to obtain spatial equation regions and use the resulting consensus as a robust routing or auxiliary supervision signal.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Identifying changing partial differential equations using Sampled Local WeakIdent arXiv:2608.12479
△ Mechanism confirmed, baseline not beaten 2026

Walk-on-Spheres stochastic target layer

Train a neural network to represent an elliptic solution using Walk-on-Spheres rollouts as stochastic targets instead of evaluating a mesh-based PDE residual. For each input point, recursively jump to a random point on the largest interior sphere, accumulate source contributions, evaluate boundary data at termination, and regress the network output to the resulting Monte Carlo estimate.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: Walk-on-Spheres Monte Carlo and deep neural network approximations of elliptic PDEs with drift and killing arXiv:2608.09494
△ Mechanism confirmed, baseline not beaten 2026

Quasi-uniform residual least squares

Train a frozen-feature or linearized neural network by residual least squares on deterministic quasi-uniform points rather than independently sampled collocation points. The paper's norm-equivalence result predicts that, once the number of residual points is proportional to the number of active features, the empirical residual controls the continuous residual and avoids random undersampling of localized errors.

Useful7/10
Difficulty4/10
Novelty6/10
Paper: Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples arXiv:2608.06687
✗ Failed on benchmark 2026

Sensitivity-Particle Training for Marginal-Only Latent ODEs

Train an augmented latent neural ODE from snapshot observations of only the visible coordinates by transporting particles from an initial latent distribution and differentiating their visible locations through forward sensitivity equations. Replace density-PDE discretization or potentially biased same-particle density objectives with a kernel marginal-matching loss whose gradient is estimated using independent particle sets.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Characteristic Sensitivity Ensembles for Inference of Hidden Dynamics from Marginal Observations arXiv:2608.06190
✗ Failed on benchmark 2026

Pre-Training Depth Feasibility Certificates

Use computable upper and lower error bounds to reject quantized-depth configurations that cannot reach the desired accuracy before training. The planner separates irreducible library mismatch from finite-depth synthesis, codebook metadata, and execution errors, then selects the smallest depth and metadata budget whose estimated bound passes the target.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: When Can Depth Replace Precision? A Resource Theory of Quantized Neural Computation arXiv:2607.23390
△ Mechanism confirmed, baseline not beaten 2026

Max-Volume Transport Skeleton

Use a maximal-volume cross approximation of the parameter-by-space transport-signature matrix to select informative training conditions and compact spatial features. This provides an active-learning alternative to random snapshot selection or ordinary PCA, targeting parameters that are difficult to interpolate from the current reduced representation.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Structure-Preserving Reduced-Order Modeling via Low-Rank Transport Signatures arXiv:2607.01696
✗ Mechanism failed 2026

Jin–Xin shock homotopy for PINNs

Train on a sequence of Jin–Xin relaxation problems with decreasing relaxation width rather than training immediately on the singular conservation law. The network predicts both the conserved state and an auxiliary flux, and each stage is initialized from the previous stage so that the learned shock profile sharpens gradually.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: A Jin--Xin Relaxation Gradual Convergence Method for Conservation-Law PINNs arXiv:2608.22493
Unverified 2026

Decision-Dependent Robustness Cutting Planes

Replace ordinary adversarial training over a fixed perturbation set with adaptive robust training in which the admissible perturbations depend on the current network state. Train on a small active set of hard scenarios, then search for a newly admissible scenario with larger loss or constraint violation and add it only when needed. This should reduce redundant adversarial examples while targeting worst-case regions induced by the current model.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Generalized Semi-Infinite Programming for Robust Optimal Control with Decision-Dependent Uncertainty arXiv:2609.01538
Unverified 2026

Polynomial-Mixing Block Training

Use the paper's separated-block construction to train recurrent or state-space networks on trajectories with slowly decaying temporal correlations, rather than treating consecutive frames as independent minibatch samples. Thresholded events such as collision, failure, saturation, constraint violation, or reward exceedance are aggregated over blocks with empirically chosen gaps and optionally replaced by finite-resolution cylinder approximations. The method predicts a measurable power-law…

Useful6/10
Difficulty5/10
Novelty7/10
Paper: Statistical properties for irregular observables in slowly mixing hyperbolic systems arXiv:2608.15569
Unverified 2026

Scaled-boundary quadrature loss

Replace random interior collocation points in domain-integrated neural losses with deterministic points generated by scaling oriented boundary patches toward an interior center. The resulting quadrature exactly integrates low-degree polynomial integrands on affine polytopes and gives a mesh-free, geometry-aware loss for PINNs and neural fields.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Scaled boundary cubature scheme in higher dimensions: integration over polytopes and curved regions arXiv:2608.15563
Unverified 2026

Error-budgeted adaptive rollout allocation

Use the paper's explicit separation of geometric, truncation, and Monte Carlo errors to adapt the number of Walk-on-Spheres paths and rollout depth per training point. Points close to the boundary receive short, low-cost rollouts, while points with high estimated target variance receive additional paths only when the resulting accuracy justifies their cost.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Walk-on-Spheres Monte Carlo and deep neural network approximations of elliptic PDEs with drift and killing arXiv:2608.09494
Unverified 2026

Stiffness-energy supervision without FEM labels

Train a finite-element surrogate by minimizing the assembled discrete potential energy rather than a loss against solved displacement labels. The objective uses only the sparse stiffness matrix and load vector, while its exact energy-gap identity makes it equivalent to supervised regression in the stiffness norm.

Useful6/10
Difficulty3/10
Novelty5/10
Paper: Discrete energy as an exact label-free training objective for finite-element surrogates arXiv:2608.05437
Unverified 2026

Physical-Adjoint Dual-Weighted PINN

Train a primal neural PDE solver and a separate physical-adjoint neural solver, then use their first-order-system residuals to adaptively allocate collocation points toward regions that control a chosen quantity of interest. Instead of minimizing only the primal residual uniformly, prioritize points according to a balanced combination of primal and adjoint local residuals, because the target-output error is controlled by their global product.

Useful6/10
Difficulty5/10
Novelty6/10
Paper: Goal-Oriented Error Estimation for Least-Squares Finite Element Methods via Physically Meaningful Adjoint PDEs arXiv:2607.23850
△ Mechanism confirmed, baseline not beaten 2026

Resolution-aware operator data budget

Couple the number of operator training pairs to the output resolution instead of increasing the output grid independently. Refine the output discretization only while the oracle reconstruction improves, and increase the training set when the learned predictor remains substantially worse than the oracle decoder.

Useful6/10
Difficulty4/10
Novelty7/10
Paper: Kernel-based Operator Learning: Error Analysis, Budget Allocation, and a Physics-Informed Extension arXiv:2607.06287
Unverified 2026

Gap-Graded Neural Collocation

Replace uniform PINN or neural-operator collocation by a graded point distribution concentrated in narrow regions between nearly touching interfaces. Use the paper's distance-dependent mesh scale to determine point spacing, and switch to a gap-dependent minimum scale when the separation becomes too small for the global mesh.

Useful5/10
Difficulty4/10
Novelty6/10
Paper: High-order finite element method for perfect conductivity and linear elasticity with nearly touching inclusions arXiv:2607.22128