# Hyperuniform Collocation and Minibatch Sampling

- ID: 2405
- Canonical URL: https://synthcore.org/idea/2405/hyperuniform-collocation-and-minibatch-sampling
- API JSON: https://synthcore.org/api/idea/2405.json
- API Markdown: https://synthcore.org/api/idea/2405.md
- Verification status: unverified
- Source: [arXiv:2608.16547](https://arxiv.org/abs/2608.16547)
- Category: training
- Solves: stability, sample-efficiency, accuracy
- ML areas: mlp, optimizer, data-augmentation, training-dynamics
- Math tags: statistics, harmonic-analysis, stochastic-processes, spectral-theory
- Ratings: usefulness 5/10; difficulty 5/10; novelty 6/10

## Idea description

Use spatially correlated training points whose low-frequency structure factor vanishes instead of iid points. For neural fields, PINNs, image-coordinate MLPs, or spatially indexed minibatches, this should suppress long-wavelength quadrature and gradient-estimation noise while preserving the represented target dynamics. The finite-order prediction is that a design with structure factor S(k)=O(|k|^{2q}) produces lower variance for smooth losses than iid sampling, especially as the domain or batch support grows.

## Mathematical statement

For a stationary point process eta on R^d with intensity rho, let N_R be the number of points in the ball B_R. Hyperuniformity means Var(N_R)=o(R^d), and the paper asserts the stronger low-frequency Bartlett-spectrum condition sigma_eta(B_epsilon)=o(epsilon^{2q}) for any prescribed finite q in dimensions d>=2. In an implementation, estimate the structure factor S(k) from sampled locations, where S(k)=Var(sum_j exp(i k dot x_j))/(rho times |D|) for a finite window D. For a smooth spatial loss integrand f with Fourier transform fhat(k), the sampling-error variance is approximately proportional to the integral of |fhat(k)|^2 S(k) over k. Therefore S(k) approximately C|k|^{2q} near k=0 suppresses dominant long-wavelength error relative to iid sampling, whose S(k) is approximately one. Surface-order designs additionally predict Var(N_R)=O(R^(d-1)) rather than Poisson scaling O(R^d).

## Key formulas

- $$\operatorname{Var}_{\eta}(N_R)=o(R^d),\qquad N_R=\#(P\cap B_R).$$
- $$\sigma_{\eta}(B_{\varepsilon})=o(\varepsilon^{2q})\qquad(\varepsilon\downarrow0).$$
- $$S(k)=\frac{1}{\rho |D|}\operatorname{Var}\!\left[\sum_{x_j\in\eta\cap D}e^{i k\cdot x_j}\right],\qquad S(k)=O(|k|^{2q})\ \text{as }k\to0.$$$
- $$\operatorname{Var}\!\left[\frac{1}{n}\sum_{j=1}^{n}f(x_j)\right]\approx \frac{1}{\rho^2}\int |\widehat f(k)|^2S(k)\,dk.$$$

## Implementation notes

1. Integration point: apply this to a coordinate MLP or PINN whose objective is an integral over x in a bounded d-dimensional domain. At every optimizer step, draw collocation points from iid uniform sampling, a randomly shifted lattice, and a hyperuniform approximation. The simplest hyperuniform approximation is a randomly shifted lattice with bounded correlated displacements; a stronger version uses particle relaxation that minimizes low-frequency Fourier power. 2. Pseudocode: initialize points x_j; for T relaxation iterations compute L_HU=sum over nonzero Fourier modes with |k|<k_c of w_k times the squared magnitude of sum_j exp(i k dot x_j); update x_j <- x_j-alpha times gradient_xj L_HU; enforce periodic boundaries or reflection; then evaluate the neural loss and gradient on x_j. Use a fresh random global shift each epoch to avoid fixed-grid bias. 3. Compute S(k) empirically on a Fourier grid, fit its log-log slope near zero, and measure covariance of minibatch gradients. The paper supplies the low-frequency suppression and hyperuniform count-variance targets; the finite-domain sampler and Fourier estimates are approximations. 4. First cheap experiment: train a four-layer width-128 coordinate MLP on a smooth 2D function with 256 points per step, comparing iid, Sobol, shifted-grid, and relaxed hyperuniform points. Measure test error, gradient variance, and Var(N_R) over many windows. The prediction is a low-k slope near at least 2 for q=1, count variance closer to R^(d-1) than R^d, and lower smooth-function gradient variance at equal batch size.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.