✓✓ Beats tuned baseline 2026

Resolution-normalized Hilbert dictionary

Implementation & benchmark of arXiv:2608.20812 — Resolution-Consistent Greedy Neural Approximation on Infinite-Dimensional Spaces

Usefulness8/10

Difficulty4/10

Novelty7/10

Architecture Generalization Accuracy Sample efficiency Mlp Embedding Training Functional analysis Approximation theory Probability

Source paper: Resolution-Consistent Greedy Neural Approximation on Infinite-Dimensional Spaces arXiv:2608.20812 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Build a shallow neural model whose input at every discretization level is embedded into a common Hilbert space with uniformly bounded norm, and constrain every neuron parameter in the corresponding dual norm. The statistical complexity then depends on the Hilbert norm bound rather than the number of retained coordinates, allowing one model design to operate across increasingly fine measurements.

Formulas

$$\text{population error}=\text{resolution error}+\text{finite-width error}+\text{statistical error}.$$

$$\phi_{w,b}^{(m)}(x)=\sigma\!\left(\langle w,P_mx\rangle+b\right),\qquad \|w\|_2\le1,\quad |b|\le1,\quad \sup_m\|P_mx\|_2\le B.$$

$$\mathcal F_R=\left\{f(x)=\int \phi_{w,b}^{(m)}(x)\,d\mu(w,b):\|\mu\|_{TV}\le R\right\},\qquad \mathfrak R_n(\mathcal F_R)\lesssim \frac{RB}{\sqrt n}.$$

$$\mathcal E(f_K)\lesssim \|f-f\circ P_m\|_{L_2(P)}^2+\frac{CR^2}{K}+\frac{RB}{\sqrt n},\qquad f_K(x)=\sum_{j=1}^K a_j\phi_{w_j,b_j}^{(m)}(x),\quad \sum_{j=1}^K|a_j|\le R.$$

Mathematical statement

Let $x$ be an infinite-dimensional input in a Hilbert space $\mathcal H$, and let $P_mx\in\mathbb R^m$ retain the first $m$ coordinates. Assume $\sup_m\|P_mx\|_2\le B$ for all inputs in the relevant support. Define normalized dictionary atoms $\phi_{w,b}^{(m)}(x)=\sigma(\langle w,P_mx\rangle+b)$ with $\|w\|_2\le1$ and $|b|\le1$, where $\sigma$ is 1-Lipschitz. The weighted variation class is $\mathcal F_R=\{f(x)=\int\phi_{w,b}^{(m)}(x)\,d\mu(w,b):\|\mu\|_{TV}\le R\}$, where $\mu$ is a signed parameter measure and $\|\mu\|_{TV}$ is its total variation. Because the feature norm is uniformly bounded, empirical Rademacher complexity scales as $\mathfrak R_n(\mathcal F_R)\lesssim RB/\sqrt n$ without explicit $m$ dependence. For a finite model with $K$ atoms, the useful decomposition is resolution truncation plus finite-width approximation plus statistical complexity.

Implementation notes

Integrate this at the input encoder of a shallow MLP or neural operator. Choose an orthonormal basis or quadrature-weighted grid basis for the input function, and convert measurements at resolution $m$ into coefficients $z_m=P_mx\in\mathbb R^m$. Ensure the basis normalization makes $\|z_m\|_2$ comparable across resolutions; estimate $B$ from a high training-set quantile of these norms. Replace an unconstrained first layer by $K$ atoms $h_j=\sigma(w_j^Tz_m+b_j)$. After every optimizer step, project each $w_j$ onto the unit Euclidean ball and clip $b_j$ to $[-1,1]$. Use output coefficients $a_j$ with either an explicit $\ell_1$ penalty or projection onto $\{a:\|a\|_1\le R\}$. The paper-derived quantities are the uniform norm constraints and the three-term error decomposition; estimate the truncation term by evaluating the same target or prediction at resolutions $m$ and $2m$. First test on Burgers or Darcy operator regression using Fourier or grid coefficients at $m\in\{16,32,64,128\}$. Compare against an unconstrained MLP at matched width and training FLOPs. Plot test error, train-test gap, activation scale, and weight norms versus $m$. Success is nearly resolution-flat generalization and transfer from a model trained at $m=64$ to inputs represented at $m=128$, without retuning learning rates or width.

Verification

Beats tuned baseline

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Significant win ✓

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a projected Hilbert-dictionary shallow model with unit-norm hidden weights, clipped biases, and an output l1-ball projection, plus numerical complexity and regression checks. The mechanism manifested: normalized Rademacher complexity had log-resolution slope -0.061 versus the predicted 0, raw unnormalized coordinates had slope 0.501 versus 0.5, preactivation scaling had slope 1.000 versus 1, and variation-radius complexity had slope 0.984 versus 1. The constrained model generally reduced test error relative to the unconstrained baseline at m=8,32,64,128, but this was a toy regression rather than a definitive operator-learning win.

Agent confidence: 8/10

Baseline: Unconstrained dictionary test MSE by m=[8,16,32,64,128]: [0.00589, 0.02060, 0.04606, 0.02902, 0.04104]; hidden-weight maxima [1.007,1.014,1.008,1.009,1.012].
Idea: Constrained dictionary test MSE by m=[8,16,32,64,128]: [0.00503,0.02330,0.02728,0.02461,0.03777]; hidden-weight maximum exactly 1.0 and output l1 norm stayed below R=3. Complexity slopes: resolution -0.061 normalized, 0.501 raw, scale 1.000, radius 0.984.

Limitations: The identity activation is used for the direct Rademacher calculation as a valid 1-Lipschitz special case; the regression uses synthetic decaying Gaussian coefficients, not Burgers or Darcy data. No multi-seed confidence intervals, true m=64-to-m=128 transfer test, FLOP-matched wall-clock benchmark, or population resolution-truncation estimate was performed.

How to run: OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 /home/maxwelhelp/main/bin/python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.20812) Sep 2, 2026

Beats the tuned baseline by 4.3% (p=0.0081, wins 8 / 8 paired seeds; metric: lower is better).

Benchmark: Resolution_hilbert_regression
Model: local_dictionary
Paired seeds: 8
Baseline mean: 0.0728
Idea mean: 0.0697
Effect (Δ): -0.0032 (−4.3%; negative = idea better)
Wins: 8 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±1.0%
Mechanism: Confirmed ✓
Practical effect: Helps
Baseline tuning: swept over 3 configs

Limitations:

The registered custom task is a fast synthetic function-measurement regression benchmark rather than Burgers, Darcy, or a real measured operator dataset. Distinct-resolution transfer, FLOP/wall-clock matching, larger widths, and alternative basis normalizations were not tested.

How to run: python3 bench_experiment.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Worked ✓

Agent confidence: 8/10

The registered custom track resolution_hilbert_regression was rerun through bench.get_dataset and bench.make_report with eight paired seeds and a parity-matched three-value learning-rate sweep. The constrained Hilbert dictionary achieved lower test MSE than the tuned unconstrained baseline in all eight pairs: 0.06969587 versus 0.07284840, delta=-0.00315253, permutation p=0.0081. The trained-model mechanism signature confirmed the projected hidden-weight bound and output l1 constraint.

Baseline: Best lr=0.003; mean test MSE 0.0728484001; sweep means [0.0833553793, 0.0728484001, 0.0748559255].
Idea: Best lr=0.003; mean test MSE 0.0696958704; per-seed results [0.0712435767, 0.0731648430, 0.0639909878, 0.0699851513, 0.0695066899, 0.0629635379, 0.0717324987, 0.0749796778].

Benchmark result