# Resolution-normalized Hilbert dictionary

- ID: 2634
- Canonical URL: https://synthcore.org/idea/2634/resolution-normalized-hilbert-dictionary
- API JSON: https://synthcore.org/api/idea/2634.json
- API Markdown: https://synthcore.org/api/idea/2634.md
- Verification status: beats_baseline
- Source: [arXiv:2608.20812](https://arxiv.org/abs/2608.20812)
- Category: architecture
- Solves: generalization, accuracy, sample-efficiency
- ML areas: mlp, embedding, training
- Math tags: functional-analysis, approximation-theory, probability
- Ratings: usefulness 8/10; difficulty 4/10; novelty 7/10

## Idea description

Build a shallow neural model whose input at every discretization level is embedded into a common Hilbert space with uniformly bounded norm, and constrain every neuron parameter in the corresponding dual norm. The statistical complexity then depends on the Hilbert norm bound rather than the number of retained coordinates, allowing one model design to operate across increasingly fine measurements.

## Mathematical statement

Let $x$ be an infinite-dimensional input in a Hilbert space $\mathcal H$, and let $P_mx\in\mathbb R^m$ retain the first $m$ coordinates. Assume $\sup_m\|P_mx\|_2\le B$ for all inputs in the relevant support. Define normalized dictionary atoms $\phi_{w,b}^{(m)}(x)=\sigma(\langle w,P_mx\rangle+b)$ with $\|w\|_2\le1$ and $|b|\le1$, where $\sigma$ is 1-Lipschitz. The weighted variation class is $\mathcal F_R=\{f(x)=\int\phi_{w,b}^{(m)}(x)\,d\mu(w,b):\|\mu\|_{TV}\le R\}$, where $\mu$ is a signed parameter measure and $\|\mu\|_{TV}$ is its total variation. Because the feature norm is uniformly bounded, empirical Rademacher complexity scales as $\mathfrak R_n(\mathcal F_R)\lesssim RB/\sqrt n$ without explicit $m$ dependence. For a finite model with $K$ atoms, the useful decomposition is resolution truncation plus finite-width approximation plus statistical complexity.

## Key formulas

- $$\text{population error}=\text{resolution error}+\text{finite-width error}+\text{statistical error}.$$
- $$\phi_{w,b}^{(m)}(x)=\sigma\!\left(\langle w,P_mx\rangle+b\right),\qquad \|w\|_2\le1,\quad |b|\le1,\quad \sup_m\|P_mx\|_2\le B.$$
- $$\mathcal F_R=\left\{f(x)=\int \phi_{w,b}^{(m)}(x)\,d\mu(w,b):\|\mu\|_{TV}\le R\right\},\qquad \mathfrak R_n(\mathcal F_R)\lesssim \frac{RB}{\sqrt n}.$$
- $$\mathcal E(f_K)\lesssim \|f-f\circ P_m\|_{L_2(P)}^2+\frac{CR^2}{K}+\frac{RB}{\sqrt n},\qquad f_K(x)=\sum_{j=1}^K a_j\phi_{w_j,b_j}^{(m)}(x),\quad \sum_{j=1}^K|a_j|\le R.$$

## Implementation notes

Integrate this at the input encoder of a shallow MLP or neural operator. Choose an orthonormal basis or quadrature-weighted grid basis for the input function, and convert measurements at resolution $m$ into coefficients $z_m=P_mx\in\mathbb R^m$. Ensure the basis normalization makes $\|z_m\|_2$ comparable across resolutions; estimate $B$ from a high training-set quantile of these norms. Replace an unconstrained first layer by $K$ atoms $h_j=\sigma(w_j^Tz_m+b_j)$. After every optimizer step, project each $w_j$ onto the unit Euclidean ball and clip $b_j$ to $[-1,1]$. Use output coefficients $a_j$ with either an explicit $\ell_1$ penalty or projection onto $\{a:\|a\|_1\le R\}$. The paper-derived quantities are the uniform norm constraints and the three-term error decomposition; estimate the truncation term by evaluating the same target or prediction at resolutions $m$ and $2m$. First test on Burgers or Darcy operator regression using Fourier or grid coefficients at $m\in\{16,32,64,128\}$. Compare against an unconstrained MLP at matched width and training FLOPs. Plot test error, train-test gap, activation scale, and weight norms versus $m$. Success is nearly resolution-flat generalization and transfer from a model trained at $m=64$ to inputs represented at $m=128$, without retuning learning rates or width.

## Verification

- Status: beats_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: helps
- Verdict: Built a projected Hilbert-dictionary shallow model with unit-norm hidden weights, clipped biases, and an output l1-ball projection, plus numerical complexity and regression checks. The mechanism manifested: normalized Rademacher complexity had log-resolution slope -0.061 versus the predicted 0, raw unnormalized coordinates had slope 0.501 versus 0.5, preactivation scaling had slope 1.000 versus 1, and variation-radius complexity had slope 0.984 versus 1. The constrained model generally reduced test error relative to the unconstrained baseline at m=8,32,64,128, but this was a toy regression rather than a definitive operator-learning win.

### Mechanism check

- Verdict: Built a projected Hilbert-dictionary shallow model with unit-norm hidden weights, clipped biases, and an output l1-ball projection, plus numerical complexity and regression checks. The mechanism manifested: normalized Rademacher complexity had log-resolution slope -0.061 versus the predicted 0, raw unnormalized coordinates had slope 0.501 versus 0.5, preactivation scaling had slope 1.000 versus 1, and variation-radius complexity had slope 0.984 versus 1. The constrained model generally reduced test error relative to the unconstrained baseline at m=8,32,64,128, but this was a toy regression rather than a definitive operator-learning win.
- Confidence: 8/10
- Limitations: The identity activation is used for the direct Rademacher calculation as a valid 1-Lipschitz special case; the regression uses synthetic decaying Gaussian coefficients, not Burgers or Darcy data. No multi-seed confidence intervals, true m=64-to-m=128 transfer test, FLOP-matched wall-clock benchmark, or population resolution-truncation estimate was performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0081
- Paired wins: 8/8
- Benchmark verdict: idea better (significant)

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/985/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/985/bench_report.json)
- [experiment.py](https://synthcore.org/code/985/experiment.py)
- [report.md](https://synthcore.org/code/985/report.md)
- [report_bench_2026-09-01T173720.md](https://synthcore.org/code/985/report_bench_2026-09-01T173720.md)
- [resolution_track.py](https://synthcore.org/code/985/resolution_track.py)
- [results.json](https://synthcore.org/code/985/results.json)
- [Download all files as ZIP](https://synthcore.org/download/985)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.