✗ Failed on benchmark 2026

Wavelet Conditional Sampler for Neural EBMs

Implementation & benchmark of arXiv:2608.31114 — Overcoming critical slowing down in frustrated spin systems by learned multiscale sampling

Usefulness7/10

Difficulty6/10

Novelty6/10

Sampling Speedup Stability Accuracy Diffusion sampling Vae Diffusion Inference speedup Statistical mechanics Renormalization Probability Stochastic processes Harmonic analysis

Source paper: Overcoming critical slowing down in frustrated spin systems by learned multiscale sampling arXiv:2608.31114 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Represent an image or spatial latent state in an orthogonal multiresolution wavelet basis and learn the conditional distribution of detail coefficients at each scale given all coarser coefficients. At inference time, sample coarse coefficients first and recursively sample finer coefficients, using a small conditional network or a few local Langevin steps at each level instead of running a long pixel-space Markov chain. The mechanism should remove critical slowing down caused by long-range correlations, with the number of scale transitions growing as $\log_2 L$ for linear image size $L$.

Formulas

$$p_0(\phi_0)=Z^{-1}e^{-E_0(\phi_0)},\qquad E_0(\phi_0)=\frac{1}{2T}\sum_{i,j}\phi_0(i)K_{ij}\phi_0(j)+\sum_i\left[\phi_0^2(i)+\lambda\left(\phi_0^2(i)-1\right)^2\right].$$

$$w=W\phi,\qquad p(w)=p(c_J)\prod_{j=J}^{1}p\!\left(d_j\mid c_J,d_J,\ldots,d_{j+1}\right).$$

$$q_\theta(w)=q_{\theta_0}(c_J)\prod_{j=J}^{1}q_{\theta_j}\!\left(d_j\mid h_j\right),\qquad h_j=\operatorname{Context}(c_J,d_J,\ldots,d_{j+1}).$$

$$\tau_{\mathrm{total}}\approx\sum_{j=1}^{J}\tau_j,\qquad \tau_j\approx\tau_*=O(1)\ \Longrightarrow\ \tau_{\mathrm{total}}=O(J)=O(\log_2 L).$$

Mathematical statement

The source model uses the dimensionless Gibbs density $p_0(\phi_0)=Z^{-1}\exp[-E_0(\phi_0)]$, with $E_0(\phi_0)=\frac{1}{2T}\sum_{i,j}\phi_0(i)K_{ij}\phi_0(j)+\sum_i[\phi_0^2(i)+\lambda(\phi_0^2(i)-1)^2]$. Here $\phi_0(i)$ is the field at lattice site $i$, $K$ is the interaction kernel, $T$ is the temperature-like parameter, $\lambda$ controls the local quartic potential, and $Z$ normalizes the density. Let $W$ be an orthogonal wavelet transform and write $w=(c_J,d_J,d_{J-1},\ldots,d_1)$, where $c_J$ is the coarsest coefficient block and $d_j$ is the detail block at scale $j$. The exact density factorizes by the chain rule as $p(w)=p(c_J)\prod_{j=J}^{1}p(d_j\mid c_J,d_J,\ldots,d_{j+1})$. WCRG replaces each exact conditional by a learned conditional $q_{\theta_j}(d_j\mid h_j)$, where $h_j$ contains coarser coefficients and local spatial context. If the conditional decorrelation time $\tau_j$ is bounded independently of scale, the total scale complexity is $O(J)=O(\log_2 L)$.

Implementation notes

Integrate this as an alternative inference sampler for a spatial neural energy model or score model. Start with a 2-D convolutional energy model $E_\psi(x)$, and choose an orthogonal Haar or Daubechies wavelet transform with $J=\log_2 L$ levels. During training, transform every training image into $w=(c_J,d_J,\ldots,d_1)$. Train one conditional density model per scale, such as a small U-Net, masked convolution, conditional normalizing flow, or diagonal Gaussian mixture, using the negative log likelihood $-\log q_{\theta_j}(d_j\mid h_j)$. At inference, sample $c_J$ from the coarsest model, then iterate from $j=J$ down to $1$: construct $h_j$, sample $d_j$, and concatenate it with the already sampled coarser coefficients before moving to the next level. Reconstruct the image with $x=W^{-1}w$. If exact conditional learning is difficult, run one to five Langevin or Metropolis updates within each conditional model rather than thousands of pixel-space updates. Pseudocode is: w=sample_coarse(); for j=J,...,1: h=Context(w_coarse); d=ConditionalNet[j].sample(h); w.append(d); x=InverseWavelet(w). The wavelet transform and chain-rule factorization come from the paper; conditional networks, likelihood calibration, and optional MCMC corrections are empirical choices. First test on 32x32, 64x64, and 128x128 Ising-like fields or CIFAR-10 with a convolutional energy model, comparing pixel Langevin, multiscale Langevin, and the learned sampler at equal network evaluations. Measure integrated autocorrelation time for low-frequency magnetization or image mean, Fourier structure factors, and held-out likelihood or FID. The prediction is that $\tau_j$ remains approximately constant across levels and total decorrelation cost grows linearly with $J$, rather than with $L$. A log-linear fit of cost versus $L$ should therefore be compatible with $O(\log_2 L)$. Reducing conditional-network width should increase observable bias and KL divergence while leaving per-level mixing time comparatively unchanged.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: Significantly worse ✗

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 7/10

Built a self-contained 2-D orthogonal Haar hierarchical conditional sampler and numerically verified the proposed mechanism in its synthetic Gaussian model. Reconstruction and Parseval errors were below 5e-16; conditional residual standard deviation stayed at the predicted sigma across L=8–64, scaled linearly with sigma, was insensitive to rho, and the transition count fit transitions=1.0000*log2(L)+0.0000. The ancestral wavelet sampler had integrated autocorrelation near 1 versus 46–116 for the tested pixel-space ULA, but this is only a toy exact conditional model, not evidence for a learned neural EBM.

Agent confidence: 7/10

Baseline: Pixel-space Gaussian ULA mean-observable IAT: L=8 46.10, L=16 115.87, L=32 58.63
Idea: Wavelet ancestral mean-observable IAT: L=8 1.040, L=16 1.003, L=32 1.154; conditional residual std for sigma=0.35 across L=8,16,32,64: 0.3502, 0.3485, 0.3501, 0.3500; log-scale transition fit slope=1.0000, intercept=0.0000; sigma sweep observed stds 0.1500, 0.3497, 0.7003

Limitations: No learned conditional networks, neural energy model, Ising/quartic Gibbs target, finite-step conditional Langevin correction, likelihood, FID, structure-factor, wall-clock, FLOP, or GPU experiment was tested. The baseline uses an estimated Gaussian precision and the wavelet sampler is exact by construction, so the comparison is illustrative rather than a fair end-to-end EBM benchmark.

How to run: python3 wavelet_sampler_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.31114) Sep 2, 2026

Significantly WORSE than the tuned baseline (+9331.4%, p=0.0081, wins 0 / 8 paired seeds).

Benchmark: Multiscale_diffusion_pde
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 0.0006
Idea mean: 0.0589
Effect (Δ): 0.0583 (+9331.4%; negative = idea better)
Wins: 0 / 8 paired seeds
p-value: 0.0081 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Not confirmed ✗
Practical effect: Hurts
Baseline tuning: swept over 3 configs

Limitations:

Only the matched multiscale_diffusion_pde track was tested, with 400 training examples, 100 test examples, 20 epochs, three configurations, and an mlp_tiny backbone. The implementation tests a wavelet-domain training loss rather than a learned conditional sampler at inference; CNN image EBMs, finite-step Langevin correction, likelihood, FID, wall-clock sampling, and larger-resolution scaling were not tested.

How to run: python3 stage2_wavelet_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 2, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented a wavelet-domain multiscale training loss with the same flattened mlp_tiny architecture, data, epochs, batch size, and learning-rate union as the baseline. Across 8 paired seeds, the idea was significantly worse: test MSE 0.0588950552 versus 0.0006244598 for baseline, paired delta +0.0582705954, permutation p=0.0081. The NN-scale mechanism signature was not confirmed because the observed detail-to-pixel residual MSE ratio was 2.29393845, above the predicted threshold of 1.

Baseline: Best config lr=0.01, weight_decay=0.0; 8-seed test MSE mean 0.0006244598, std 0.0001381612.
Idea: Best config lr=0.01, weight_decay=0.0, detail_weight=2.0; 8-seed test MSE mean 0.0588950552, std 0.0075814336.
Mechanism signature: Mean trained-model detail residual MSE divided by pixel residual MSE = 2.29393845; confirmed=false.
Paired delta: delta_mean=+0.0582705954; idea_wins=0/8; permutation p-value=0.0081; verdict=idea worse (significant).

Benchmark result