△ Mechanism confirmed, baseline not beaten 2026

PSD Spectral CNN Block

Implementation & benchmark of arXiv:2608.23073 — Addendum to "Factoring non-negative operator valued trigonometric polynomials in two variables"

Usefulness6/10

Difficulty4/10

Novelty5/10

Architecture Stability Accuracy Generalization Cnn Training Ssm Harmonic analysis Linear algebra Functional analysis

Source paper: Addendum to "Factoring non-negative operator valued trigonometric polynomials in two variables" arXiv:2608.23073 ⓘ · analyzed Aug 29, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Parameterize a multi-channel two-dimensional convolutional operator through a learned filter bank B, then use the composed operator B*B as the layer response. Its Fourier response is positive semidefinite exactly at every spatial frequency, enabling stable smoothing or diffusion-like residual updates without frequency-grid penalty terms.

Formulas

$$A(z,w)=\sum_{j,k=-d}^{d}A_{j,k}z^jw^k,\qquad A(z,w)\succeq0\quad\text{for all }|z|=|w|=1.$$

$$B(z,w)=\sum_{j,k=0}^{e}B_{j,k}z^jw^k,\qquad A(z,w)=B(z,w)^*B(z,w).$$

$$u^*A(e^{i\omega_1},e^{i\omega_2})u=\left\|B(e^{i\omega_1},e^{i\omega_2})u\right\|_2^2\geq0.$$

$$x_{t+1}=x_t-\tau\,\mathcal B^*\mathcal Bx_t,\qquad 0<\tau\leq\frac{2}{\lambda_{\max}(\mathcal B^*\mathcal B)+\varepsilon}.$$

Mathematical statement

Theorem 1.1 states that for integers d and n, if the matrix-valued Laurent polynomial A(z,w) = sum_{j,k=-d}^d A_{j,k} z^j w^k satisfies A(z,w) positive semidefinite for every |z| = |w| = 1, then there exist integers e and m and matrices B_{j,k} in complex m-by-n matrices, with 0 <= j,k <= e, such that A(z,w) = B(z,w)* B(z,w). Here n is the number of input channels, m is the intermediate filter width, z = exp(i omega_1) and w = exp(i omega_2) encode spatial frequencies, and * denotes conjugate transpose. Since u* A(z,w) u = ||B(z,w)u|| squared, the response is nonnegative for every channel vector u. In a CNN, learn the finite-support convolution kernel B directly and define the operator A as B*B, where B* is the channel-transposed, spatially flipped convolution. The theorem guarantees positive semidefiniteness for all frequencies, not merely on a sampled grid.

Implementation notes

Integrate this as a replacement for one full multi-channel spatial convolution in a small CNN. Let the input have n channels and let B be an m-by-n convolution with kernel support 0 <= j,k <= e. Do not learn an independent output kernel. First compute u = B*x, then compute y = B* u, where B* uses channel-transposed kernels and spatially flipped offsets. For a diffusion-style block, use y = x - tau times B*B*x; for a feature transform, use y = B*B*x followed by normalization and an activation. Compact pseudocode is: u = conv2d(x, B, padding='circular'); y = conv2d(u, adjoint_flip(B), padding='circular'); if diffusion: out = x - tau*y; else: out = activation(norm(y)). Use circular padding initially so the Fourier interpretation is exact. Estimate the largest operator eigenvalue by evaluating B(omega_1, omega_2) on a frequency grid and setting lambda_hat_max = max over frequencies of sigma_max(B(omega_1, omega_2)) squared. Choose tau = 1 divided by lambda_hat_max plus a small epsilon. The theorem supplies the exact PSD construction; the finite parameterization, frequency-grid estimate, and normalization are engineering choices. Start with CIFAR-10 and a six-layer CNN, replacing one 3-by-3 convolution by a B*B block with n = m = 64. Compare an equal-parameter unconstrained convolution and a sampled-frequency PSD-penalty baseline. Measure validation accuracy, gradient spikes, high-frequency gain, and robustness to Gaussian input noise. A successful result would show fewer unstable runs, reduced high-frequency amplification, or improved noise robustness at equal FLOPs. For the diffusion variant, separately verify stable linearized dynamics when tau satisfies the estimated spectral bound.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a circular multi-channel PSD convolution block as B*B, with an explicit adjoint and Fourier-spectrum checker. The core claim was numerically confirmed: adjoint relative error was 4.7e-8, the minimum sampled Fourier eigenvalue was 0.198, and the diffusion step using tau=0.1383 had maximum tested norm ratio 0.788. The tiny denoising comparison did not show an accuracy win: PSD test MSE was 0.10220 versus 0.10209 for the unconstrained baseline, while using 54 versus 36 parameters.

Agent confidence: 8/10

Baseline: Unconstrained residual convolution: test MSE 0.1020901, train MSE 0.1087475, 36 parameters
Idea: PSD B*B residual block: test MSE 0.1021957, train MSE 0.1086864, 54 parameters; Fourier minimum eigenvalue 0.198356, maximum 7.230365, diffusion norm ratio 0.787504

Limitations: This was a small synthetic denoising experiment, not CIFAR-10 or a six-layer CNN. Only sampled Fourier frequencies and random-vector diffusion ratios were checked; no exhaustive operator-norm computation, frequency-grid penalty baseline, robustness sweep, gradient-spike analysis, FLOP-normalized comparison, or multiple-seed statistical evaluation was performed.

How to run: python3 run_experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.23073) Sep 3, 2026

No significant difference vs. the tuned baseline (+2.1%, p=0.1898, wins 4 / 8 paired seeds).

Benchmark: Vision (image classification)
Model: cnn_small_psd_matched
Paired seeds: 8
Baseline mean: 0.7859
Idea mean: 0.8028
Effect (Δ): 0.0169 (+2.1%; negative = idea better)
Wins: 4 / 8 paired seeds
p-value: 0.1898 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±3.8%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

Only the fixed small CIFAR-10 vision track and 12 training epochs were tested. The Fourier signature used a finite 16x16 frequency grid; no exhaustive continuous-frequency proof, high-frequency gain, Gaussian-noise robustness, gradient-spike, FLOP, or larger-CIFAR experiment was run.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 3, 2026 latest Failed ✗

Agent confidence: 10/10

Implemented the PSD Spectral CNN block as a matched replacement for the middle spatial convolution in a CIFAR-10 cnn_small system, using circular B*B convolution and the canonical bench training loop. The trained PSD system had higher error than the swept baseline (0.8028125 vs 0.7859375; paired delta +0.016875), with permutation p=0.18975, so there was no significant win. The trained-model Fourier signature confirmed a nonnegative response on the tested 16x16 frequency grid, but this mechanism confirmation did not improve task performance.

Baseline: Vision matched system, best lr=0.006, weight_decay=0.0; 8-seed test error mean 0.7859374881, std 0.0261885418.
Idea: PSD B*B system, best lr=0.003; 8-seed test error mean 0.8028124869, std 0.0279071407; paired delta +0.0168749988, p=0.18975, 4/8 paired wins.
Mechanism signature: Trained PSD model: predicted minimum Fourier eigenvalue 0.0; observed minimum eigenvalue 7.6315516e-06 on a 16x16 frequency grid; observed_nonnegative=true; confirmed=true.

Benchmark result