# Entropy-stable split quadratic layer

- ID: 2020
- Canonical URL: https://synthcore.org/idea/2020/entropy-stable-split-quadratic-layer
- API JSON: https://synthcore.org/api/idea/2020.json
- API Markdown: https://synthcore.org/api/idea/2020.md
- Verification status: failed_benchmark
- Source: [arXiv:2608.07355](https://arxiv.org/abs/2608.07355)
- Category: dynamics
- Solves: stability, accuracy, speedup
- ML areas: ssm, world-model, training-dynamics
- Math tags: numerical-analysis, dynamical-systems, functional-analysis
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Replace the naive pseudospectral evaluation of a quadratic neural-operator nonlinearity with a two-point split-form product. Use the entropy-stable (alpha, beta) = (1/3, 2/3) split as the default, or learn alpha under the consistency constraint alpha + beta = 1 while monitoring energy growth. The goal is to suppress weakly underresolved aliasing and prevent long-horizon rollout blow-up without full 2/3-rule zero-padding.

## Mathematical statement

The paper represents the quadratic two-point flux as F(u_i,u_j) = (alpha/4)(u_i^2 + u_j^2) + (beta/2)u_i u_j, with splitting coefficients alpha and beta satisfying alpha + beta = 1 so that F(u,u) = u^2/2. For a derivative matrix D with the skew/summation-by-parts structure used by the collocation scheme, the split nonlinear derivative is N_i = 2 sum_j D_ij F(u_i,u_j). Theorem 1 states that, for a valid collocation projection Pi of quadrature strength tau and primary aliasing coefficient d_Pi, the leading aliasing coefficient for modal indices 0 <= i,j,k <= p with i+j+k = tau+2 is Delta_ijk(alpha,beta) = (d_Pi/2) [binom(2i,i) binom(2j,j) binom(2k,k) / binom(2tau+2,tau+1)] [beta(i+j) - alpha(tau+2-(i+j))]. Here binom(a,b) is the binomial coefficient, tau describes the quadrature strength, and d_Pi is the projection's primary aliasing coefficient. The result identifies how the split controls the sign and magnitude of dominant unresolved interactions. The entropy-stable (1/3,2/3) split yields a skew-symmetric aliasing pattern needed for long-time stability. In a neural spectral layer, u is the spatial field, D is a fixed spectral derivative operator, and F and N are applied independently over batch and channel or feature groups.

## Key formulas

- $$F(u_i,u_j)=\frac{1}{4}\alpha(u_i^2+u_j^2)+\frac{1}{2}\beta u_i u_j,\qquad \alpha+\beta=1,$$
- $$N_i^{\alpha,\beta}(u)=2\sum_j D_{ij}F(u_i,u_j),$$
- $$\Delta_{ijk}(\alpha,\beta)=\frac{d_{\Pi}}{2}\frac{\binom{2i}{i}\binom{2j}{j}\binom{2k}{k}}{\binom{2\tau+2}{\tau+1}}\left(\beta(i+j)-\alpha(\tau+2-(i+j))\right),\quad i+j+k=\tau+2,$$
- $$\alpha_{\mathrm{ES}}=\frac{1}{3},\qquad \beta_{\mathrm{ES}}=\frac{2}{3},\qquad F_{\mathrm{ES}}(u_i,u_j)=\frac{u_i^2+u_j^2}{12}+\frac{u_i u_j}{3}.$$

## Implementation notes

Integrate this at the quadratic nonlinear block of a Fourier neural operator, neural PDE timestepper, or learned state-space/world model that currently computes a product by inverse FFT, pointwise multiplication, and forward FFT. Let z have shape [batch, channels, grid], and let D be the periodic spectral derivative, implemented for a small prototype as a dense matrix or as FFT multiplication by ik. First create a learned projection z = W x, optionally grouping channels so the quadratic operation acts within each group. Implement the reference operation directly: `F_ij = alpha/4*(z_i**2 + z_j**2) + beta/2*z_i*z_j`; `N_i = 2*sum_j(D_ij*F_ij)`. Then insert `y = linear(x) + gamma*N`, with gamma initialized to 0.1. For the FFT-efficient implementation, use the algebraically equivalent split `N = alpha*D(z**2/2) + beta*z*D(z)`, because `2 sum_j D_ij F(z_i,z_j)` reduces to this expression when D has zero row sums and the diagonal/self terms are handled consistently. Use alpha=1/3 and beta=2/3 for the entropy-stable variant. Compare conservative `(1,0)`, skew/entropy-stable `(1/3,2/3)`, symmetric `(1/2,1/2)`, and a zero-padded 2/3-dealiasing baseline. The formulas provide the split flux and predicted leading aliasing coefficient; estimate tau from the grid/quadrature order and estimate d_Pi by measuring the dominant projection error on modal test functions, but do not require these estimates for training. Start with a 1D Burgers or Kuramoto-Sivashinsky dataset on 64 grid points using a small FNO or residual spectral model. Equalize parameters, FFT count, and training steps. Track one-step MSE, rollout MSE, energy E_t = mean(z_t^2)/2, maximum energy growth, and time-to-divergence. The hypothesis is that the entropy-stable split substantially reduces unstable long rollouts relative to the conservative split, while approaching padded-dealiasing accuracy with lower memory and compute.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: no_effect
- Verdict: Built and numerically tested the split quadratic spectral derivative with alpha+beta=1, including the entropy-stable (1/3,2/3) choice, padded reference, dense two-point flux, and a toy Burgers rollout. The affine aliasing prediction matched to 8.9e-16, the predicted cancellation roots matched across the index sweep to approximately 1e-16, and the dense flux/FFT identity was checked numerically. Relative to the conservative baseline alpha=1, the entropy-stable split reduced mean absolute energy injection from 0.1317 to 0.1317 (equal here because alpha=1 and 1/3 are symmetric in this metric), reduced padded-reference RMSE from 1.6446 to 1.8598 (worse), but reduced 250-step maximum energy growth from 1.6785x to 1.3644x; thus the mechanism manifested, although it did not dominate all accuracy metrics and the symmetric split was best in this toy.

### Mechanism check

- Verdict: Built and numerically tested the split quadratic spectral derivative with alpha+beta=1, including the entropy-stable (1/3,2/3) choice, padded reference, dense two-point flux, and a toy Burgers rollout. The affine aliasing prediction matched to 8.9e-16, the predicted cancellation roots matched across the index sweep to approximately 1e-16, and the dense flux/FFT identity was checked numerically. Relative to the conservative baseline alpha=1, the entropy-stable split reduced mean absolute energy injection from 0.1317 to 0.1317 (equal here because alpha=1 and 1/3 are symmetric in this metric), reduced padded-reference RMSE from 1.6446 to 1.8598 (worse), but reduced 250-step maximum energy growth from 1.6785x to 1.3644x; thus the mechanism manifested, although it did not dominate all accuracy metrics and the symmetric split was best in this toy.
- Confidence: 8/10
- Limitations: This is an analytic/tiny spectral toy, not a trained FNO or PDE dataset experiment. It does not test learned alpha, parameter/FLOP-equalized training, noisy long-horizon supervised rollouts, multiple resolutions, or a rigorous entropy bound; the padded-reference implementation and energy metric are only prototype diagnostics. The rollout compares several fixed split forms and did not produce threshold divergence within 250 steps.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.28455
- Paired wins: 4/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_report.json](https://synthcore.org/code/818/bench_report.json)
- [report.md](https://synthcore.org/code/818/report.md)
- [report_bench_2026-09-01T111823.md](https://synthcore.org/code/818/report_bench_2026-09-01T111823.md)
- [results.json](https://synthcore.org/code/818/results.json)
- [split_bench.py](https://synthcore.org/code/818/split_bench.py)
- [split_quadratic_experiment.py](https://synthcore.org/code/818/split_quadratic_experiment.py)
- [Download all files as ZIP](https://synthcore.org/download/818)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.