Histogram-Preserving Variation Projection
Implementation & benchmark of arXiv:2607.03450 — A variation on the Pólya-Segő principle in one dimension
Source paper: A variation on the Pólya-Segő principle in one dimension arXiv:2607.03450 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Insert a rearrangement operation on scalar feature maps sampled along an ordered coordinate such as time, spatial position, or a neural-field input grid. The operation sorts values into non-increasing order, preserving the empirical histogram exactly while provably not increasing the Riesz fractional variation in the ideal one-dimensional continuous setting.
Formulas
Mathematical statement
For an interval I=[a,b], the Riesz (p,alpha)-variation is \(\mathcal{V}_{p}^{\alpha}(f;I)=\sup_{\mathcal{I}}(\sum_{I_j\in\mathcal{I}}({\rm osc}(f;I_j)/|I_j|^{\alpha})^p)^{1/p}\), where \(\mathcal{I}\) ranges over families of pairwise non-overlapping subintervals, \({\rm osc}(f;E)=\sup_{x\in E}f(x)-\inf_{x\in E}f(x)\), \(1\le p<\infty\), and \(0\le\alpha\le 1/p'\) with \(p'=p/(p-1)\) for \(p>1\) and \(1'=\infty\). If \(f^*\) is the non-increasing rearrangement of f, then the paper proves \(\mathcal{V}_{p}^{\alpha}(f^*;I)\leq\mathcal{V}_{p}^{\alpha}(f;I)\). For a sampled vector \(z\in\mathbb{R}^n\), use the discrete rearrangement \(z^*=\operatorname{sort}(z,\mathrm{descending})\), and estimate variation over contiguous index intervals by \(\widehat{\mathcal V}_{p}^{\alpha}(z)=\max_{\mathcal P}(\sum_{[l,r]\in\mathcal P}((\max_{i\in[l,r]}z_i-\min_{i\in[l,r]}z_i)/(r-l+1)^\alpha)^p)^{1/p}\). The continuous theorem motivates the projection, while the exact finite-sample guarantee must be checked empirically or established separately for the chosen discrete estimator.
Implementation notes
(1) Integration point: apply the operation to selected scalar channels of a tensor with a meaningful ordered axis, such as the time axis of an SSM or temporal MLP, or the coordinate axis of a one-dimensional neural field. For X with shape [batch, channels, length], compute a sorted version along the last axis and mix it into the original activation. (2) Pseudocode: z=X[b,c,:]; order=argsort(z,descending=True); zstar=z[order]; Y[b,c,:]=(1-lambda)*z+lambda*zstar, where lambda is ramped from 0 to 1 or applied only to a subset of channels. Hard sorting has discontinuous rank changes, so use stop-gradient through the permutation, a straight-through estimator, or a differentiable soft-sort relaxation. (3) Compute the paper's rearrangement exactly for each finite vector, but estimate the continuous variation with a discrete interval estimator. For short sequences enumerate all contiguous interval partitions using dynamic programming; for long sequences sample random multiscale partitions. Compare variation before and after the layer and verify that the empirical activation histogram is unchanged. (4) First experiment: train a small temporal MLP or SSM on synthetic noisy sequence classification and a real 1D benchmark, comparing no projection, hard sorting, and soft sorting at equal parameter count and FLOPs. Record validation accuracy, robustness to temporal noise, calibration, empirical fractional variation, and gradient norm variance. Success means lower variation and improved noise robustness without a substantial clean-accuracy loss; failure is positional-information destruction or unstable gradients.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.