RG Spectral Feature Gate
Implementation & benchmark of arXiv:2607.27236 — Data Field Theory: Theory and Applications of the Functional Renormalization Group for Signal Detection
Source paper: Data Field Theory: Theory and Applications of the Functional Renormalization Group for Signal Detection arXiv:2607.27236 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Replace fixed PCA-rank selection in a hidden layer with a renormalization-group-inspired gate over covariance eigenvalue bands. The gate retains modes whose effective quartic interaction remains unstable or strongly scale-dependent, while pruning bands that flow toward the Gaussian noise fixed point. Unlike top-eigenvalue truncation, this is designed for extensive-rank signal distributed throughout the bulk spectrum.
Formulas
Mathematical statement
The paper defines the empirical correlation matrix C from activations X in R^{N times P} by C_{ij} = (C_0)_{ij} divided by the square root of (C_0)_{ii}(C_0)_{jj}, with covariance C_0 = X^T X divided by N, where N is the number of samples and P is the feature dimension. Pure noise has the Marchenko--Pastur eigenvalue density mu_MP(lambda) = square root of ((lambda_+ - lambda)(lambda - lambda_-)) divided by (2 pi sigma^2 q lambda), where q = P/N, noise variance is sigma^2, and lambda_+ and lambda_- equal sigma^2(1 plus or minus square root of q)^2. The field is expanded in the correlation eigenbasis as phi_i = sum_lambda u_i^(lambda) varphi_lambda, and the retained modes are coupled through the quartic interaction V(phi) = u_4 sum_i phi_i^4 divided by 4!. The paper's detection rule uses the scale-dependent canonical dimension of u_4: the critical scale lambda_c satisfies dim_tau(u_4) evaluated at lambda_c equals zero, while an optimal scale is a local minimum satisfying the derivative of dim_tau(u_4) with respect to beta equals zero. We use spectral scale tau_r = log(lambda_r + epsilon), estimate a bandwise quartic coefficient u_{4,r} from excess fourth cumulants, and use its finite-difference logarithmic slope as a practical proxy for the canonical dimension.
Implementation notes
1. Integration point: apply the gate to the feature dimension of a transformer MLP output, convolutional channel activation, or embedding table before the next linear projection. During calibration, collect a rolling matrix X of centered activation vectors from N samples, compute the correlation matrix C, and eigendecompose C = U diag(lambda_1,...,lambda_P) U^T. Do not simply keep the largest eigenvalues. 2. Pseudocode: X <- centered activation batch; C <- corr(X); lambda,U <- eigh(C); form logarithmic or quantile spectral bands B_r; z_r <- X U[:,B_r]; kappa4_r <- mean(z_r^4) - 3 mean(z_r^2)^2; u4_r <- -kappa4_r/(mean(z_r^2)^2 + eps); d_r <- finite_difference(log(abs(u4_r)+eps), log(lambda_band+eps)); rc <- first r with a sign change in d_r that persists for two bands; mask[B_r] <- 1 for bands on the signal side of rc and 0 otherwise; h_next <- W(mask * U z). Use a straight-through estimator for the binary mask during training, or freeze the mask after calibration. 3. Compute the MP edges from q = P/N and estimate sigma^2 from the median bulk eigenvalue. Estimate u4_r and the crossing from minibatches, using an exponential moving average and bootstrap resampling to prevent unstable gates. 4. First experiment: use a 6-layer transformer on CIFAR-10 or a small language-model benchmark, insert the gate after every MLP, and compare dense training, fixed top-k PCA, and magnitude pruning at equal retained width. Measure validation accuracy, retained FLOPs, and mask stability across batches. Success means higher accuracy than top-k PCA at the same width, or at least 20 percent lower width at matched accuracy, with a persistent spectral crossing rather than a validation-tuned cutoff.
Verification
Stage 1 · Toy mechanism gate: Failed ✗
Stage 2 · Mechanism transferred to benchmark: Not tested
Stage 2 · Practical benchmark result: Not run
Methodology: Toy-system gate first; the benchmark stage runs only after a pass. How verification works
Stage 1 — Mechanism check agent confidence 9/10
Built and numerically checked an RG spectral feature-gate MVP with correlation eigendecomposition, bandwise excess-kurtosis estimates, logarithmic slope scoring, and a synthetic downstream comparison. Noise scaling matched N^-1/2 (-0.523 observed vs -0.5 predicted), and |u4| grew approximately linearly with non-Gaussian strength, but low-exponent slope recovery was biased and the RG gate reached 0.525 accuracy versus 0.957 for top-PCA at equal retained width. Therefore the proposed mechanism and practical win were not established.
- Agent confidence
- 9/10
- Baseline
- Top-16 PCA accuracy: 0.9573
- Idea
- RG gate accuracy: 0.5247; noise exponent: -0.5231 observed vs -0.5 predicted; controlled slopes: [0.849, 1.389, 2.009] vs [0.5, 1.0, 2.0] predicted; |u4| by strength: [0.050, 0.160, 0.308, 0.514, 0.716]
Limitations: Only a synthetic NumPy experiment was tested. No transformer, CIFAR-10, training-time straight-through gate, bootstrap/EMA calibration, FLOP or memory measurement, or minibatch mask-stability test was performed; the practical gate is a heuristic rather than the paper's full canonical-dimension flow.
How to run: python3 rg_gate_experiment.py
Verdict computed by deterministic test code from paired-seed statistics — not by the language model.
Artifacts
- report.md 1.4 KB View
- results.json 1.6 KB View Raw
- rg_gate_experiment.py 5.9 KB View Raw