Degree-Weighted Fourier Collision Regularizer

1. Integration point: attach the regularizer to two scalar heads f_theta(x) and g_phi(x) of an MLP or multi-task network, where x∈{0,1}^n and each head outputs a sigmoid probability. During each training step, compute the task losses normally and add the spectral penalty to the loss. For the exact experiment, use thresholded outputs or sample Bernoulli outputs from the sigmoid; for differentiable training, use sigmoid probabilities directly as a continuous surrogate. Enforce increasingness either structurally with nonnegative linear weights and monotone activations, or with the finite-difference penalty shown above. 2. Pseudocode: sample a minibatch of B binary inputs; evaluate f_b=f_theta(x_b), g_b=g_phi(x_b); estimate c=mean(f_b*g_b)-mean(f_b)*mean(g_b). For n≤12, enumerate all 2^n cube points, evaluate both heads, compute the full Walsh-Hadamard transforms F_S=mean_x f(x)χ_S(x) and G_S=mean_x g(x)χ_S(x), then set q=sum_{S≠∅}|S|*F_S^2*G_S^2 and L_spec=max(0,4*q-c). Backpropagate L_task+lambda_spec*L_spec. For larger n, sample R independent cube points and estimate F_S and G_S for selected subsets S, preferably all subsets up to degree d; use q_d=sum_{1≤|S|≤d}|S|F_S^2G_S^2 as a cheaper surrogate. 3. The factor 4 and degree weighting come directly from the theorem; Fourier coefficients, covariance, and monotonicity violations are estimated from the current minibatch or exact cube enumeration. Log the unpenalized violation 4q−c as the primary diagnostic. 4. First experiment: train two monotone binary-input MLP heads on a synthetic multi-task dataset with noisy singleton and pairwise interaction labels, comparing the baseline, a covariance-only penalty, and this regularizer at matched parameter count and FLOPs. Measure validation loss, calibration under single-bit perturbations, empirical violation rate, and Fourier energy by degree. A successful result should reduce high-degree spectral collision and perturbation instability without increasing task loss; the initial target is lower validation error or improved robustness at equal training cost.