Sharp random-reveal nuisance regularizer

Integrate the regularizer at the output of a classifier or regressor when the input contains n binary nuisance or augmentation coordinates. For minibatch examples (x_i,z_i), with z_i in {-1,1}^n, let f_theta(x_i,z_i) be one scalar logit or prediction and define h_i=f_theta(x_i,z_i)-stopgrad(B^{-1} sum_b f_theta(x_b,z_b)) so that h is approximately mean-zero. Add lambda times R_RR to the task loss. At each training step, sample a random permutation pi of [n], set S={pi_1,...,pi_k}, and draw M independent nuisance vectors u^(m) from the empirical nuisance distribution while keeping x_i and z_{i,S} fixed. Evaluate f_theta(x_i,(z_{i,S},u^(m)_{-S})) and average over m to estimate E_[n]\S h. For the derivative term, either flip every bit j or sample one or two coordinates per example; evaluate f_theta(x_i,z_i^oplus j) and compute partial_j h_i=(h_i-h_i^oplus j)/2. Estimate each L^p quantity with minibatch averages of absolute values raised to p, use p=4 or p=8, take the outer 1/p power, and multiply by p/log(p). The theorem determines the derivative-plus-global-norm structure and its p/log(p) scaling; the conditional expectation, derivative sampling, and universal constant are estimated empirically. Begin with a small MLP or CNN on Colored-MNIST or CIFAR-10 augmented with a synthetic binary color or style code. Compare ERM, ordinary random masking or dropout, and this regularizer at equal training FLOPs. Measure clean accuracy, accuracy under nuisance-code shifts, worst-group accuracy, and the empirical random-reveal projection norm. A successful result is higher shifted-nuisance or worst-group accuracy at matched clean accuracy, alongside a reduction in the measured conditional-projection norm.