{
 "artifacts": null,
 "category": "regularization",
 "description": "Represent binary nuisance variables or augmentation bits as coordinates of a Hamming cube and penalize the model response that remains predictable from a random k-coordinate subset. Use the theorem's derivative-plus-global-norm certificate as the regularizer, retaining its p/log p dependence instead of using an arbitrary masking penalty. Random coordinate permutations provide a cheap stochastic approximation to the subset average.",
 "formulas_latex": [
  "$$\\left(\\frac{1}{\\binom{n}{k}}\\sum_{\\substack{S\\subseteq[n]\\\\|S|=k}}\\|E_{[n]\\setminus S}h\\|_{p}^{p}\\right)^{1/p}\\lesssim\\frac{p}{\\log p}\\left(\\frac{k}{n}\\sum_{j=1}^{n}\\|\\partial_{j}h\\|_{p}^{p}+\\left(\\frac{k}{n}\\right)^{p/2}\\|h\\|_{p}^{p}\\right)^{1/p}.$$",
  "$$\\partial_j h(x)=\\frac{h(x)-h(x^{\\oplus j})}{2},\\qquad h_c(x)=h(x)-\\mathbb{E}_{x'}h(x').$$",
  "$$\\widehat{E}_{[n]\\setminus S}h(x)=\\frac{1}{M}\\sum_{m=1}^{M}h\\left(x_S,u^{(m)}_{[n]\\setminus S}\\right),\\qquad S=\\{\\pi_1,\\ldots,\\pi_k\\}.$$",
  "$$R_{\\mathrm{RR}}=\\frac{p}{\\log p}\\left[\\frac{k}{n}\\sum_{j=1}^{n}\\widehat{\\|\\partial_j h_c\\|_p^p}+\\left(\\frac{k}{n}\\right)^{p/2}\\widehat{\\|h_c\\|_p^p}\\right]^{1/p}.$$"
 ],
 "id": 2826,
 "implementation": "Integrate the regularizer at the output of a classifier or regressor when the input contains n binary nuisance or augmentation coordinates. For minibatch examples (x_i,z_i), with z_i in {-1,1}^n, let f_theta(x_i,z_i) be one scalar logit or prediction and define h_i=f_theta(x_i,z_i)-stopgrad(B^{-1} sum_b f_theta(x_b,z_b)) so that h is approximately mean-zero. Add lambda times R_RR to the task loss. At each training step, sample a random permutation pi of [n], set S={pi_1,...,pi_k}, and draw M independent nuisance vectors u^(m) from the empirical nuisance distribution while keeping x_i and z_{i,S} fixed. Evaluate f_theta(x_i,(z_{i,S},u^(m)_{-S})) and average over m to estimate E_[n]\\S h. For the derivative term, either flip every bit j or sample one or two coordinates per example; evaluate f_theta(x_i,z_i^oplus j) and compute partial_j h_i=(h_i-h_i^oplus j)/2. Estimate each L^p quantity with minibatch averages of absolute values raised to p, use p=4 or p=8, take the outer 1/p power, and multiply by p/log(p). The theorem determines the derivative-plus-global-norm structure and its p/log(p) scaling; the conditional expectation, derivative sampling, and universal constant are estimated empirically. Begin with a small MLP or CNN on Colored-MNIST or CIFAR-10 augmented with a synthetic binary color or style code. Compare ERM, ordinary random masking or dropout, and this regularizer at equal training FLOPs. Measure clean accuracy, accuracy under nuisance-code shifts, worst-group accuracy, and the empirical random-reveal projection norm. A successful result is higher shifted-nuisance or worst-group accuracy at matched clean accuracy, alongside a reduction in the measured conditional-projection norm.",
 "math_summary": "Theorem 1 states that for p\u003e=2, n in the natural numbers, k in [n], and a mean-zero function h:Omega_n to R, where Omega_n={-1,1}^n, E_[n]\\S h is conditional expectation over coordinates outside S, and partial_j h is the discrete coordinate derivative, one has (1/binomial(n,k) sum over S with |S|=k of ||E_[n]\\S h||_p^p)^(1/p) \u003c= C(p/log p)((k/n) sum from j=1 to n of ||partial_j h||_p^p + (k/n)^(p/2)||h||_p^p)^(1/p). Here ||f||_p=(E_x |f(x)|^p)^(1/p), x^oplus j flips bit j, and partial_j h(x)=(h(x)-h(x^oplus j))/2. The left side measures the average L^p magnitude of the response predictable from a random k-coordinate subset. The right side is a sharp certificate in terms of coordinate sensitivity and total centered response; C is a universal constant and p/log p is optimal up to a universal multiplicative constant. The paper obtains the subset average by revealing coordinates in a uniformly random permutation and applying a sharp martingale Rosenthal inequality.",
 "math_tags": [
  "probability",
  "stochastic-processes",
  "harmonic-analysis"
 ],
 "ml_areas": [
  "regularization",
  "data-augmentation",
  "mlp"
 ],
 "paper": {
  "arxiv_id": "2608.29367",
  "arxiv_url": "https://arxiv.org/abs/2608.29367",
  "summary_what_math_gives_to_ml": "The paper proves a sharp higher-order inequality on the Hamming cube: the average L^p magnitude of a conditional projection onto a random k-coordinate subset is controlled by coordinate-wise discrete derivatives and the global centered L^p norm, with the sharp factor p/log p. This suggests a robustness regularizer for neural networks whose inputs contain binary nuisance or augmentation variables, suppressing predictable responses to randomly revealed nuisance coordinates while explicitly controlling coordinate sensitivity. The random-reveal martingale construction gives an efficient unbiased estimator based on random permutations and partial masking rather than enumerating all subsets. The most direct test is on nuisance-invariance benchmarks, measuring robustness at matched clean accuracy and training cost.",
  "title": "Sharp Metric $X_p$ Inequalities via Martingales",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 7,
  "usefulness": 6
 },
 "solves": [
  "generalization",
  "stability",
  "accuracy"
 ],
 "title": "Sharp random-reveal nuisance regularizer",
 "url": "https://synthcore.org/idea/2826/sharp-random-reveal-nuisance-regularizer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}