{
 "artifacts": null,
 "category": "regularization",
 "description": "Add a quartet-based dependence penalty between a learned representation and a nuisance or sensitive variable. Unlike covariance or correlation penalties, the Bergsma–Dassios construction targets arbitrary nonlinear dependence and is valid for discrete, continuous, tied, and mixed data distributions. Replace the discontinuous sign function by a temperature-controlled \\(\\tanh\\) during backpropagation, while evaluating the exact rank statistic separately for monitoring.",
 "formulas_latex": [
  "$$a(x_1,x_2,x_3,x_4)=\\operatorname{sgn}\\!\\big((x_1-x_3)(x_2-x_4)\\big)+\\operatorname{sgn}\\!\\big((x_1-x_4)(x_2-x_3)\\big),\\qquad \\operatorname{sgn}(0)=0.$$",
  "$$\\tau^{*}(X,Y)=\\mathbb{E}\\left[a(X_1,X_2,X_3,X_4)\\,a(Y_1,Y_2,Y_3,Y_4)\\right].$$",
  "$$\\tau^{*}(X,Y)\\ge 2\\mathcal{B}(X,Y),\\qquad \\tau^{*}(X,Y)=0\\ \\Longleftrightarrow\\ X\\perp Y.$$",
  "$$\\widetilde a_{\\epsilon}(x_1,x_2,x_3,x_4)=\\tanh\\!\\left(\\frac{(x_1-x_3)(x_2-x_4)}{\\epsilon}\\right)+\\tanh\\!\\left(\\frac{(x_1-x_4)(x_2-x_3)}{\\epsilon}\\right),\\quad \\widetilde\\tau^{*}_{\\epsilon}=\\frac{1}{M}\\sum_{m=1}^{M}\\widetilde a_{\\epsilon}(z_{i_1},z_{i_2},z_{i_3},z_{i_4})\\widetilde a_{\\epsilon}(y_{i_1},y_{i_2},y_{i_3},y_{i_4}).$$"
 ],
 "id": 2886,
 "implementation": "Integrate the penalty after the encoder and before the task head. Let a minibatch contain encoder outputs \\(z_i\\in\\mathbb{R}^d\\), nuisance values \\(y_i\\in\\mathbb{R}\\) or one scalar projection of a nuisance vector, and task labels. At each training step, sample \\(M\\) distinct index quadruples \\((i_1,i_2,i_3,i_4)\\) from the batch. Choose one representation coordinate, or draw a fresh normalized random projection \\(r\\sim\\mathcal N(0,I_d/d)\\) and set \\(x_i=r^\\top z_i\\). For every quadruple compute \\(u_1=(x_{i_1}-x_{i_3})(x_{i_2}-x_{i_4})\\), \\(u_2=(x_{i_1}-x_{i_4})(x_{i_2}-x_{i_3})\\), and the analogous \\(v_1,v_2\\) from nuisance values. Set \\(a_x=\\tanh(u_1/\\epsilon)+\\tanh(u_2/\\epsilon)\\) and \\(a_y=\\tanh(v_1/\\epsilon)+\\tanh(v_2/\\epsilon)\\), then compute \\(L_{\\mathrm{quartet}}=M^{-1}\\sum_m a_x^{(m)}a_y^{(m)}\\). Optimize \\(L=L_{\\mathrm{task}}+\\lambda L_{\\mathrm{quartet}}\\). Use multiple quartet pools or an exponential moving average because finite-sample estimates can be slightly negative; initially do not clamp inside the gradient. Set \\(\\epsilon=0.1\\) times the batch standard deviation of pairwise representation differences and anneal it to \\(0.01\\). The theorem and inequality come from the paper; soft differentiation, random projections, sampling, and annealing are adaptations. First test a 2-layer MLP on Colored MNIST or Adult, with color or sex as nuisance, comparing ERM, covariance regularization, HSIC, and this penalty at equal batch size. Track task accuracy, a separately trained nuisance predictor, exact held-out hard \\(\\tau^{*}\\), and wall-clock overhead. Success means comparable task accuracy, lower nuisance predictability, and a larger reduction in hard \\(\\tau^{*}\\) under nonlinear nuisance relationships or ties.",
 "math_summary": "For four iid observations \\((X_r,Y_r)\\), \\(r=1,2,3,4\\), define \\(\\operatorname{sgn}(0)=0\\) and the quartet score \\(a(x_1,x_2,x_3,x_4)=\\operatorname{sgn}((x_1-x_3)(x_2-x_4))+\\operatorname{sgn}((x_1-x_4)(x_2-x_3))\\). The Bergsma–Dassios sign covariance is \\(\\tau^{*}(X,Y)=\\mathbb{E}[a(X_1,X_2,X_3,X_4)a(Y_1,Y_2,Y_3,Y_4)]\\), where the four pairs are independent copies from the joint law. The paper proves for arbitrary real-valued bivariate laws, including ties and singular laws, that \\(\\tau^{*}(X,Y)=0\\) if and only if \\(X\\) and \\(Y\\) are independent, and gives the quantitative lower bound \\(\\tau^{*}(X,Y)\\ge 2\\mathcal{B}(X,Y)\\), where \\(\\mathcal{B}\\) is the unscaled Blum–Kiefer–Rosenblatt dependence functional. For differentiable training, substitute \\(\\operatorname{sgn}(u)\\) with \\(s_{\\epsilon}(u)=\\tanh(u/\\epsilon)\\), defining a soft quartet score and its minibatch estimator. The exact independence theorem applies to the hard statistic; the soft version is an optimization surrogate whose temperature should be annealed toward zero.",
 "math_tags": [
  "statistics",
  "probability",
  "combinatorics",
  "information-theory"
 ],
 "ml_areas": [
  "loss",
  "regularization",
  "embedding"
 ],
 "paper": {
  "arxiv_id": "2608.30331",
  "arxiv_url": "https://arxiv.org/abs/2608.30331",
  "summary_what_math_gives_to_ml": "The paper supplies a distribution-free, rank-based independence certificate that remains valid for arbitrary real-valued bivariate laws, including ties, mixed distributions, and singular support. Its key transferable asset is the quantitative relation \\(\\tau^{*}\\ge 2\\mathcal{B}\\): driving the quartet statistic toward zero controls a Blum–Kiefer–Rosenblatt dependence functional rather than merely matching correlations or pairwise moments. A practical neural-network adaptation is a minibatch quartet regularizer computed with a differentiable soft-sign approximation, used to remove nonlinear dependence between a learned representation and a nuisance variable while preserving task information.",
  "title": "Bergsma--Dassios Sign Covariance Characterises Independence for Arbitrary Real-Valued Bivariate Laws",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 7,
  "usefulness": 6
 },
 "solves": [
  "generalization",
  "accuracy",
  "sample-efficiency"
 ],
 "title": "Soft Quartet Independence Regularizer",
 "url": "https://synthcore.org/idea/2886/soft-quartet-independence-regularizer",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}