Spatial-depth robust loss gating

Integrate this at the loss-weighting stage of a classifier or contrastive encoder. Let z_i = h_theta(x_i) / (||h_theta(x_i)||_2 + eta) be the representation for minibatch item i, with batch size B and embedding dimension d. Compute pairwise residuals r_{ij} = z_i - z_j and spatial signs q_{ij} = r_{ij} / max(||r_{ij}||_2, epsilon), setting q_{ii}=0. Estimate the spatial distribution at z_i by S_hat_i = (1/B) sum_j q_{ij}. Compute depth d_i = clamp(1 - ||S_hat_i||_2^2, 0, 1). Use detached weights w_i = stopgrad(max(d_i, delta)), normalize them to have mean one, and multiply the per-example cross-entropy or contrastive loss by w_i. Compact pseudocode is: Z=encoder(X); R=Z[:,None,:]-Z[None,:,:]; Q=R/(norm(R,dim=-1,keepdim=True).clamp_min(eps)); Q[diag]=0; S=Q.mean(dim=1); d=(1-S.square().sum(dim=-1)).clamp(0,1); w=stopgrad(d.clamp_min(delta)); loss=(w*per_example_loss).sum()/w.sum(); loss.backward(). The paper provides the dimension-independent empirical-process rationale; the implementation computes the estimator directly and does not require estimating an unknown theorem constant. Start with a 2-layer MLP on CIFAR-10 or a small Transformer on AG News, comparing ordinary training, norm-based outlier weighting, and spatial-depth gating at equal updates. Measure validation accuracy, corruption accuracy, gradient-norm variance, and performance after injecting 10% mislabeled or contaminated examples. Success means lower validation loss and gradient variance under contamination without reducing clean accuracy by more than 0.5 percentage points, together with similar behavior when embedding width increases from 64 to 1024.