Separability-Ambiguity Regularizer

Integrate the method at the penultimate representation z = Phi(x), using a small auxiliary separator ensemble. For each minibatch B, generate K balanced random binary labelings of a subset of examples. Fit a linear separator w_k for each labeling with logistic regression or ridge least squares, and retain only separators whose training accuracy exceeds a chosen threshold and whose validation margin exceeds m. For every current representation z_i, calculate the soft ambiguity estimate P_hat_i as the average over k of sigmoid((delta - absolute value of z_i transposed times w_k) divided by T). Here delta is the distance tolerance defining practical surface membership, T is a temperature controlling smoothness, and lambda is the regularization strength. Add lambda times the mean P_hat_i to cross-entropy, preferably weighting the penalty only on correctly classified examples so that intrinsic class overlap is not forcibly separated. The paper's exact object is the uniform ratio in equation (5.1); the sampled separators and sigmoid are empirical approximations. Begin with a two-dimensional manifold embedded into 128 dimensions, then test a small ResNet on CIFAR-10. Compare cross-entropy, ordinary margin regularization, and the ambiguity penalty at equal parameter count. Track accuracy, expected calibration error, prediction changes under feature noise, and the fraction of points close to many separator hyperplanes. Success means lower calibration error and smoother predictions near the data manifold at matched accuracy; failure would appear as underfitting or degradation on classes with genuine overlap.