Projected Non-Gaussian Confidence Loss

Use a probabilistic MLP or deterministic predictor with input perturbations z sampled from a zero-mean Gaussian distribution with covariance Sigma_x. At each training batch, choose R equal to 4 through 16 unit directions in output space, including the residual direction and random orthogonal directions. Draw K perturbed inputs x plus z_k, evaluate y_k = f_theta(x + z_k), and center the outputs using their batch mean. For each direction u_r, compute q_kr = u_r transposed times (y_k minus y_bar), then calculate m2_r = K^{-1} sum q_kr squared, m3_r = K^{-1} sum q_kr cubed, and m4_r = K^{-1} sum q_kr to the fourth power. Set gamma_r = m3_r divided by (m2_r plus epsilon) to the three-halves power, and kappa_r = m4_r divided by (m2_r plus epsilon) squared minus 3. Add alpha times the projected moment loss to the supervised loss. A cheaper analytic variant computes directional Jacobian-vector products and Hessian-vector products, then uses Gaussian moment identities instead of K forward passes. The paper supplies the directional contraction strategy; empirical training supplies target moments, direction selection, alpha, and epsilon. First test on a two-layer MLP for UCI California Housing, comparing MSE, Gaussian mean-variance NLL, and this loss at equal forward-pass budgets. Evaluate with 4096 Monte Carlo samples. The prediction is that sampling error in gamma and kappa decreases as K^{-1/2}; the Taylor estimator should be accurate while the perturbation scale s satisfies s times the Hessian norm divided by the Jacobian norm less than approximately 0.2, followed by a sharp degradation beyond that regime. Non-Gaussian calibration should improve most on inputs where E_uuv is measurably nonzero.