V-Fold Jackknife Neural Uncertainty

Integrate this at the estimator-output level, not inside the optimizer. Choose a prediction functional: for regression use $\Psi(x)=f_\theta(x)$; for classification use one probability or logit; for a scalar evaluation metric use the validation metric computed on a fixed evaluation set. Split the training data into V disjoint folds, with V=5 as the first experiment. Train one full model to obtain $\hat\Psi$ and V models on data excluding fold v to obtain $\hat\Psi_{-v}$. For every evaluation point x, execute: J[v] = V*full_pred[x] - (V-1)*loo_pred[v,x]; Jbar = mean(J, axis=0); var[x] = sum((J[:,x]-Jbar[x])**2)/(V*(V-1)); se[x] = sqrt(var[x]+1e-8); interval[x] = full_pred[x] +/- t_quantile(V-1, 1-alpha/2)*se[x]. For vector outputs, estimate the correlation matrix from centered pseudo-values and either sample a multivariate t distribution with degrees of freedom V-1 or use a bootstrap over the V pseudo-value vectors to estimate the critical value for max_k abs(T[k]). The mathematical quantities are the pseudo-value transformation, variance factor, and t calibration; neural predictions and fold-to-fold covariance are estimated empirically. First test on a small MLP regressor on a UCI dataset, comparing ordinary dropout intervals and a deep ensemble at equal total refit count. Measure empirical 90% coverage, interval width, calibration error, and coverage under distribution shift. Success means coverage closer to nominal without requiring a large ensemble, particularly where dropout or Hessian-based standard errors are anti-conservative.