Curvature-Calibrated Exponential Expert Averaging

(1) Integration point: use this as a prediction-level aggregation layer over M frozen or trainable neural experts, such as LoRA adapters, checkpoints, small subnetworks, or MoE experts. On each validation minibatch, compute every expert's prediction and scalar loss; do not apply the weights to hidden states initially. For squared loss with output and target clipping to [0,b], initialize T=4b^2. For general bounded losses, estimate B from the maximum clipped minibatch loss, estimate L by the maximum finite-difference slope of loss versus prediction on a calibration batch, and use a conservative curvature estimate mu from the loss definition; increase T until (L^2/T)*exp(B/T) <= mu/2. (2) Pseudocode: maintain cumulative losses c[j]=0. For each calibration batch (x,y), compute pred[j]=expert_j(x), batch_loss[j]=mean(loss(pred[j],y)), update c[j] <- c[j]+batch_loss[j], set logits[j] <- -c[j]/T, weights <- softmax(logits), and output pred <- sum_j weights[j]*pred[j]. Recompute weights every K steps or once per epoch. Apply a floor w[j] >= epsilon followed by renormalization if experts are jointly trained. (3) The inequality, the T=4b^2 rule, and the predicted risk scale T log(M)/(n+1) come from the paper; B, L, mu, and the effective sample count n are estimated or known from the implementation. (4) First experiment: CIFAR-10 with M=8 ResNet-18 checkpoints or adapters, compare uniform averaging, ordinary tuned softmax temperatures, T=0.1, and the calibrated temperature. Track validation accuracy, expert entropy, worst-seed variance, and cumulative excess loss. Success is calibrated aggregation avoiding collapse and matching or improving the best expert, especially under distribution shift, with no more than a small inference overhead.