Certified ambiguity gating for LLM supervision

1. Integration point: insert an ambiguity-audit pass immediately before fine-tuning on LLM-generated labels, preference pairs, reward-model data, or weak labels. For every training input x, query the frozen supervisor with the exact production prompt u=\Pi(f_{\mathrm{NL}},x), compute label probabilities through constrained decoding or repeated sampled generations plus the verbalizer V, and retain the resulting admissible set A_ au(x). The trainable network receives x as usual, but its loss is modified using the ambiguity score a_ au(x) or a continuous margin m(x)=p_{(1)}(x)-p_{(2)}(x), where p_{(1)} and p_{(2)} are the largest and second-largest label probabilities. 2. Pseudocode: for x in audit_set: p=llm_label_distribution(Pi(spec,x)); A={y:p[y]>=tau}; a=1 if len(A)>=2 else 0; Dhat=mean(a); eps=sqrt(log(1/delta)/(2*n)); floor=max(0,Dhat-eps)/2; during training use p_teacher=llm_label_distribution(Pi(spec,x)); w=1-a_tau(x); loss=w*CE(student(x),hard_label) for gating, or loss=CE(student(x),normalize(p_teacher)) on ambiguous examples. 3. Compute A_tau, a_tau, Dhat, eps, and the certified floor from the formulas. Estimate p(y|u) using normalized verbalizer logits, constrained beam probabilities, or M stochastic generations; exact summation over all token strings is not required but the decoding protocol must remain fixed. 4. First experiment: use a binary sentiment or toxicity dataset, label it with a frozen 3B instruction model, and fine-tune a small RoBERTa classifier. Compare hard-label cross-entropy, ordinary confidence filtering, and this admissibility gate or soft-label loss at matched retention rates and teacher-query budgets. Audit 1,000 held-out unlabeled inputs and report Dhat, its lower confidence bound, test accuracy, calibration error, and discarded fraction. Success is lower test error or expected calibration error at equal student FLOPs; a negative result is no improvement over confidence filtering after matching retention.