# Distributed E-Value Prediction Sets

- ID: 939
- Canonical URL: https://synthcore.org/idea/939/distributed-e-value-prediction-sets
- API JSON: https://synthcore.org/api/idea/939.json
- API Markdown: https://synthcore.org/api/idea/939.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.14906](https://arxiv.org/abs/2607.14906)
- Category: regularization
- Solves: accuracy, stability, generalization
- ML areas: graph-nn, regularization, training-dynamics, inference-speedup
- Math tags: probability, statistics, information-theory, graph-theory
- Ratings: usefulness 7/10; difficulty 5/10; novelty 5/10

## Idea description

Equip each neural-network expert or robot with a locally calibrated e-value for every candidate label, then fuse neighboring e-values using uncertainty-attenuated convex weights. At inference time, retain all labels whose fused e-value does not cross the finite-sample rejection threshold, so the model abstains instead of making an unsupported point prediction. This transfers the paper's coverage-recovery mechanism to ensembles, federated models, and graph neural networks.

## Mathematical statement

For candidate label y, expert i produces a nonnegative e-value e_i(x,y), where x is the input. Under the null hypothesis that y is the true label, local validity means E[e_i(X,Y)] <= 1. Let w_i(x) be nonnegative uncertainty-attenuation weights over the active neighborhood S(x), with sum_i w_i(x)=1. The fused e-value is e_F(x,y)=sum_i w_i(x)e_i(x,y). If the weights are fixed from calibration-time information or are otherwise valid under the null, then E[e_F(X,Y)] <= 1 without requiring independence between experts. Markov's inequality gives P(e_F(X,Y) >= 1/alpha) <= alpha, so the prediction set C_alpha(x)={y:e_F(x,y)<1/alpha} has coverage at least 1-alpha. A possible attenuation rule is w_i(x)=exp(-kappa u_i(x))/sum_j exp(-kappa u_j(x)), where u_i(x) is a calibrated uncertainty score and kappa is nonnegative.

## Key formulas

- $$e_i(x,y)\ge 0,\qquad \mathbb{E}[e_i(X,Y)]\le 1$$
- $$w_i(x)=\frac{\exp[-\kappa u_i(x)]}{\sum_{j\in S(x)}\exp[-\kappa u_j(x)]},\qquad e_F(x,y)=\sum_{i\in S(x)}w_i(x)e_i(x,y)$$
- $$C_\alpha(x)=\left\{y:e_F(x,y)<\frac{1}{\alpha}\right\},\qquad \Pr(Y\notin C_\alpha(X))\le \alpha$$
- $$\Pr\left(e_F(X,Y)\ge\frac{1}{\alpha}\right)\le \alpha\,\mathbb{E}[e_F(X,Y)]\le\alpha$$

## Implementation notes

1. Integration point: use an ensemble of M neural classifiers, federated clients, or graph-neural-network nodes. Each model outputs logits, probabilities, and a scalar uncertainty u_i(x), such as entropy or calibrated energy. Reserve a calibration split and convert each model's nonconformity score into a locally valid e-value. A simple MVP is e_i(x,y)=exp(-lambda s_i(x,y))/Z_i, with the normalizer Z_i selected on calibration data so the held-out mean e-value for each label is at most one. 2. Pseudocode: for each input x, collect outputs from active neighbors; compute w_i=softmax(-kappa u_i); for every label y compute e_F(y)=sum_i w_i e_i(y); return C={y:e_F(y)<1/alpha}; abstain if C contains multiple labels or is empty. 3. Computed directly from the mechanism are nonnegative e-values, convex fusion, and the 1/alpha threshold. Estimated empirically are calibration normalizers, uncertainty scores, and graph effects. Avoid weights trained on the same target label without recalibration, because data-dependent weights can invalidate the expectation bound. 4. First cheap experiment: train five small MLPs or CNNs on CIFAR-10, calibrate each on a held-out split, and compare one model, unweighted fusion, and uncertainty-weighted fusion at alpha=0.1. Measure marginal coverage, average set size, and abstention while varying the graph from disconnected to complete. The prediction is coverage at least 0.90 up to sampling error for every graph and noise distribution, while denser graphs reduce ambiguous prediction sets. The empirical rejection frequency should track the Markov boundary e_F>=10. Falsification occurs if coverage is below 0.90 despite valid local calibration or if abstention does not generally decrease with neighborhood size.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: no_effect
- Verdict: Built and ran a fixed-seed distributed e-value ensemble MVP with calibration normalizers, uncertainty-attenuated convex fusion, and prediction-set inference. Fused empirical mean e-values matched the convex weighted prediction to at most 1.1e-16 across all kappa values, coverage stayed above 0.9995 at alpha=0.1, and increasing kappa reduced average set size from 3.57 to 2.84 while increasing singleton rate from 0.052 to 0.161. The mechanism manifested, but the idea did not beat the strongest-expert baseline on set size at alpha=0.1.

### Mechanism check

- Verdict: Built and ran a fixed-seed distributed e-value ensemble MVP with calibration normalizers, uncertainty-attenuated convex fusion, and prediction-set inference. Fused empirical mean e-values matched the convex weighted prediction to at most 1.1e-16 across all kappa values, coverage stayed above 0.9995 at alpha=0.1, and increasing kappa reduced average set size from 3.57 to 2.84 while increasing singleton rate from 0.052 to 0.161. The mechanism manifested, but the idea did not beat the strongest-expert baseline on set size at alpha=0.1.
- Confidence: 8/10
- Limitations: This is a synthetic toy ensemble, not CIFAR-10, federated learning, or a graph neural network. Calibration validity is empirical with a 1% safety factor rather than a formal finite-sample construction; uncertainty weights are fixed externally, and learned data-dependent weights, distribution shift, repeated random seeds, runtime, communication cost, and larger neural models were not tested.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.7525
- Paired wins: 1/8
- Benchmark verdict: no measurable effect

## Artifacts

- [bench_experiment.py](https://synthcore.org/code/398/bench_experiment.py)
- [bench_report.json](https://synthcore.org/code/398/bench_report.json)
- [distributed_evalue_experiment.py](https://synthcore.org/code/398/distributed_evalue_experiment.py)
- [report.md](https://synthcore.org/code/398/report.md)
- [report_bench_2026-08-31T112055.md](https://synthcore.org/code/398/report_bench_2026-08-31T112055.md)
- [results.json](https://synthcore.org/code/398/results.json)
- [Download all files as ZIP](https://synthcore.org/download/398)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.