✗ Failed on benchmark 2026

MI-Guided Latent Protection

Implementation & benchmark of arXiv:2608.16227 — Adaptive Unequal Error Protection for Semantic Split Learning over Wireless Channels

Usefulness7/10

Difficulty4/10

Novelty6/10

Regularization Accuracy Stability Memory Embedding Quantization Regularization Loss Information theory Statistics Optimization

Source paper: Adaptive Unequal Error Protection for Semantic Split Learning over Wireless Channels arXiv:2608.16227 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the estimated task relevance of each latent coordinate to allocate corruption, precision, or redundancy non-uniformly rather than applying uniform dropout or quantization noise. Coordinates with larger mutual-information sensitivity receive lower noise or more bits, while low-relevance coordinates are compressed or corrupted more aggressively.

Formulas

$$\min I(X;Z)\quad\text{s.t.}\quad I(Z;T(X))\geq\epsilon,$$

$$s_k=\mathbb{E}_{(z,t)}\left[\left|\frac{\partial\widehat I(Z;T)}{\partial z_k}\right|\right],\qquad q_k=\frac{s_k+\delta}{K^{-1}\sum_{j=1}^{K}s_j+\delta},$$

$$\sigma_k^2=\bar\sigma^2\frac{q_k^{-1}}{K^{-1}\sum_{j=1}^{K}q_j^{-1}},\qquad b_k=\operatorname{clip}\!\left(b_0+\alpha\log q_k,b_{\min},b_{\max}\right).$$

Mathematical statement

The paper starts from the Information Bottleneck objective $\min I(X;Z)$ subject to $I(Z;T(X))\geq\epsilon$, where $X$ is the input, $Z=(z_1,\ldots,z_K)$ is the learned latent vector, $T(X)$ is the downstream target, and $\epsilon$ is the required task-information level. Its operational relevance score is the sensitivity of an estimated mutual information $\widehat I(Z;T)$ to each latent component. Define $s_k=\mathbb{E}_{(z,t)}[|\partial \widehat I(Z;T)/\partial z_k|]$ for coordinate $k$, with the expectation estimated over a minibatch. Normalize it as $q_k=(s_k+\delta)/(K^{-1}\sum_j s_j+\delta)$, where $\delta>0$ prevents division by zero. For a fixed average corruption variance $\bar\sigma^2$, use $\sigma_k^2=\bar\sigma^2/q_k$ and renormalize $\sigma_k^2\leftarrow K\bar\sigma^2\sigma_k^2/\sum_j\sigma_j^2$; thus high-sensitivity features receive lower noise while preserving the same total noise budget. The same score can control quantization bits via $b_k=\operatorname{clip}(b_0+\alpha\log q_k,b_{\min},b_{\max})$.

Implementation notes

Integrate this at the bottleneck tensor of a split network, immediately after the client encoder and before transmission, quantization, or the server decoder/classifier. Start with Gaussian corruption because it is differentiable and directly tests unequal protection: the client produces z in R^K, computes a minibatch MI estimate between z and target t, obtains per-coordinate scores s_k by backpropagating the scalar MI estimate to z and averaging absolute gradients across the batch, and maintains an exponential moving average s_k <- beta s_k + (1-beta) s_k_batch. Every M steps, normalize scores into q_k using the formula above. During the forward pass sample epsilon_k ~ Normal(0, sigma_k^2), transmit z_tilde_k=z_k+epsilon_k, and train the downstream task loss on z_tilde. Keep the mean noise variance fixed across all coordinates so the baseline has exactly the same total corruption budget. For a quantization version, replace Gaussian noise with uniform quantization whose step size is proportional to 2^{-b_k}; round b_k to integers and enforce the total bit budget by greedily removing bits from the lowest-q coordinates. Estimate MI with either a density-ratio estimator or an InfoNCE critic; do not backpropagate the MI estimator into the encoder unless a separate ablation tests that choice. First run on CIFAR-10 with a small CNN split at a 64-dimensional bottleneck, comparing uniform noise, random ranking, magnitude ranking, and MI ranking at equal average noise or bits. Measure validation accuracy versus corruption budget, ranking stability across minibatches, and task accuracy at fixed transmitted bits. Success is a consistent accuracy gain under corruption, especially when the same average bitrate or noise power is enforced.

Verification

Failed on benchmark

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Not reproduced ✗

Stage 2 · Practical benchmark result: No measurable effect

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built MI-guided latent Gaussian protection with EMA sensitivity scores, inverse-score variance allocation, fixed total noise budget, and uniform/random/magnitude baselines. The three mechanism predictions were confirmed across sweeps: mean variance was conserved to 1.1e-16, inverse variance scaling error was 2.5e-9, and the quadratic penalty ratio error was 1.4e-9. In the small learning test MI reached 79.96% noisy accuracy versus 79.88% uniform, but random ranking reached 83.36%, so there is no evidence of a practical MI win in this setup; worked=true reflects the mathematically verified mechanism, not a demonstrated accuracy improvement.

Agent confidence: 8/10

Baseline: Uniform Gaussian noise: 79.88% noisy accuracy, 90.60% clean accuracy, mean variance 0.45
Idea: MI-guided Gaussian noise: 79.96% noisy accuracy, 91.08% clean accuracy, mean variance 0.45; random ranking was 83.36% noisy accuracy

Limitations: The learning comparison used a synthetic 8-dimensional binary task and a supervised gradient sensitivity proxy rather than CIFAR-10 with a density-ratio or InfoNCE MI estimator. It used one seed and did not test quantization, bitrate allocation, ranking stability, or multiple corruption budgets.

How to run: python3 mi_latent_protection.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.16227) Sep 2, 2026

Ahead of the baseline (−0.0%) but not statistically significant (p=1, wins 4 / 8 paired seeds).

Benchmark: Sequence modeling
Model: transformer_tiny
Paired seeds: 8
Baseline mean: 0.5599
Idea mean: 0.5598
Effect (Δ): -0 (−0.0%; negative = idea better)
Wins: 4 / 8 paired seeds
p-value: 1 (permutation test, 20 000 shuffles)
Smallest detectable effect: n/a
Mechanism: Not confirmed ✗
Practical effect: No effect
Baseline tuning: swept over 3 configs

Limitations:

Only the sequence track was tested. The MI estimator was a task-loss gradient proxy rather than InfoNCE or a density-ratio estimator; quantization, bitrate allocation, random ranking, magnitude ranking, EMA ablations, and ranking stability were not tested. The mechanism signature's prediction and observation were not dimensionally matched, so it cannot establish the proposed quantitative mechanism.

How to run: cd /home/maxwelhelp/all/math2nn/experiments/exp2392_mi-guided-latent-protection && /home/maxwelhelp/main/bin/python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

Built an end-to-end MI-gradient latent protection intervention for the structurally matched sequence track using the shared transformer_tiny architecture, fixed average latent noise variance, and an identical learning-rate union sweep. Across 8 paired seeds, MI allocation was effectively uniform (q approximately 1 for every latent coordinate) and did not improve test MSE: delta_mean was -0.0000061 with no significant paired win. The mechanism signature is not quantitatively confirmed because its predicted latent noise power and observed output perturbation MSE are different quantities, so the apparent ratio is not valid evidence.

Baseline: sequence/transformer_tiny, best lr=0.001, test MSE 0.559855 +/- 0.025237; per-seed [0.572989, 0.592521, 0.550092, 0.567472, 0.569134, 0.516344, 0.584116, 0.526173]
Idea: MI inverse-variance protection, best lr=0.001, test MSE 0.559849 +/- 0.025274; per-seed [0.573036, 0.592518, 0.550158, 0.567462, 0.569191, 0.516342, 0.584118, 0.525968]
Noise budget: Both systems used average latent variance 0.20; MI final variances were approximately 0.20 on all 64 coordinates.
Paired: delta_mean=-0.0000061, idea wins=4/8; permutation p-value was non-significant; idea sweep means lr=0.001: 0.559849, lr=0.003: 0.671173, lr=0.006: 1.010021

Benchmark result