{
 "artifacts": [
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1222/report.md"
  },
  {
   "name": "toy_experiment.py",
   "url": "https://synthcore.org/code/1222/toy_experiment.py"
  }
 ],
 "category": "architecture",
 "description": "Replace ordinary token merging or graph pooling with a learned block map whose output preserves information about a remote target conditioned on the surrounding coarse representation. The paper's majority-spin counterexample gives a concrete failure mode: two microscopic configurations mapped to the same pooled token can imply different predictions for distant variables.",
 "download_zip": "https://synthcore.org/download/1222",
 "formulas_latex": [
  "$$\\left(\\mathcal{E}_{j}\\right)^{s^{\\prime}}_{a,b,c}=\\delta_{s^{\\prime},\\operatorname{maj}(a,b,c)}$$",
  "$$I(L:R|s=+1)=I(\\operatorname{maj}(L):R|s=+1)$$",
  "$$P(s_L=+1|L=(+1,+1,+1))=\\frac{(1-p)^3}{(1-p)^3+p^3},\\qquad P(s_L=+1|L=(+1,+1,-1))=1-p$$",
  "$$\\widehat I(L:R\\mid S)=\\frac{1}{B}\\sum_{i=1}^{B}\\left[\\log q_{\\phi}(r_i\\mid l_i,s_i)-\\log q_{\\psi}(r_i\\mid s_i)\\right]$$"
 ],
 "id": 3040,
 "implementation": "Integrate this into a hierarchical Transformer token-merging layer or a graph-NN pooling layer. Partition tokens into fixed blocks of three, compute a pooled token $s_j=\\sum_{k=1}^{3}\\alpha_{jk}h_{jk}$ with learned softmax weights $\\alpha_{jk}$, and initialize the weights to uniform averaging. For each minibatch, sample a remote block $R$ separated from source block $L$ by at least one block. Form $S=\\operatorname{pool}(L)$ and train two small predictor heads: $q_\\phi(r\\mid L,S)$, which sees the microscopic source block and pooled token, and $q_\\psi(r\\mid S)$, which sees only the pooled token. Add $\\lambda\\max(0,\\widehat I-\\tau)$ to the task loss, with $\\widehat I=B^{-1}\\sum_i[\\log q_\\phi(r_i\\mid l_i,s_i)-\\log q_\\psi(r_i\\mid s_i)]$, information budget $\\tau$, and minibatch size $B$. The predictors should be optimized to estimate the gap; use stop-gradient or alternating updates when necessary so the pooling layer cannot minimize the term merely by making both predictors poor. For image features, let $R$ be a distant patch and predict its class or projected representation; for language, let $R$ be a future token span and use a small autoregressive auxiliary head. The paper-derived object is the conditional-information sufficiency criterion; the neural predictors empirically estimate it. First test on CIFAR-10 with a four-stage hierarchical ViT, comparing average pooling, learned attention pooling, and information-preserving pooling at equal token counts and FLOPs. Measure validation accuracy, accuracy after 4x token reduction, and the held-out predictor gap. Success is higher accuracy at the same sequence length and a lower remote-prediction gap than ordinary pooling, particularly under local patch corruption.",
 "math_summary": "The paper defines a local majority coarse-graining channel on a three-spin block by $\\left(\\mathcal{E}_{j}\\right)^{s^{\\prime}}_{a,b,c}=\\delta_{s^{\\prime},\\operatorname{maj}(a,b,c)}$, where $a,b,c\\in\\{-1,+1\\}$ are microscopic spins, $s'\\in\\{-1,+1\\}$ is the retained coarse spin, $\\delta$ is the Kronecker delta, and $\\operatorname{maj}$ is the sign of the sum. The key diagnostic is the conditional-mutual-information equality $I(L:R\\mid s=+1)=I(\\operatorname{maj}(L):R\\mid s=+1)$, where $L$ and $R$ are microscopic left and right triples and $s$ is a conditioning logical spin. Equality means the pooled statistic is sufficient for predicting $R$ given $s$; failure means pooling discarded remote-relevant information. The explicit failure is $P(s_L=+1\\mid L=(+1,+1,+1))=\\frac{(1-p)^3}{(1-p)^3+p^3}$ versus $P(s_L=+1\\mid L=(+1,+1,-1))=1-p$ for $0\u003cp\u003c1/2$, although both inputs have majority $+1$. For neural features, estimate the information gap with $\\widehat I(L:R\\mid S)=B^{-1}\\sum_i[\\log q_\\phi(r_i\\mid l_i,s_i)-\\log q_\\psi(r_i\\mid s_i)]$, where $S$ is the learned pooled token, $q_\\phi$ sees microscopic features and $S$, $q_\\psi$ sees only $S$, and $B$ is minibatch size.",
 "math_tags": [
  "information-theory",
  "probability",
  "optimization"
 ],
 "ml_areas": [
  "transformer",
  "graph-nn",
  "loss",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2609.02141",
  "arxiv_url": "https://arxiv.org/abs/2609.02141",
  "summary_what_math_gives_to_ml": "The paper supplies a useful design principle for hierarchical neural representations: coarse-graining should retain variables that mediate long-range dependence, while testing whether discarded within-block information remains predictive of distant features. Its majority-spin example is a concrete sufficient-statistic construction, and its counterexample shows that a locally plausible pooling rule can destroy global conditional information even when pairwise mutual information is small. The most promising neural translation is a learned hierarchical pooling layer trained with a conditional-information preservation objective, optionally paired with a locally reversible decoder to reduce activation memory.",
  "title": "Renormalization group and long-range conditional mutual information in hierarchical models",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 7
 },
 "solves": [
  "accuracy",
  "memory",
  "generalization"
 ],
 "title": "Conditional-information-preserving pooling",
 "url": "https://synthcore.org/idea/3040/conditional-information-preserving-pooling",
 "verification": {
  "peer_reviewed": false,
  "stage1_mechanism_check": {
   "worked": false,
   "confidence": 9,
   "verdict": "Built and ran a reproducible majority-spin conditional-information toy experiment plus a small learned scalar-pooling remote-prediction comparison. The exact check found I(L:R)=0.3043 bits versus I(majority(L):R)=0.2933 bits, with 0.0110 bits discarded and clearly different posteriors for two same-majority inputs. However, learned pooling did not beat mean pooling on held-out remote NLL or accuracy, so the proposed neural method has no demonstrated win here.",
   "metrics": {
    "baseline": "Mean pooling: held-out remote NLL 0.48705, accuracy 0.80450; majority pooling: NLL 0.49432, accuracy 0.80450; full-source predictor NLL 0.48756.",
    "idea": "Learned softmax scalar pooling: held-out remote NLL 0.48739, accuracy 0.80450, estimated NLL gap -0.00017 versus full source. Exact injective pooling retained 0.30425 bits, but the trained pooling weights remained near-uniform and did not improve over mean pooling."
   },
   "how_to_run": "python3 toy_experiment.py",
   "files": [
    "toy_experiment.py"
   ],
   "limitations": "Only a synthetic noisy-spin system was tested; no CIFAR-10, hierarchical Transformer, local corruption, task-loss training, explicit information-budget regularizer, multiple random seeds, or statistically repeated trials were run. The neural demo is a small auxiliary-prediction test rather than an end-to-end pooling benchmark."
  },
  "status": "mechanism_failed",
  "status_label": "Mechanism failed",
  "updated_at": "2026-09-03T12:39:01",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": false,
    "tested": true
   }
  }
 }
}