{
 "artifacts": null,
 "category": "memory",
 "description": "Construct each hierarchy transition as a coarse-graining map together with a local decoder that recovers the retained local representation. This creates a reversible-information path for features needed by later computation while still allowing high-frequency or locally irrelevant components to be discarded.",
 "formulas_latex": [
  "$$\\sigma=(\\operatorname{id}_{X}\\otimes\\Gamma)(\\omega)$$",
  "$$\\left[(\\mathcal{D}\\circ R)\\otimes\\operatorname{id}\\right](\\sigma)=(\\operatorname{id}_{X}\\otimes\\Gamma)\\left[(\\mathcal{D}\\circ R)\\otimes\\operatorname{id}\\right](\\omega)=\\sigma$$",
  "$$\\mathcal{L}_{\\mathrm{rev}}=\\frac{1}{B}\\sum_{i=1}^{B}\\left\\|D_{\\theta,r}\\left(E_{\\theta,r}(x_{i,r})\\right)-P_rx_{i,r}\\right\\|_2^2$$",
  "$$h^{(r+1)}=E_r(h^{(r)}),\\qquad \\widetilde h^{(r)}=D_r(h^{(r+1)}),\\qquad \\mathcal{L}=\\mathcal{L}_{\\mathrm{task}}+\\lambda\\sum_r\\|\\widetilde h^{(r)}-P_rh^{(r)}\\|_2^2$$"
 ],
 "id": 3041,
 "implementation": "Modify a hierarchical Transformer, U-Net, or multiscale CNN at every downsampling stage. Split the activation $h^{(r)}$ into local blocks $x_j$, apply a strided local encoder $E_r$ to produce coarse tokens $h^{(r+1)}_j$, and attach a decoder $D_r$ that reconstructs either the original block or a projected version $P_rx_j$. Use the objective $\\mathcal{L}=\\mathcal{L}_{task}+\\lambda\\sum_r\\|D_r(E_r(x_j))-P_rx_j\\|_2^2$, where $P_r$ is a fixed low-pass convolution, channel projection, or identity map. Keep $D_r$ strictly local: it may read only the corresponding coarse token and positional information, not neighboring blocks. To test the paper's outside-insensitivity property, perturb, drop, or randomly replace all blocks outside $j$ before decoding and verify that reconstruction of block $j$ is unchanged. For memory reduction, checkpoint only coarse activations and recompute local encoders during backward; an alternative is an additive coupling block whose inverse is exact. The exact identity and locality come from the paper, while reconstruction error measures the approximation in the learned network. First run on CIFAR-100 with a 12-layer hierarchical ViT, comparing standard pooling, pooling plus a local autoencoder, and the proposed projected reversible pooling at matched parameter count. Record peak activation memory, examples per second, validation accuracy, reconstruction error under outside perturbations, and gradient-norm variance. Success is at least 25% lower peak memory with no validation-accuracy loss and stable gradients at depth.",
 "math_summary": "The paper considers a retained gate $R$ supported on region $X$, a recovery channel $\\mathcal{D}$, a state $\\omega$, and an outside channel $\\Gamma$ acting only on $\\bar X$. If $R$ is exactly recovered, then applying $\\mathcal{D}\\circ R$ to $\\omega$ returns $\\omega$. For $\\sigma=(\\operatorname{id}_{X}\\otimes\\Gamma)(\\omega)$, the paper proves $[(\\mathcal{D}\\circ R)\\otimes\\operatorname{id}](\\sigma)=\\sigma$. Here $\\operatorname{id}$ is the identity channel, $\\otimes$ denotes independent subsystem action, and $\\bar X$ is the complement of $X$. The transferable property is locality of recovery: processing or removing gates outside the support $X$ does not affect recovery inside $X$. A neural analogue uses an encoder $E_r$ and local decoder $D_r$ with $D_r\\circ E_r\\approx P_r$, where $P_r$ is either the identity or a low-dimensional projection retaining task-relevant local information.",
 "math_tags": [
  "information-theory",
  "dynamical-systems",
  "optimization"
 ],
 "ml_areas": [
  "transformer",
  "memory",
  "training-dynamics",
  "cnn"
 ],
 "paper": {
  "arxiv_id": "2609.02141",
  "arxiv_url": "https://arxiv.org/abs/2609.02141",
  "summary_what_math_gives_to_ml": "The paper supplies a useful design principle for hierarchical neural representations: coarse-graining should retain variables that mediate long-range dependence, while testing whether discarded within-block information remains predictive of distant features. Its majority-spin example is a concrete sufficient-statistic construction, and its counterexample shows that a locally plausible pooling rule can destroy global conditional information even when pairwise mutual information is small. The most promising neural translation is a learned hierarchical pooling layer trained with a conditional-information preservation objective, optionally paired with a locally reversible decoder to reduce activation memory.",
  "title": "Renormalization group and long-range conditional mutual information in hierarchical models",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 5,
  "usefulness": 6
 },
 "solves": [
  "memory",
  "stability",
  "accuracy"
 ],
 "title": "Locally reversible hierarchical block",
 "url": "https://synthcore.org/idea/3041/locally-reversible-hierarchical-block",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}