{
 "artifacts": null,
 "category": "architecture",
 "description": "Replace part of a narrow ReLU MLP or residual branch with an explicit coordinatewise fold that computes absolute value in one residual block. The fold introduces controlled non-injectivity without increasing representation width, potentially allowing width-constrained networks to represent multimodal or topologically folded decision boundaries that monotone feedforward layers represent inefficiently.",
 "formulas_latex": [
  "$$|x|=x+2\\operatorname{ReLU}(-x)$$",
  "$$F(x)=x+G(x),\\qquad G(x)=2\\operatorname{ReLU}(-x)\\quad\\Longrightarrow\\quad F(x)=|x|$$",
  "$$F_c(x)=c+|x-c|=x+2\\operatorname{ReLU}(c-x),\\qquad x,c\\in\\mathbb{R}^{d}$$"
 ],
 "id": 287,
 "implementation": "Integrate this at the residual-block level of a width-constrained MLP, small CNN bottleneck, or tokenwise transformer feed-forward sublayer. The input tensor is $X\\in\\mathbb{R}^{B\\times T\\times d}$; apply the fold independently to the final representation dimension, optionally after LayerNorm. Use either a learned center $c\\in\\mathbb{R}^{d}$ shared across tokens or a center generated by a small linear projection from the block input. A minimal block is: (1) $U=X-c$; (2) $V=\\operatorname{ReLU}(-U)$; (3) $Y=X+2V$, which is exactly $c+|X-c|$. For a more flexible residual block, use $Y=X+\\gamma W_o\\operatorname{ReLU}(W_i(X-c)+b_i)+b_o$, initialize $W_i=-I$, $W_o=2I$, and $\\gamma=1$, then allow fine-tuning while retaining the identity skip connection. Pseudocode is `U = X - c; R = relu(-U); Y = X + 2*R`; followed by the usual output projection, normalization, and dropout. The mathematical component is the exact fold initialization and its non-injectivity; learned deviations are empirical. Compare against a same-width plain ReLU MLP, a standard pre-activation ResNet block, and an invertible coupling block. First run a cheap synthetic test in $d=3$: classify two interleaved or folded curve/surface classes after projecting inputs through a width-3 bottleneck, then test CIFAR-10 with width-8 and width-16 MLP bottlenecks. Track accuracy at equal parameters, layers needed to reach a target accuracy, Jacobian rank-collapse frequency, and optimization loss. Success means the explicit fold reaches the same accuracy with fewer layers or improves fixed-width accuracy without an unstable increase in gradient norms.",
 "math_summary": "The constructive identity used by the paper is $|x|=x+2\\operatorname{ReLU}(-x)$. For a vector $x\\in\\mathbb{R}^{d}$, with coordinatewise ReLU, a residual map $F(x)=x+G(x)$ and residual branch $G(x)=2\\operatorname{ReLU}(-x)$ gives $F(x)=|x|$ coordinatewise. A translated fold around a learned or fixed center $c\\in\\mathbb{R}^{d}$ is $F_c(x)=c+|x-c|=x+2\\operatorname{ReLU}(c-x)$, where $c-x$ and the ReLU are coordinatewise. Repeated folds can create non-injective maps and can change linking configurations; this is impossible for a continuous invertible flow, whose layer maps are diffeomorphisms and preserve topological invariants. The transferable property is therefore an inexpensive, explicitly parameterized violation of injectivity, not merely the use of ReLU.",
 "math_tags": [
  "topology",
  "geometry",
  "dynamical-systems"
 ],
 "ml_areas": [
  "mlp",
  "architecture",
  "transformer"
 ],
 "paper": {
  "arxiv_id": "2606.31856",
  "arxiv_url": "https://arxiv.org/abs/2606.31856",
  "summary_what_math_gives_to_ml": "The paper supplies a concrete geometric criterion for distinguishing architectures: whether their layer maps can change the linking number of two disjoint embedded data manifolds in a three-dimensional representation space. The most transferable result is not the invariant itself, but the constructive observation that discrete residual blocks can synthesize folds such as absolute value, while continuous invertible flows cannot change linking number. This suggests narrow bottleneck architectures with explicit non-injective fold primitives, and layerwise topology diagnostics or regularizers for tasks whose classes require folding and unlinking.",
  "title": "Low-dimensional topology of deep neural networks",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 3,
  "novelty": 5,
  "usefulness": 7
 },
 "solves": [
  "accuracy",
  "scalability"
 ],
 "title": "Explicit fold residual block",
 "url": "https://synthcore.org/idea/287/explicit-fold-residual-block",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}