{
 "artifacts": [
  {
   "name": "bench_critical_sharing.py",
   "url": "https://synthcore.org/code/1048/bench_critical_sharing.py"
  },
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1048/bench_report.json"
  },
  {
   "name": "critical_sharing_experiment.py",
   "url": "https://synthcore.org/code/1048/critical_sharing_experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1048/report.md"
  },
  {
   "name": "report_bench_2026-09-01T194105.md",
   "url": "https://synthcore.org/code/1048/report_bench_2026-09-01T194105.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1048/results.json"
  }
 ],
 "category": "architecture",
 "description": "Construct deep or recurrent networks whose layer weights are correlated across depth with a prescribed power-law covariance, rather than either fully tying or fully independently sampling layers. The paper predicts two usable design boundaries: \\(\\gamma=1/2\\) for divergence of correlation-induced fourth moments and \\(\\gamma=1\\) for loss of summable-correlation flatness.",
 "download_zip": "https://synthcore.org/download/1048",
 "formulas_latex": [
  "$$\\operatorname{Cov}\\!\\left(W^{(\\ell)}_{ij},W^{(m)}_{ik}\\right)=\\frac{\\sigma^2}{d}\\,\\delta_{jk}\\,c_{|\\ell-m|},\\qquad c_t\\asymp t^{-\\gamma},$$",
  "$$\\sum_{t=1}^{L}c_t^2\\asymp\\begin{cases}L^{1-2\\gamma},\u0026\\gamma\u003c\\tfrac12,\\\\ \\log L,\u0026\\gamma=\\tfrac12,\\\\ 1,\u0026\\gamma\u003e\\tfrac12,\\end{cases}$$",
  "$$\\sum_{t=1}^{L}|c_t|\\asymp\\begin{cases}L^{1-\\gamma},\u0026\\gamma\u003c1,\\\\ \\log L,\u0026\\gamma=1,\\\\ 1,\u0026\\gamma\u003e1.\\end{cases}$$"
 ],
 "id": 2716,
 "implementation": "1. Integration point: implement a depth-indexed correlated parameterization for a residual MLP, Transformer block stack, or linear state-space model. Sample a latent Gaussian process \\(g_{i,j}^{(\\ell)}\\) over layer index \\(\\ell\\) for each matrix coordinate, and use it directly as the layer weight or combine it with iid noise: \\(W^{(\\ell)}=\\sqrt{\\alpha}\\,G^{(\\ell)}+\\sqrt{1-\\alpha}\\,E^{(\\ell)}\\). Use circulant embedding or a Cholesky factor of the \\(L\\times L\\) covariance matrix to sample the power-law process. 2. Pseudocode: choose \\(\\gamma\\), construct \\(C_{\\ell m}=c_{|\\ell-m|}\\), sample each coordinate's vector across depth from \\(\\mathcal N(0,C)\\), normalize every layer to identical Frobenius variance, and run the network. During training, log adjacent-layer covariance, the end-to-end Jacobian singular values, and residual-branch norms. 3. Taken from the paper: the two correlation accumulation thresholds and their different roles. Estimated empirically: whether the neural Jacobian or gradient covariance inherits these scalings; this inheritance is the falsifiable transfer hypothesis, not a theorem supplied by the paper. 4. First cheap experiment: use 24-layer residual MLPs of width 256 on CIFAR-10 or sequence-copying tasks, comparing iid layers, exact tying, AR(1) sharing, and power-law sharing with \\(\\gamma=0.3,0.5,0.75,1.0,1.25\\). Repeat at depths \\(L=12,24,48,96\\). The prediction is a kink in the depth scaling of the normalized gradient fourth moment near \\(\\gamma=1/2\\), approximately \\(L^{1-2\\gamma}\\) below it, and a change from growing to bounded cumulative layer-correlation near \\(\\gamma=1\\). If these signatures do not occur in Jacobians or gradient covariance, discard the transfer.",
 "math_summary": "Let \\(W^{(\\ell)}\\) denote the weight matrix at depth \\(\\ell\\), with independent rows but cross-layer covariance \\(\\operatorname{Cov}(W^{(\\ell)}_{ij},W^{(m)}_{ik})\\propto\\delta_{jk}c_{|\\ell-m|}\\), where \\(c_t\\asymp t^{-\\gamma}\\). The paper's long-range-correlation mechanism implies that the squared-correlation accumulation \\(\\sum_t c_t^2\\) changes scaling at \\(\\gamma=1/2\\), while the absolute-correlation accumulation \\(\\sum_t|c_t|\\) is finite only for \\(\\gamma\u003e1\\). Use \\(\\gamma\u003e1\\) for a conservative regime, \\(1/2\u003c\\gamma\\le1\\) for a controlled long-memory regime, and \\(\\gamma\\le1/2\\) as a deliberately stress-tested critical regime.",
 "math_tags": [
  "random-matrix",
  "spectral-theory",
  "statistics",
  "dynamical-systems",
  "linear-algebra"
 ],
 "ml_areas": [
  "transformer",
  "rnn",
  "ssm",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2608.23944",
  "arxiv_url": "https://arxiv.org/abs/2608.23944",
  "summary_what_math_gives_to_ml": "The paper provides a concrete mechanism by which temporal or coordinate-wise correlations in Gaussian random matrices deform the bulk spectrum: correlated entries sharing a row create a combinatorial hub contribution, while long-range correlations produce a fourth-moment transition at correlation exponent \\(\\gamma_c=1/2\\). A second threshold, \\(\\gamma=1\\), separates summable from nonsummable correlations and controls the flatness assumptions needed for stable matrix-Dyson edge behavior. These mechanisms can be transferred to neural networks by designing and stress-testing correlated weight initialization or cross-layer weight sharing, with explicit spectral and moment thresholds rather than relying only on validation accuracy.",
  "title": "Bulk Phase Transition and Edge Behavior in Temporally Correlated Random Matrices",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 8,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "memory",
  "scalability"
 ],
 "title": "Critical Cross-Layer Weight Sharing",
 "url": "https://synthcore.org/idea/2716/critical-cross-layer-weight-sharing",
 "verification": {
  "benchmark": {
   "track": "dynamics",
   "model": "rnn_small_step_gru",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 0,
    "idea_mean": 0.025829498656094074,
    "delta_mean": -0.00145388999953866,
    "per_seed_diffs": [],
    "idea_wins": 6,
    "n_pairs": 8,
    "p_value": 0.31345,
    "mde": 0.0032509812188714292,
    "mde_rel_pct": 11.915606451621084,
    "verdict": "no significant win",
    "system_worked": false
   },
   "baseline": {
    "best_cfg": {
     "lr": 0.005,
     "mode": "iid"
    },
    "sweep": [
     {
      "cfg": {
       "lr": 0.001,
       "mode": "iid"
      },
      "mean": 0.055274722166359425
     },
     {
      "cfg": {
       "lr": 0.002,
       "mode": "iid"
      },
      "mean": 0.03774398844689131
     },
     {
      "cfg": {
       "lr": 0.003,
       "mode": "iid"
      },
      "mean": 0.03395134303718805
     },
     {
      "cfg": {
       "lr": 0.005,
       "mode": "iid"
      },
      "mean": 0.027184664737433195
     }
    ],
    "full": {
     "mean": 0,
     "std": 0,
     "per_seed": null,
     "n": 0
    }
   },
   "idea": {
    "mean": 0.025829498656094074,
    "std": 0.004832996344710331,
    "per_seed": [
     0.024646440520882607,
     0.03045663796365261,
     0.021562719717621803,
     0.035005223006010056,
     0.02129276655614376,
     0.025303291156888008,
     0.019978370517492294,
     0.028390539810061455
    ],
    "n": 8
   },
   "mechanism_signature": {
    "confirmed": true,
    "gamma": 0.3,
    "mean_abs_corr_by_lag": [
     0.6810568388520706,
     0.5442689459568522,
     0.4600729069148357,
     0.39131814292561884,
     0.3391274167389965,
     0.295807543011521,
     0.255015405002512
    ],
    "observed_loglog_slope": -0.7032384334932155,
    "predicted_loglog_slope": -0.3,
    "quantity": "trained hidden-state cross-step correlation on dynamics test trajectories"
   },
   "custom_track": null
  },
  "mechanism_confirmed": true,
  "peer_reviewed": false,
  "practical_verdict": "no_effect",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 9,
   "verdict": "Implemented correlated power-law cross-layer Gaussian weights and a residual-MLP Jacobian experiment. For gamma=0.3, observed tail exponents were 0.701 for absolute-correlation accumulation and 0.409 for squared-correlation accumulation, versus predictions 0.7 and 0.4. Critical gamma=0.5 and gamma=1 sums showed logarithmic growth, and empirical fourth moments matched the exact Wick formula within 0.8%. The correlated network had worse depth-128 Jacobian stability than IID weights, so the mechanism manifested but no performance win was demonstrated.",
   "metrics": {
    "baseline": "IID depth-128 max Jacobian singular value 1.357; normalized gradient fourth-moment proxy 1.007.",
    "idea": "Power-law gamma=0.3 depth-128 max Jacobian singular value 5.328; normalized gradient fourth-moment proxy 1.229. S2 exponent 0.409 versus predicted 0.4; Wick relative error 0.16%-0.74%."
   },
   "how_to_run": "python3 critical_sharing_experiment.py",
   "files": [
    "critical_sharing_experiment.py",
    "results.json",
    "run_output.txt"
   ],
   "limitations": "Only a toy Gaussian-coordinate and small residual-MLP Jacobian experiment was tested; no CIFAR or sequence training, loss/FLOP comparison, tied or AR(1) baseline, learned mixing coefficient, or large-width/depth study was performed."
  },
  "status": "mech_ok_no_baseline",
  "status_label": "Mechanism confirmed, baseline not beaten",
  "updated_at": "2026-09-01T19:41:05",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}