{
 "artifacts": null,
 "category": "dynamics",
 "description": "Treat the correlation decay exponent and Hermite rank as explicit hyperparameters controlling the roughness of the depth-wise residual trajectory. Use smoother long-memory drivers for stable deep propagation and less correlated drivers when optimization needs more layer-wise diversity, while retaining critical scaling so the network does not collapse to the identity.",
 "formulas_latex": [
  "$$\\phi(x)=\\sum_{r=q}^{\\infty}a_r\\,\\mathrm{He}_r(x),\\qquad a_q\\neq0,\\quad a_1=\\cdots=a_{q-1}=0.$$",
  "$$\\rho(k)\\sim c k^{-\\alpha},\\qquad H=1-\\frac{\\alpha q}{2},\\qquad \\lambda_L=L^{-H}.$$",
  "$$\\gamma\u003eH\\Rightarrow \\lambda_L\\sum_{l\u003cL}(w_l-\\mathbb{E}w_l)\\to0,\\qquad \\gamma=H\\Rightarrow O(1),\\qquad \\gamma\u003cH\\Rightarrow \\text{growth with }L.$$",
  "$$\\widehat H=\\frac{1}{2}\\frac{\\log \\widehat{\\operatorname{Var}}\\left(\\sum_{l=0}^{m-1}(w_l-\\bar w)\\right)-\\log \\widehat{\\operatorname{Var}}\\left(\\sum_{l=0}^{m/2-1}(w_l-\\bar w)\\right)}{\\log m-\\log(m/2)}.$$"
 ],
 "id": 3113,
 "implementation": "Integrate this as a depth-wise parameter generator for residual blocks, with candidate H selected before training. Begin with q=1 because it is easiest to implement and corresponds to fractional Brownian motion. For each H in {0.5, 0.6, 0.7, 0.8}, choose alpha=2(1-H), generate a stationary Gaussian vector g[0:L] with correlation approximately c*k^{-alpha}, standardize it, and assign W_l=sigma_W*g_l to scalar gates or channel-wise low-rank gates. To test Hermite rank separately, use w_l=He_q(g_l)/sqrt(q!) for q=1 or q=2, standardize empirically, and choose alpha=2(1-H)/q so the same H is obtained. Keep the marginal variance of W_l fixed across settings and use lambda=L^{-H}. Pseudocode: for each H,q, set alpha=2*(1-H)/q; g=sample_stationary_gaussian(alpha,L); z=hermite_q(g); z=(z-mean(z))/std(z); gate[l]=sigma_gate*z[l]; lambda=L**(-H); output[l]=input[l]+lambda*gate[l]*F_l(input[l]). Estimate achieved H with the displayed block-sum log-log estimator and independently estimate covariance decay; these are diagnostics, not learned quantities. First experiment: train 20-layer and 100-layer residual MLPs on MNIST or CIFAR-10 using iid, H=0.5, H=0.7, and H=0.8 gates with three seeds and matched FLOPs. Measure early loss slope, gradient-noise variance, activation drift, final accuracy, and learning-rate sensitivity. The hypothesis is that an intermediate H improves deep optimization or accuracy while keeping activations bounded; it is falsified if H has no reproducible effect after variance matching or if larger H consistently worsens trainability.",
 "math_summary": "Let g_l be a stationary Gaussian sequence with long-range correlation \\rho(k)\\sim c k^{-\\alpha}, and let w_l=\\phi(g_l). Hermite rank q means that the Hermite expansion \\phi(x)=\\sum_{r\\ge q}a_r\\,\\mathrm{He}_r(x) has a_q\\ne0 and a_1=\\cdots=a_{q-1}=0. The effective memory exponent is H=1-\\alpha q/2. For q=1, normalized partial sums converge to fractional Brownian motion; for q\u003e1, they converge to a higher-order Hermite process. Larger H produces more persistent and smoother accumulation across depth, while H=1/2 is the iid/Brownian reference. The critical residual scale is \\lambda_L=L^{-H}. If \\lambda_L=L^{-\\gamma} with \\gamma\u003eH, the accumulated perturbation vanishes and initialization becomes identity-like; if \\gamma=H it remains O(1); if \\gamma\u003cH it grows with depth. This provides a measurable roughness and stability knob while marginal parameter variance is held fixed.",
 "math_tags": [
  "stochastic-processes",
  "probability",
  "dynamical-systems",
  "statistics"
 ],
 "ml_areas": [
  "resnet",
  "initialization",
  "training-dynamics",
  "regularization"
 ],
 "paper": {
  "arxiv_id": "2609.03589",
  "arxiv_url": "https://arxiv.org/abs/2609.03589",
  "summary_what_math_gives_to_ml": "The paper identifies a concrete depth-dependent initialization law for residual networks with long-range correlated layer weights: if layer correlations decay as k^{-\\alpha} and the feature map has Hermite rank q, the accumulated residual noise scales like L^H with H=1-\\alpha q/2, so the nontrivial regime uses \\lambda_L\\asymp L^{-H}. This gives a principled continuum between Brownian-like independent initialization and smoother, strongly correlated depth dynamics, rather than choosing the usual 1/L or 1/\\sqrt{L} scaling heuristically. The most transferable asset is a generator for correlated parameters together with a scaling rule that preserves an O(1) feature evolution as depth changes. A practical first test is to compare ordinary iid residual initialization against fractional-Gaussian or Hermite-transformed layer noise at equal depth, parameter count, and marginal activation variance.",
  "title": "Correlated initialization of deep residual networks",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 8,
  "usefulness": 6
 },
 "solves": [
  "stability",
  "generalization",
  "accuracy"
 ],
 "title": "Correlation-controlled residual roughness",
 "url": "https://synthcore.org/idea/3113/correlation-controlled-residual-roughness",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}