{
 "artifacts": null,
 "category": "regularization",
 "description": "Add a training-time diagnostic and optional regularizer that detects whether a multi-branch block has entered the paper's low-temperature, winner-take-all regime. Estimate concentration from actual branch log-gains and penalize extreme dominance when the observed system behaves as though beta is at least sqrt(2), preserving diverse paths instead of allowing one branch to determine the block Jacobian.",
 "formulas_latex": [
  "$$\\beta=\\sqrt{\\frac{2(N-1)}{n\\log m}},\\qquad \\beta_c=\\sqrt{2}.$$$",
  "$$\\log s_1(X)\\ \\text{has low-temperature contribution}\\ -\\frac{(\\beta-\\sqrt{2})^2\\log m}{4}\\left(1+o_{\\mathbb P}(1)+O_{\\mathbb P}\\!\\left(\\frac{\\log n}{\\beta^2\\log m}\\right)\\right),\\qquad \\beta\\ge\\sqrt{2}.$$$",
  "$$E_i=-\\log(\\|B_i h\\|_2+\\varepsilon),\\qquad p_i=\\frac{e^{-\\beta E_i}}{\\sum_{j=1}^{m}e^{-\\beta E_j}},\\qquad H(p)=-\\sum_{i=1}^{m}p_i\\log p_i.$$$",
  "$$\\mathcal L_{\\mathrm{conc}}=\\lambda\\,\\operatorname{ReLU}(\\widehat\\beta-\\sqrt2)^2\\left(\\log m-H(p)\\right).$$"
 ],
 "id": 257,
 "implementation": "Integrate this into each parallel-path module immediately after branch outputs B_i h are computed during training. Here h is the block input, B_i is branch i, and m is the number of branches. For every minibatch, compute g_i = log(||B_i h||_2 + epsilon), averaged over tokens and examples to reduce noise. Set E_i = -g_i and compute p_i = softmax(beta_hat times g)_i, entropy H(p) = -sum_i p_i log(p_i), and the top-branch share max_i p_i. Use the architecture estimate beta_arch = sqrt(2(N-1)/(n log(m))). Also form an empirical concentration estimate beta_hat = sqrt(2 Var_i(g_i)/(log(m) + epsilon)), with Var_i computed across branches and smoothed by an exponential moving average across minibatches. This estimator is empirical rather than a theorem. Add L_conc = lambda ReLU(beta_hat - sqrt(2))^2 (log(m) - H(p)) to the task loss, with lambda initially between 1e-4 and 1e-2; stop gradients through beta_hat if it is noisy. Log entropy, top-branch share, aggregate norm, and branch gradient cosine similarity. Test a 6-block 16-branch residual MLP on CIFAR-10 against no regularizer and branch dropout. Measure accuracy, seed variance, branch concentration, and Jacobian-norm drift. Success means lower top-branch share and more stable gradients without reducing validation accuracy.",
 "math_summary": "For X = (1/sqrt(m)) sum_i X_i, the paper predicts a transition at beta_c = sqrt(2), with beta = sqrt(2(N-1)/(n log(m))). Here N is multiplicative depth, n is width, and m is branch count. In the low-temperature regime, the extracted asymptotic includes the extreme-value contribution -((beta - sqrt(2))^2 log(m))/4, up to relative error terms, showing that behavior changes once beta exceeds the critical value. For a trained neural block, define branch energies E_i = -log(||B_i h||_2 + epsilon), temperature tau = 1/beta, Gibbs weights p_i = exp(-E_i/tau) / sum_j exp(-E_j/tau), and entropy H(p) = -sum_i p_i log(p_i). Low entropy is an empirical proxy for extreme-branch domination. The proposed regularizer penalizes concentration only when an estimated beta is above the paper's threshold.",
 "math_tags": [
  "random-matrix",
  "probability",
  "statistical-mechanics",
  "spectral-theory"
 ],
 "ml_areas": [
  "regularization",
  "mlp",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2607.04047",
  "arxiv_url": "https://arxiv.org/abs/2607.04047",
  "summary_what_math_gives_to_ml": "The paper identifies a random-energy-model phase transition governing the largest singular value of a sum of deep random matrix products. The controlling parameter is not simply the depth-to-width ratio, but the effective inverse temperature beta = sqrt(2(N-1)/(n log m)), which compares path depth with the logarithm of the number of summed branches. This gives a concrete design rule for deep multi-branch networks: keep beta below the critical value sqrt(2) when stable aggregation is desired, because above it the largest path can dominate the entire sum. The same criterion can become an online diagnostic or regularizer for activation and gradient concentration in residual, multi-branch, and mixture-of-experts-like networks.",
  "title": "Top Singular Value in Sum-Products of Random Matrices",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 6
 },
 "solves": [
  "stability",
  "generalization",
  "accuracy"
 ],
 "title": "Extreme-Branch Concentration Monitor",
 "url": "https://synthcore.org/idea/257/extreme-branch-concentration-monitor",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}