{
 "artifacts": [
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1216/bench_report.json"
  },
  {
   "name": "forest_dropout_experiment.py",
   "url": "https://synthcore.org/code/1216/forest_dropout_experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1216/report.md"
  },
  {
   "name": "report_bench_2026-09-03T121422.md",
   "url": "https://synthcore.org/code/1216/report_bench_2026-09-03T121422.md"
  },
  {
   "name": "stage2_forest_dropout.py",
   "url": "https://synthcore.org/code/1216/stage2_forest_dropout.py"
  }
 ],
 "category": "regularization",
 "description": "Replace independent feature, expert, or edge dropout by sampling subsets from a forest-induced strongly Rayleigh distribution. The distribution is multiaffine and negatively dependent, so selecting one computational route suppresses redundant correlated routes without the positive correlations created by independent or blockwise heuristics. Use the sampled subset as a structured mask during training and compare it against Bernoulli dropout at equal expected sparsity.",
 "download_zip": "https://synthcore.org/download/1216",
 "formulas_latex": [
  "$$p(z)=\\sum_{F\\in\\mathcal F}w(F)\\prod_{e\\in F}z_e,\\qquad w(F)=\\prod_{e\\in F}c_e,\\quad c_e\u003e0.$$",
  "$$\\Pr(S)=\\frac{w(S)}{p(\\mathbf 1)},\\qquad \\Pr(A\\cap B)\\leq \\Pr(A)\\Pr(B)\\quad\\text{for disjoint increasing events }A,B.$$",
  "$$Z(A)Z(B)\\geq Z(A\\cup B)Z(A\\cap B).$$"
 ],
 "id": 3020,
 "implementation": "Integrate this at the mask-generation step of a sparse MoE layer, graph neural network, or structured channel-pruning module. First choose a small dependency graph whose edges represent candidate routes; for an MoE, use vertices as experts and edges as admissible co-selection links, while for a GNN use physical graph edges directly. Maintain positive logits theta_e and set c_e=softplus(theta_e)+epsilon. At each training step, construct the multiaffine generating polynomial p or its determinantal representation from the corresponding weighted Laplacian. For a small MVP, enumerate all admissible forests S, compute w(S)=product(c_e), normalize the weights, and sample one subset with the desired cardinality. For larger graphs, use a fixed-size strongly Rayleigh sampler based on sequential conditional probabilities: at each step estimate the marginal of every remaining edge by evaluating p with that variable set to one versus zero, sample one edge, then condition the polynomial by deleting or contracting the chosen edge. Apply the binary mask to expert outputs, channels, or message edges and rescale by empirical inclusion probabilities. The mathematical quantities are the positive weights, forest polynomial, and negative-dependence law; sampling approximations and cardinality control are engineering choices. Start with a 4- or 8-expert MoE on CIFAR-100 or a small Transformer on WikiText-2, matching Bernoulli dropout's expected active experts and FLOPs. Measure validation loss, duplicate expert co-activation, gradient variance, and load balance. Success means equal-or-better accuracy with lower co-activation correlation and no increase in training instability; a useful target is a 1-2 point validation improvement or the same accuracy at 20% fewer active routes.",
 "math_summary": "For a graph G=(V,E) with positive edge weights c_e, let F be the relevant spanning-forest family and w(F)=product_{e in F} c_e. The paper's unsigned principal forest numerators assemble into a multiaffine real-stable polynomial p(z)=sum_{F in F} w(F) product_{e in F} z_e, where z_e is an indeterminate for edge e. Normalizing at z_e=1 gives the subset law Pr(S)=w(S)/p(1), with S the selected edge or feature coordinates. Real stability implies the law is strongly Rayleigh, hence negatively associated: for disjoint increasing events A and B, Pr(A intersection B) is at most Pr(A)Pr(B). The paper also gives the Hadamard-Fischer/log-submodular inequality Z(A)Z(B) \u003e= Z(A union B)Z(A intersection B), where Z(A) is the unsigned numerator associated with coordinate set A. In a neural network, map graph edges to candidate experts, attention heads, channels, or message-passing routes, and use c_e=exp(theta_e) or a positive softplus parameter.",
 "math_tags": [
  "graph-theory",
  "combinatorics",
  "probability",
  "linear-algebra"
 ],
 "ml_areas": [
  "moe-routing",
  "graph-nn",
  "regularization",
  "pruning"
 ],
 "paper": {
  "arxiv_id": "2609.01672",
  "arxiv_url": "https://arxiv.org/abs/2609.01672",
  "summary_what_math_gives_to_ml": "The paper provides an unusually concrete bridge between electrical-network determinants, graphic incidence, and negatively dependent subset measures. Its most transferable asset is that arbitrary determinant minors can be expanded over spanning forests with coefficients in exactly {0, ±1}, while the associated unsigned principal numerators form a real-stable multiaffine polynomial and therefore a strongly Rayleigh distribution. This suggests structured dropout or pruning schemes that preserve diversity through negative dependence rather than independently deleting features, as well as determinant-based regularizers for graph-structured neural modules. The constructions are most practical on small or sparse dependency graphs, where matrix-tree or Laplacian determinant evaluations replace explicit forest enumeration.",
  "title": "All-Minors Matrix-Tree Theory for Superport Networks: Completed Quotient-Incidence Determinants and Conductance-Weighted Subdivision Extensions",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 7,
  "novelty": 8,
  "usefulness": 7
 },
 "solves": [
  "generalization",
  "accuracy",
  "stability"
 ],
 "title": "Strongly-Rayleigh Forest Dropout",
 "url": "https://synthcore.org/idea/3020/strongly-rayleigh-forest-dropout",
 "verification": {
  "benchmark": {
   "track": "tabular",
   "model": "mlp_tiny",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 25.43371081352234,
    "idea_mean": 19.25827932357788,
    "delta_mean": -6.175431489944458,
    "per_seed_diffs": [
     -5.344308853149414,
     -8.419403076171875,
     -4.393280029296875,
     -5.037660598754883,
     -5.378490447998047,
     -8.562921524047852,
     -5.9797821044921875,
     -6.287605285644531
    ],
    "idea_wins": 8,
    "n_pairs": 8,
    "p_value": 0.0081,
    "mde": 1.2869647545735994,
    "mde_rel_pct": 5.060074654498938,
    "verdict": "idea better (significant)",
    "system_worked": true
   },
   "baseline": {
    "best_cfg": {
     "epochs": 12,
     "lr": 0.0015,
     "p": 0.5
    },
    "sweep": [
     {
      "cfg": {
       "epochs": 12,
       "lr": 0.0015,
       "p": 0.3
      },
      "mean": 39.63210868835449
     },
     {
      "cfg": {
       "epochs": 12,
       "lr": 0.0015,
       "p": 0.4
      },
      "mean": 31.991859912872314
     },
     {
      "cfg": {
       "epochs": 12,
       "lr": 0.0015,
       "p": 0.5
      },
      "mean": 24.693001747131348
     },
     {
      "cfg": {
       "epochs": 12,
       "lr": 0.003,
       "p": 0.3
      },
      "mean": 41.53972053527832
     },
     {
      "cfg": {
       "epochs": 12,
       "lr": 0.003,
       "p": 0.4
      },
      "mean": 35.30523777038574
     },
     {
      "cfg": {
       "epochs": 12,
       "lr": 0.003,
       "p": 0.5
      },
      "mean": 27.46730661392212
     },
     {
      "cfg": {
       "epochs": 12,
       "lr": 0.006,
       "p": 0.3
      },
      "mean": 47.46395492553711
     },
     {
      "cfg": {
       "epochs": 12,
       "lr": 0.006,
       "p": 0.4
      },
      "mean": 42.76505661010742
     },
     {
      "cfg": {
       "epochs": 12,
       "lr": 0.006,
       "p": 0.5
      },
      "mean": 34.259535789489746
     }
    ],
    "full": {
     "mean": 25.43371081352234,
     "std": 2.830439554726994,
     "per_seed": [
      22.917491912841797,
      27.197723388671875,
      24.012426376953126,
      24.644365310668945,
      23.099069595336914,
      32.163185119628906,
      24.368993759155273,
      25.066431045532227
     ],
     "n": 8
    }
   },
   "idea": {
    "mean": 19.25827932357788,
    "std": 1.7861984392123207,
    "per_seed": [
     17.573183059692383,
     18.7783203125,
     19.6191463470459,
     19.606704711914062,
     17.720579147338867,
     23.600263595581055,
     18.389211654663086,
     18.778825759887695
    ],
    "n": 8
   },
   "mechanism_signature": {
    "confirmed": true,
    "observed_max_pair_covariance": 8.23993651088983e-18,
    "observed_mean_inclusion": 0.40000000000000024,
    "predicted_max_pair_covariance": 8.23993651088983e-18,
    "predicted_mean_inclusion": 0.4,
    "trained_model_output_mask_variance": 4.495115280151367
   },
   "custom_track": null
  },
  "mechanism_confirmed": true,
  "peer_reviewed": false,
  "practical_verdict": "helps",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 7,
   "verdict": "Built an exact K4 spanning-tree forest sampler and a small masked-regression comparison against Bernoulli dropout at equal expected active features (3 of 6). The math check reproduced multiaffinity, zero log-submodularity violations, and nonpositive pairwise covariances (minimum -0.0625); the forest model also had lower mean validation MSE (0.2283 vs 0.2589 across 3 seeds) and lower run-to-run variation. This is a promising small-scale signal, not conclusive evidence of a general neural-network win.",
   "metrics": {
    "baseline": "Bernoulli p=0.5: validation MSE 0.2589 +/- 0.0831 over 3 runs; expected 3 active features",
    "idea": "Uniform spanning-tree forest mask: validation MSE 0.2283 +/- 0.0239 over 3 runs; exactly 3 active features; minimum measured pairwise inclusion covariance -0.0625"
   },
   "how_to_run": "python3 forest_dropout_experiment.py",
   "files": [
    "forest_dropout_experiment.py"
   ],
   "limitations": "Only a tiny synthetic regression task, K4 uniform forests, 6 input routes, 500 optimization steps, and 3 random seeds were tested. No learned positive edge weights, larger graphs, MoE/GNN architecture, FLOP benchmark, gradient-variance measurement, or direct empirical co-activation statistic was included."
  },
  "status": "beats_baseline",
  "status_label": "Beats tuned baseline",
  "updated_at": "2026-09-03T12:14:22",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": true,
    "tested": true
   },
   "practical_benchmark": {
    "beats_baseline": true,
    "tested": true,
    "verdict": "helps"
   },
   "toy_mechanism_gate": {
    "confirmed": true,
    "tested": true
   }
  }
 }
}