{
 "artifacts": [
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1086/bench_report.json"
  },
  {
   "name": "dissipative_softmax.py",
   "url": "https://synthcore.org/code/1086/dissipative_softmax.py"
  },
  {
   "name": "experiment.py",
   "url": "https://synthcore.org/code/1086/experiment.py"
  },
  {
   "name": "official_bench.py",
   "url": "https://synthcore.org/code/1086/official_bench.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1086/report.md"
  },
  {
   "name": "report_bench_2026-09-02T001547.md",
   "url": "https://synthcore.org/code/1086/report_bench_2026-09-02T001547.md"
  },
  {
   "name": "stage2_local_bench.py",
   "url": "https://synthcore.org/code/1086/stage2_local_bench.py"
  },
  {
   "name": "token_expert_sequence_snapshot.py",
   "url": "https://synthcore.org/code/1086/token_expert_sequence_snapshot.py"
  }
 ],
 "category": "architecture",
 "description": "Add a finite-state stochastic latent layer with conditional states i=1,...,K and an auxiliary reset state 0. The network predicts thermodynamic logits X_i, while transition rates are constructed so that the conditional stationary distribution approaches p_i=exp(X_i)/Z_C under rapid reset, even though the full latent graph retains directed probability currents. This creates a calibrated stochastic layer with controllable mixing and a separate mechanism for maintaining exploration.",
 "download_zip": "https://synthcore.org/download/1086",
 "formulas_latex": [
  "$$X_i=\\beta\\left(\\Delta\\mu-\\mathcal{F}^{\\mathrm{cost}}_i\\right),\\qquad Z_{\\mathcal C}=1+\\sum_{i=1}^{K}e^{X_i},\\qquad p_i=\\frac{e^{X_i}}{Z_{\\mathcal C}}.$$",
  "$$P_i=(1-P_0)p_i,\\qquad \\lim_{r/k\\to\\infty}P_i=p_i.$$",
  "$$\\log\\frac{k_{ij}}{k_{ji}}=X_j-X_i,\\qquad \\sum_{(i,j)\\in c}\\log\\frac{k_{ij}}{k_{ji}}=0\\quad\\text{for every conditional cycle }c.$$",
  "$$\\lVert\\mathrm{d}X-\\log(k_{ij}/k_{ji})\\rVert=O(|J|),\\qquad \\sigma=O(J^2)\\quad\\text{near }J=0.$$"
 ],
 "id": 2750,
 "implementation": "Integrate this layer into a small VAE, discrete diffusion model, or mixture-of-experts router. Given hidden representation h, use an MLP to output costs F_i^cost(h), then set X_i=beta(Delta_mu-F_i^cost(h)). Use K conditional states plus reset state 0. For each conditional pair, let a_ij=softplus(g_ij(h))+epsilon and define k_ij=a_ij*exp((X_j-X_i)/2) and k_ji=a_ij*exp((X_i-X_j)/2); this enforces the exact conditional rate-ratio law. Add transitions from conditional states to 0 with rate r and reset-to-conditional transitions chosen to form a directed reset cycle. Sample by Gillespie simulation or uniformization, run a short burn-in, and pass the resulting state to the decoder or router. The paper-derived quantities are the occupation law, rate-ratio construction, sector factorization, and rapid-reset limit. Estimate P_0, mixing time, cycle currents, and conditional KL divergence from sampled trajectories. First test: train a small MNIST VAE with K=8 states against Gumbel-Softmax and categorical baselines, sweeping r/k over 0.1, 1, 10, and 100. The prediction is that KL(q_cond||p) decreases approximately linearly with k/r for sufficiently large r/k and that measured occupations approach p_i within 20 percent when r/k is at least 10, while the directed reset flux remains nonzero.",
 "math_summary": "Let i,j in {1,...,K} denote conditional states, 0 denote the reset state, beta\u003e0 be an inverse-temperature scale, Delta_mu be a maintained driving affinity, and F_i^cost be a neural-network-predicted cost. Define X_i=beta(Delta_mu-F_i^cost) and Z_C=1+sum_i exp(X_i), giving p_i=exp(X_i)/Z_C. If P_0 is the stationary probability of the reset state and P_i is the full-graph probability, sector separation gives P_i=(1-P_0)p_i. Construct conditional transition rates k_ij with log(k_ij/k_ji)=X_j-X_i, while reset transitions carry the dissipative cycle. If r is the reset rate and k is a typical conditional redistribution rate, the strong-reset limit r/k to infinity gives P_i to p_i. Near autonomous redistribution, the integrability defect is proportional to residual cycle current J, whereas entropy production sigma is proportional to J squared.",
 "math_tags": [
  "stochastic-processes",
  "statistical-mechanics",
  "graph-theory"
 ],
 "ml_areas": [
  "vae",
  "diffusion-sampling",
  "moe-routing"
 ],
 "paper": {
  "arxiv_id": "2608.26621",
  "arxiv_url": "https://arxiv.org/abs/2608.26621",
  "summary_what_math_gives_to_ml": "The paper gives a constructive method for embedding an approximately thermodynamic occupation manifold inside a globally dissipative Markov steady state. Its transferable mechanism is sector separation: exclude an entropy-producing reset state from the conditional manifold, impose an exact potential difference on conditional rate ratios, and use rapid reset to obtain a calibrated occupation law without eliminating full-graph circulation. This can become a stochastic latent or routing layer whose conditional probabilities follow neural energy logits while an auxiliary reset sector maintains mixing and exploration. The paper also supplies a diagnostic with a sharp scaling signature: near autonomous redistribution, integrability defects are linear in residual cycle current while entropy production is quadratic.",
  "title": "When dissipative steady states admit thermodynamic occupation laws",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 6,
  "novelty": 8,
  "usefulness": 7
 },
 "solves": [
  "stability",
  "accuracy",
  "sample-efficiency"
 ],
 "title": "Dissipative Softmax Latent Layer",
 "url": "https://synthcore.org/idea/2750/dissipative-softmax-latent-layer",
 "verification": {
  "benchmark": {
   "track": "token_expert_sequence",
   "model": "mlp_tiny",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 0.0015624999650754035,
    "idea_mean": 0.4946874864399433,
    "delta_mean": 0.4931249864748679,
    "per_seed_diffs": [
     0.5399999618530273,
     0.49999999767169356,
     0.48499998450279236,
     0.5049999952316284,
     0.442499992903322,
     0.517499980982393,
     0.47999998927116394,
     0.47499998938292265
    ],
    "idea_wins": 0,
    "n_pairs": 8,
    "p_value": 0.0081,
    "mde": 0.024702187634040935,
    "mde_rel_pct": 1580.9400439153835,
    "verdict": "idea worse (significant)",
    "system_worked": false
   },
   "baseline": {
    "best_cfg": {
     "epochs": 25,
     "lr": 0.01
    },
    "sweep": [
     {
      "cfg": {
       "epochs": 25,
       "lr": 0.001
      },
      "mean": 0.0012499999720603228
     },
     {
      "cfg": {
       "epochs": 25,
       "lr": 0.003
      },
      "mean": 0.0012499999720603228
     },
     {
      "cfg": {
       "epochs": 25,
       "lr": 0.01
      },
      "mean": 0.0006249999860301614
     }
    ],
    "full": {
     "mean": 0.0015624999650754035,
     "std": 0.0017399263244939971,
     "per_seed": [
      0,
      0.0024999999441206455,
      0,
      0,
      0.0024999999441206455,
      0.0024999999441206455,
      0,
      0.004999999888241291
     ],
     "n": 8
    }
   },
   "idea": {
    "mean": 0.4946874864399433,
    "std": 0.02705426452774727,
    "per_seed": [
     0.5399999618530273,
     0.5024999976158142,
     0.48499998450279236,
     0.5049999952316284,
     0.4449999928474426,
     0.5199999809265137,
     0.47999998927116394,
     0.47999998927116394
    ],
    "n": 8
   },
   "mechanism_signature": {
    "confirmed": false,
    "mismatch_at_r": null,
    "mismatch_at_r_0.5": null,
    "observed_cycle_proxy": null,
    "predicted_effect": "occupation mismatch decreases as reset_rate increases"
   },
   "custom_track": {
    "domain": "moe-routing",
    "file": "/home/maxwelhelp/all/math2nn/bench/custom_tracks/token_expert_sequence.py",
    "name": "token_expert_sequence"
   }
  },
  "mechanism_confirmed": false,
  "peer_reviewed": false,
  "practical_verdict": "harms",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 8,
   "verdict": "Built a finite-state dissipative softmax CTMC with exact conditional rate-ratio construction, reset exchange, stationary solving, cycle-current measurement, and a routing mini-comparison. The mechanism manifested: conditional error scaled approximately as (r/k)^-0.92, KL scaled approximately as (r/k)^-1.84, and weak-affinity sweeps gave current scaling A^1.000 and entropy production scaling A^2.000. The dissipative router approached the direct-softmax NLL as r/k increased, but did not provide a meaningful loss win.",
   "metrics": {
    "baseline": "Direct categorical softmax NLL = 1.697034",
    "idea": "Stationary conditional NLL was 1.699703 at r/k=1, 1.697657 at r/k=10, and 1.696939 at r/k=100. Exact conditional rate-ratio error was 2.22e-16; occupation-error slope was -0.923 versus predicted -1; KL slope was -1.841 versus predicted -2; weak-affinity current slope was 1.000 and entropy-production slope was 2.000; current remained approximately 0.024 at large reset ratio."
   },
   "how_to_run": "python3 dissipative_softmax.py \u0026\u0026 /home/maxwelhelp/main/bin/python3 experiment.py",
   "files": [
    "dissipative_softmax.py",
    "experiment.py",
    "article.md",
    "idea_context.json"
   ],
   "limitations": "The requested MNIST VAE/Gumbel-Softmax training experiment was not implemented; only a fixed synthetic routing/NLL comparison was run. The construction verifies conditional occupation convergence q(i|conditional) toward p(i), while symmetric reset exchange keeps P0 near 1/2, so full-graph probabilities do not converge to p without an additional P0 design. No Gillespie trajectory sampling, GPU benchmark, mixing-time estimate, or learned MLP costs were tested."
  },
  "status": "failed_benchmark",
  "status_label": "Failed on benchmark",
  "updated_at": "2026-09-02T00:15:47",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}