{
 "artifacts": null,
 "category": "regularization",
 "description": "Add a discrete structure-selection gate before a neural predictor, maintaining separate masks for explanatory structure and predictive performance. Use entropy reduction only when the discretization resolution is finer than the observed stochasticity; otherwise use a validation-calibrated predictive mask or retain both masks through a mixture-of-experts gate.",
 "formulas_latex": [
  "$$S_H(j)=\\widehat H(Y\\mid M)-\\widehat H(Y\\mid M,X^j),\\qquad \\widehat H(Y\\mid Z)=-\\sum_z\\widehat p(z)\\sum_y\\widehat p(y\\mid z)\\log\\widehat p(y\\mid z).$$",
  "$$S_R(j)=\\widehat R(M)-\\widehat R(M\\cup\\{j\\}),\\qquad \\widehat R(M)=\\frac{1}{n}\\sum_{r=1}^{n}\\left(y_r-f_\\theta(x_{M,r})\\right)^2.$",
  "$$\\kappa=\\frac{\\widehat H(Y\\mid M)}{H_{\\mathrm{res}}},\\qquad \\gamma=\\frac{\\max_j S_H(j)}{\\operatorname{sd}_{b=1}^{B}\\!\\left[S_H^{(b)}(j^*)\\right]+\\epsilon},\\qquad M=\\begin{cases}M_H,\u0026\\kappa\u003c\\kappa_c\\ \\text{and}\\ \\gamma\u003e\\gamma_c,\\\\M_R,\u0026\\text{otherwise.}\\end{cases}$$",
  "$$\\Delta_R=\\widehat R(M_H)-\\widehat R(M_R),\\qquad \\text{prediction: }\\Delta_R\\text{ increases with }\\prod_{j\\in M_H}q_j\\text{ and decreases approximately as }n^{-1/2}\\text{ or faster.}$$"
 ],
 "id": 3124,
 "implementation": "1. Integration point: use this as a front-end selector for a tabular MLP, an RNN/SSM input window, or a transformer whose candidate tokens are variable-lag pairs. During each selection refresh, discretize each continuous feature using fixed quantile bins, and discretize the target or residual target into $q_Y$ bins solely for structure discovery; the neural predictor still receives the original continuous values. 2. Pseudocode: construct candidate mask $M$; compute plug-in counts for $\\widehat H(Y|M)$ and $\\widehat H(Y|M,j)$; compute $S_H(j)$; generate $B$ bootstrap resamples and estimate $\\gamma$; train or briefly evaluate the network for each candidate addition to estimate $S_R(j)$; calculate $\\kappa$; choose $M_H$ if $\\kappa\u003c\\kappa_c$ and $\\gamma\u003e\\gamma_c$, else choose $M_R$; train the final network on the chosen mask. A safer variant concatenates both masks and learns a gate $g=\\sigma(a(\\kappa_c-\\kappa))$, using $g f_{M_H}+(1-g)f_{M_R}$. 3. Computed versus estimated: alphabet sizes, entropy formulas, and the mask rule are computed exactly from counts; $\\kappa_c$, $\\gamma_c$, and prediction risk are estimated on held-out data. Start with $\\kappa_c=1$ and $\\gamma_c=2$, then sweep them. 4. First cheap experiment: use a synthetic two-state Markov system with one true binary lag, several distractor lags, and controllable flip noise $Y_t=X_{t-1}\\oplus N_t$, where $N_t\\sim\\operatorname{Bernoulli}(p)$. Compare entropy-only, MSE-only, and gated selection with a small MLP. Sweep bin resolution and $p$ over 0 to 0.5, using sequence lengths 256, 1024, and 4096. The predicted signature is a crossover near the point where conditional noise entropy approaches the output discretization entropy: entropy-mask recovery should fall sharply there and should not recover merely by increasing $n$ at fixed excessive noise. Below the crossover, the entropy mask should match the true lag more often; above it, MSE should have lower test risk. Measure whether the entropy-mask regret $\\Delta_R$ increases with the number of distractor combinations and decreases with sample size.",
 "math_summary": "Let $X^j_t$ be input variable $j$ at time $t$, $Y_t$ the target, and $M$ a candidate mask containing selected lagged variables. For discretized variables with alphabet sizes $q_j$, score explanatory structure by conditional entropy reduction $S_H(j)=H(Y\\mid M)-H(Y\\mid M,X^j)$, where $H$ is empirical Shannon entropy. Score prediction by validation risk reduction $S_R(j)=\\widehat{R}(M)-\\widehat{R}(M\\cup\\{j\\})$, with squared risk $\\widehat{R}=n^{-1}\\sum_{r=1}^n(y_r-\\hat y_r)^2$. Define an operational resolution-stochasticity ratio $\\kappa=\\widehat H(Y\\mid M)/H_{\\mathrm{res}}$, where $H_{\\mathrm{res}}=\\log q_Y$ for a $q_Y$-state output, or more conservatively the entropy of the within-bin residual distribution. Low $\\kappa$ means the output uncertainty is resolved by the bins; high $\\kappa$ means stochasticity is comparable to or larger than the available resolution. Select the entropy mask only when its bootstrap signal-to-noise ratio $\\gamma=\\max_j S_H(j)/(\\operatorname{sd}_{b}[S_H^{(b)}(j)]+\\epsilon)$ exceeds a threshold and $\\kappa\u003c\\kappa_c$. Otherwise select by $S_R$. The mechanism predicts that causal-mask recovery degrades sharply near and above $\\kappa_c$, while prediction regret from using the entropy mask grows with the number of input combinations and decreases with sample size.",
 "math_tags": [
  "information-theory",
  "statistics",
  "dynamical-systems",
  "optimization"
 ],
 "ml_areas": [
  "training-dynamics",
  "optimizer",
  "transformer",
  "rnn"
 ],
 "paper": {
  "arxiv_id": "2609.03074",
  "arxiv_url": "https://arxiv.org/abs/2609.03074",
  "summary_what_math_gives_to_ml": "The paper provides a transferable mechanism for deciding when discretized, entropy-based structure discovery is statistically reliable: a resolution-stochasticity ratio creates a hard identifiability boundary that cannot be removed by simply collecting more samples. It also separates explanatory variable selection from predictive selection, showing that an entropy-selected mask can incur excess prediction risk when inputs are numerous or the entropy signal is weak. A useful neural-network transfer is a dual-mask selector that estimates this reliability ratio before committing to interpretable sparse inputs, and falls back to prediction-oriented selection when discretization noise dominates. The key falsifiable signature is a sharp collapse of entropy-mask recovery as stochasticity exceeds discretization resolution, together with increasing prediction regret as the number of selected categorical combinations grows.",
  "title": "Finite-Sample Limits of Entropy-Based Structure Identification in Discretized Nonlinear Systems",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 7,
  "usefulness": 7
 },
 "solves": [
  "generalization",
  "accuracy",
  "stability"
 ],
 "title": "Resolution-Gated Dual Masking",
 "url": "https://synthcore.org/idea/3124/resolution-gated-dual-masking",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": null,
    "tested": false
   },
   "practical_benchmark": {
    "beats_baseline": null,
    "tested": false
   },
   "toy_mechanism_gate": {
    "confirmed": null,
    "tested": false
   }
  }
 }
}