Audited (legacy)
2026
Use the attention probability distribution over an ordered context to choose contiguous token groups whose pooled attention masses have entropy as close as possible to a prescribed upper budget R. Replace the corresponding key/value vectors by one weighted representative per group, preserving token order and reducing the KV-cache length from n to m. Unlike unconstrained token merging, the entropy constraint gives a direct control knob over how concentrated or diffuse the retained attention…
Useful6/10
Difficulty5/10
Novelty7/10
✗ Mechanism failed
2026
Use the paper's heavy-ball recursion as a runtime diagnostic for momentum optimizers. Detect when recent parameter differences form an approximately periodic orbit or when the estimated local two-step transition matrix has spectral radius near or above one, then reduce the learning rate and momentum temporarily. This targets the failure mode proved in the paper: fixed momentum parameters can produce attracting cycles even on smooth potentials with bounded curvature.
Useful6/10
Difficulty5/10
Novelty6/10
✗ Mechanism failed
2026
Replace the random or gradient-aligned perturbation in sharpness-aware minimization with a unit perturbation direction selected by a polynomial of the local Hessian. With \(\mathscr{P}(s)=(s-\rho)^2\), the direction converges toward Hessian eigenspaces whose eigenvalues are closest to the target curvature \(\rho\), allowing regularization of a chosen curvature band instead of indiscriminately penalizing only the sharpest direction.
Useful6/10
Difficulty6/10
Novelty7/10
✗ Mechanism failed
2026
Use a constant-sum point vector to encode ordered pairwise outcomes and train a neural scorer with an adjacent-categories ordinal likelihood whose slope parameters are tied to those points. The accumulated point score is then a theoretically motivated compressed statistic for repeated comparisons, rather than an arbitrary regression target or one-hot label.
Useful6/10
Difficulty4/10
Novelty6/10
✗ Failed on benchmark
2026
Replace Gaussian perturbations in a low-dimensional neural-network optimizer with independent double-geometric integer mutations and adapt each mutation scale using its exponential-family natural gradient. Apply the method to layerwise quantization scales, adapter coefficients, pruning thresholds, or other integer/discrete hyperparameters rather than to every individual weight.
Useful6/10
Difficulty5/10
Novelty7/10
✓✓ Beats tuned baseline
2026
Add a geometry-guided infill operator to a population optimizer used for black-box neural-network tuning. Fit a local Jacobian from recent parameter perturbations and validation-residual vectors, generate a damped Gauss-Newton candidate for exploitation, and sample exploratory candidates in the same Jacobian-derived metric. The host optimizer retains selection, population survival, covariance adaptation, and its total evaluation budget; only a configurable fraction of new candidates is replaced…
Useful6/10
Difficulty5/10
Novelty7/10
△ Mechanism confirmed, baseline not beaten
2026
Use the observed label-availability indicator as an auxiliary supervision signal when labels are preferentially missing for uncertain or difficult examples. Train the classifier with a joint likelihood containing both the class-label likelihood for labeled examples and a missingness likelihood whose probability depends on the classifier's posterior uncertainty.
Useful6/10
Difficulty4/10
Novelty5/10
✗ Mechanism failed
2026
Replace AdamW or SGD updates on simplex-valued routing probabilities with a logarithmic-barrier mirror step. The update remains strictly positive, avoids projection-induced zero coordinates, and can approach a boundary solution asymptotically while retaining the paper's theoretically motivated O(log k/k) convex convergence behavior.
Useful6/10
Difficulty5/10
Novelty6/10
✓✓ Beats tuned baseline
2026
Use an approximate decision diagram to select a structured subset of neurons, channels, attention heads, or attention edges when their quadratic interactions are sparse or inverse-sparse. Merge states that agree on a local interaction boundary and accept a tunable epsilon loss in the pruning objective, obtaining a representation whose size is linear in model width for fixed accuracy tolerance.
Useful6/10
Difficulty6/10
Novelty7/10
✗ Failed on benchmark
2026
Represent an intermediate feature as a low-rank PSD matrix and compress it using nonnegative measurements \(\langle A_i,X\rangle\), while penalizing the empirical ratio between maximum and minimum measurement distortion over low-rank feature pairs. This directly discourages collapsed directions and excessively amplified directions in a covariance or Gram-feature bottleneck.
Useful6/10
Difficulty5/10
Novelty6/10
✗ Failed on benchmark
2026
Add a loss term requiring a neural optimizer or recurrent module to decrease a nonnegative Lyapunov-like energy over M update steps, rather than forcing monotonic one-step decrease. The term includes an empirically estimated mismatch allowance, so stochastic or delayed updates are tolerated while persistent instability remains penalized.
Useful6/10
Difficulty4/10
Novelty7/10
✗ Mechanism failed
2026
Choose the consensus gain and gradient-tracking gain in decentralized training from the communication Laplacian spectrum rather than tuning them independently. The gains minimize the worst asymptotic pole radius for the paper's exact quadratic model, providing a principled initialization and a conservative stability safeguard for neural-network optimization.
Useful6/10
Difficulty6/10
Novelty6/10
△ Mechanism confirmed, baseline not beaten
2025
Regularize a neural dynamical map so that its log-volume expansion is cohomologous to a constant rather than forcing the Jacobian determinant to be constant at every state. Learn a scalar potential that explains transient expansion and penalize only the non-telescoping component, which should reduce long-horizon gradient explosion or collapse while retaining useful average expansion.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Add asynchronous nonlinear consensus steps between local-SGD updates in a federated or decentralized system. Instead of averaging a participating client with all peers, replace one client parameter vector by the coordinatewise p-mean of neighboring replicas, allowing sparse communication and reduced sensitivity to atypical client models.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Use state-dependent Langevin noise to perturb prototype or router-codebook optimization when balanced quantization reaches a poor stationary configuration. Retain the best-so-far state for deployment, allowing exploration to escape non-minimizing Lloyd fixed points without permanently corrupting the learned representation.
Useful5/10
Difficulty4/10
Novelty5/10
Unverified
2026
Replace ad hoc Gaussian or Laplace noise injection with a Laplace majorant calibrated to the observed finite-range sub-Gamma parameters of a neural perturbation. For convex perturbation losses, the calibration guarantees that the expected loss under the scaled Laplace noise upper-bounds the expected loss under every centered random perturbation satisfying the same Bernstein-type MGF constraint.
Useful5/10
Difficulty6/10
Novelty6/10
Unverified
2026
Represent an original neural-network block and a proposed rewritten block as constrained optimization formulations over inputs and trainable parameters, then certify that the rewrite preserves feasibility and the ordering of losses over a bounded domain. This gives a compiler or pruning pipeline a formal reject/accept gate instead of relying only on numerical regression tests.
Useful5/10
Difficulty7/10
Novelty8/10
Unverified
2026
Use the graph-coloring stability concept to route graph nodes to experts. Nodes rank experts by router logits, adjacent nodes are constrained to use different experts, and a blocking cycle is a directed cycle in which every node prefers the expert currently assigned to the next node. Eliminate profitable feasible cycles or penalize their existence so routing reaches a locally stable assignment instead of oscillating between equally plausible expert allocations.
Useful5/10
Difficulty6/10
Novelty9/10
Unverified
2026
Use the paper's analytic merging threshold to choose the Scaled-NAP exponent from an intended community size rather than treating alpha as an arbitrary hyperparameter. A warm-started schedule can begin with persistence-like fine structure and increase alpha only when the model has learned reliable local groups.
Useful5/10
Difficulty3/10
Novelty8/10
Unverified
2026
Use a Goldfarb–Idnani-style active-set solver as a neural constrained layer or optimizer substep, but never trust a guessed active set solely because its linear system solved. Remove duplicate or dependent constraints, solve the reduced KKT system, and accept the result only after checking primal feasibility, dual sign conditions, and stationarity. This gives exact enforcement of linear inequalities and a diagnostic certificate when the constraint set is infeasible.
Useful5/10
Difficulty6/10
Novelty5/10
Unverified
2026
Replace ordinary per-coordinate activation normalization or covariance whitening with a learned or frozen affine map based on a minimum-volume ellipsoid calibrated to the activation convex hull. Estimate the hull's Minkowski symmetry and use the bound \(\rho=\sqrt{d/\alpha}\) to set a principled clipping radius, where \(d\) is feature dimension and \(\alpha\) is the measured symmetry. This should reduce worst-case clipping of asymmetric activation distributions without requiring a much larger…
Useful5/10
Difficulty6/10
Novelty4/10
Unverified
2026
Replace two unconstrained scalar quadratic feature heads with a pair whose quadratic forms admit a positive-definite linear combination. This prevents the two heads from simultaneously vanishing on any nonzero hidden vector, which can reduce representation collapse and improve the conditioning of downstream gates or auxiliary objectives. The constraint can be implemented softly with a spectral-margin penalty, or exactly by parameterizing one learned pencil as positive definite.
Useful5/10
Difficulty4/10
Novelty8/10
Unverified
2026
Replace a hard nonnegative slack or ReLU output by the barrier-derived map \(x_s(w)=\frac{w+\sqrt{w^2+4s}}{2}\). Unlike an arbitrary smooth activation, this output is the unique positive solution of \(x(x-w)=s\), so the network can explicitly monitor complementarity and anneal \(s\) toward the true inequality-constrained solution. Use it in a constrained output head or in hidden layers whose activations represent nonnegative resource, probability, or routing slack variables.
Useful5/10
Difficulty3/10
Novelty4/10
Unverified
2026
Replace continuous stochastic-gradient updates by a flashing schedule with alternating ON phases, where gradients act normally, and OFF phases, where gradients are suppressed or weakened and controlled noise allows escape from local traps. Estimate directional asymmetry of the local loss basin from forward and backward probe distances, then set the flashing frequency using the ratchet resonance law so that noise-assisted transitions preferentially produce net progress toward lower loss.
Useful5/10
Difficulty6/10
Novelty8/10