ML: Moe routing

Machine-learning ideas tagged Moe routing in the ML taxonomy of the Math2NN corpus.

299 ideas found

✗ Mechanism failed 2026

Behavior-Gap Clustered Neural Controllers

Cluster recurrent modules or MoE experts by the geometry of their observed finite-horizon input-output behaviors rather than by parameter distance. Train one shared optimizer/controller or low-rank adapter per cluster while retaining module-specific parameters and routing. This should reduce control and optimizer overhead without merging modules whose temporal responses are dynamically incompatible.

Useful8/10
Difficulty5/10
Novelty8/10
Paper: Data-Based Clustering and Control of Similar Biological Systems arXiv:2609.03921
△ Mechanism confirmed, baseline not beaten 2026

Farkas-Certified Neural Safety Shield

Insert a constraint-reduction layer between a policy network and its executed action. The policy proposes an action, while the layer retains only geometrically extreme collision and obstacle constraints and verifies that every discarded halfspace is implied by the retained ones through nonnegative conic multipliers. The reduced projection or quadratic program is therefore equivalent to the full tightened safety filter whenever certification succeeds, but uses substantially fewer constraints.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Scalable Tube-Tightened Multi-Agent Safety via Certified Constraint Reduction arXiv:2608.25323
△ Mechanism confirmed, baseline not beaten 2026

Delay-Aware Plug-and-Play Residual Capacity

Construct a residual network from independently attachable modules, but permit only a number of modules whose aggregate feedback gain lies inside a delay-dependent admissible interval. Estimate deployed end-to-end latency and each module's local Jacobian gain, then reject or bypass additional modules when the predicted delayed-loop stability boundary is crossed. This turns variable-width or depth scaling into a falsifiable control problem rather than an empirical choice.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Admissible Unit Range of Plug-and-Play Distributed Energy Resource (DER) Systems Under Delay: A Scalable Design Framework arXiv:2608.23328
✓✓ Beats tuned baseline 2026

Posterior-Routed Low-Rank Diffusion Experts

Use the noisy diffusion state to estimate posterior probabilities over data clusters, then route the denoising computation to one or a few cluster-specific score experts. Each expert stores a low-rank covariance or low-rank score correction, so after posterior commitment the denoiser operates at the cluster intrinsic dimension rather than ambient dimension. During the early mixing phase, retain a shared full-dimensional score or evaluate a small weighted set of experts.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Diffusion Models for High-Dimensional Clustered Data: Intrinsic-Dimension Adaptivity via Bayesian Classification arXiv:2608.19067
△ Mechanism confirmed, baseline not beaten 2026

Passivity-Certified Softmax Optimizer

Replace direct logit gradient updates for a simplex-valued neural module with a cascade consisting of a passive LTI filter followed by softmax. The filter can provide useful memory or momentum, but its transfer function is constrained to remain strictly passive, preventing the destabilization mechanism identified for nonpassive higher-order replicator dynamics.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Stabilization Limits of Payoff-Based Higher-Order Replicator Dynamics arXiv:2608.15308
✗ Failed on benchmark 2026

KL Mirror-Prox for coupled routing

Replace a standard softmax-gradient update for a probability vector with a two-stage KL Mirror-Prox update. The predictor evaluates the population-dependent cost at the current distribution, and the corrector evaluates it at the predicted distribution, reducing oscillation when routing or attention costs are coupled across tokens or samples.

Useful8/10
Difficulty5/10
Novelty5/10
Paper: Kullback-Leibler Mirror-Prox for Measure-Valued Variational Inequalities and Mean-Field Equilibria arXiv:2608.10293
✗ Mechanism failed 2026

Shared-Observation Collective Shield

For z neural branches that share a target, state, or routing observation, add a penalty on fluctuations in the branch direction visible to that shared signal. This implements the paper's centered-square conditioning mechanism: branches remain locally independent in hidden directions, while collective deviations that would produce inconsistent shared outputs are suppressed.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: A Shared Observation Shields Collective Fluctuations while Preserving Local Independence arXiv:2608.08358
✗ Failed on benchmark 2026

Koopman Deadline Controller

Learn a low-dimensional Koopman operator from successive states of an iterative neural system, such as debate agents, recurrent refinement blocks, or diffusion denoising trajectories. Use the magnitude of the subdominant eigenvalue to predict the remaining number of rounds required for disagreement to fall below a target tolerance, and stop computation when the predicted deadline is reached rather than using a fixed round budget.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Certifying Collective Reasoning in Multi-Agent Systems via Koopman Spectral Analysis arXiv:2608.05956
△ Mechanism confirmed, baseline not beaten 2026

Link-Wise Attack Observer for Switching Message Passing

Add a low-dimensional dynamical observer to every graph-NN or mixture-of-experts communication link and estimate additive message corruption before aggregation. The observer is switched together with the network mode, such as changing adjacency, expert assignment, attention mask, or operating regime; the corrected message is the received message minus the estimated attack. This should remain effective against attacks with arbitrarily large amplitude if their temporal rate is bounded and the…

Useful8/10
Difficulty6/10
Novelty8/10
Paper: Resilient Control of Switched Vehicle Platoons under False Data Injection Attacks arXiv:2607.28794
✓✓ Beats tuned baseline 2026

STL-Robust Mixture-of-Experts Gating

Replace a standard mixture-of-experts router or recurrent transition-mode classifier with a gate whose logits are adapted by the robustness of temporal safety specifications. Experts represent distinct dynamical regimes, while robustness increases the probability of experts whose predicted trajectories satisfy the specification and suppresses modes producing imminent violations. This should improve mode switches and long-horizon rollout quality precisely near safety-critical transitions.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Online Monitoring and Risk Assessment of Non-Cooperative UAVs via STL-Aware Adaptive Fusion Kalman Filtering arXiv:2607.26527
✗ Failed on benchmark 2026

Cycle-aware softmax temperature control

Use an online estimate of the positive feedback gain among logits, routing probabilities, and representations to adjust the softmax temperature. Increase temperature when the estimated cyclic gain approaches the instability regime, preventing exponential amplification and router collapse without globally weakening all layers.

Useful8/10
Difficulty6/10
Novelty6/10
Paper: Feedback Cycles in Exploratory Equilibria arXiv:2607.18128
✗ Failed on benchmark 2026

Path-Complete Robust Critic

Replace a single value critic with graph-indexed critics V_alpha and graph-indexed policy heads pi_A, where the labeled graph covers every possible environment mode at every step. Train sampled Bellman inequalities rather than only equality-based temporal-difference errors, and select the policy head using the paper's min-max reachability rule. This targets robust RL settings in which the transition mode can change arbitrarily, such as unknown actuator regimes, domain randomization modes, or…

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Robust Optimal Control of Arbitrarily Switched Systems: A Path-Complete Framework arXiv:2607.15055
△ Mechanism confirmed, baseline not beaten 2026

Tail-Aware Verifier Portfolio

Use the paper's tail comparison to decide when another call from the same verifier family is useless and when to switch to a different model, modality, or evidence source. The objective is to reduce the high-alpha survivor population—the incorrect examples that consistently fool one verifier—rather than maximizing average one-shot verifier accuracy.

Useful8/10
Difficulty5/10
Novelty7/10
Paper: Partially Correlated Verifier Cascades in LLM Harnesses: Concave Log-Odds, Polynomial Reliability, and Blind-Spot Ceilings arXiv:2607.13918
✓✓ Beats tuned baseline 2026

Mean-square-stable Markov-switching recurrent layer

Replace an unconstrained recurrent or state-space transition with a finite set of mode matrices selected by a Markov routing process, while explicitly constraining the associated Kronecker operator to have spectral radius below one. This targets exploding hidden-state variances caused by rare but repeatedly visited unstable modes, a failure mode not detected by average spectral radius or ordinary Lyapunov stability.

Useful8/10
Difficulty6/10
Novelty7/10
Paper: Spectral Characterisation of Covariance Existence in Markov-Switching Affine Recurrences arXiv:2607.09994
✗ Mechanism failed 2026

Retry-aware ignition-threshold router

Route requests between model-quality tiers using retry-adjusted satisfied-answer throughput instead of nominal completion throughput. Add hysteresis so degradation begins only above an upper backlog threshold and ends only after the backlog is safely below a lower threshold with negative retry-adjusted drift.

Useful8/10
Difficulty4/10
Novelty7/10
Paper: The Shadow Price of Intelligence: Quality Degradation in LLM Inference as a Supply Chain Problem arXiv:2608.23986
△ Mechanism confirmed, baseline not beaten 2026

Envelope-Max Neural Operator

Build each one-step operator as a maximum over a compact set of learned or discretized action branches, with a branch-dependent penalty. This directly imports the envelope structure used for nonlinear semigroups and gives a neural architecture suited to HJB equations, robust prediction, and stochastic control under model uncertainty.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Neural operators approximate strongly continuous convex monotone semigroups arXiv:2609.02727
✗ Mechanism failed 2026

Polar Slack Attention

Use a spherical-design codebook and the paper's polar slack factorization to create a nonnegative geometric interaction bias for attention or expert routing. The resulting kernel is generated by a rank-one term and a rank-at-most-d term, and entries close to zero can define a structured sparse mask instead of relying only on learned top-k selection.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Dual Geometry of Spherical Designs: Polarity, Self-Polar Rigidity, and Quadrature Structure arXiv:2609.02439
✗ Mechanism failed 2026

Dual recoverability gate for policy switching

Attach two independent critics to a learned controller: an empirical probability that executing the controller reaches a desired terminal set, and a recoverability value estimating whether the current state remains inside a safe reach-avoid region. Execute the learned controller only when both critics pass their thresholds; otherwise route to a conservative fallback policy or expert.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Humanoid Safe Stop via Learned Stoppability Value arXiv:2609.02358
✓✓ Beats tuned baseline 2026

Strongly-Rayleigh Forest Dropout

Replace independent feature, expert, or edge dropout by sampling subsets from a forest-induced strongly Rayleigh distribution. The distribution is multiaffine and negatively dependent, so selecting one computational route suppresses redundant correlated routes without the positive correlations created by independent or blockwise heuristics. Use the sampled subset as a structured mask during training and compare it against Bernoulli dropout at equal expected sparsity.

Useful7/10
Difficulty7/10
Novelty8/10
Paper: All-Minors Matrix-Tree Theory for Superport Networks: Completed Quotient-Incidence Determinants and Conductance-Weighted Subdivision Extensions arXiv:2609.01672
△ Mechanism confirmed, baseline not beaten 2026

Mean-field rainbow relation router

Replace independent edge-type logits in a relational graph neural network with a mean-field fixed-point router derived from a colored ERGM. Each edge's color distribution is influenced by its own relation bias and by the expected number of rainbow triangles it forms with neighboring edges, allowing the model to learn coordinated multilayer structures.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Colorful Exponential Random Graph Models arXiv:2608.31130
✓✓ Beats tuned baseline 2026

Spectral pinning of neural modules

Represent communicating layers, experts, or distributed workers as nodes of a weighted graph and apply strong corrective updates only to a small pinned subset. Select pins by the increase they produce in the grounded Laplacian smallest eigenvalue, because this spectral gap predicts the decay rate of representation disagreement.

Useful7/10
Difficulty6/10
Novelty6/10
Paper: Event-Triggered Pinning Impulsive Control of Complex Networks with Actuation Delays: Stability Analysis and Zeno-Free Conditions arXiv:2608.24074
✓✓ Beats tuned baseline 2026

Small-Gain Constrained Neural Modules

Partition a neural network into interacting modules and constrain the product of their local finite-region gains and coupling strengths so that the resulting gain matrix has spectral radius below one. This transfers the paper's small-gain-like mechanism and gives a quantitative large-signal boundary: instability or exploding activations should emerge as the spectral radius approaches one, while a weighted Lyapunov function should contract below that boundary.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: A Small-Gain-Like Framework for Large-Signal Stability Evaluation of Multi-Converter Systems arXiv:2608.29570
✗ Mechanism failed 2026

Two-Solve Robust Expert Router

Replace point-estimate expert routing with a nominal allocation and a least-favourable allocation under uncertainty in expert quality. If both allocations agree, use that route confidently; if they disagree, profile or evaluate only the expert-input pairs responsible for the disagreement. The same mechanism can be used offline to assign workloads to LLMs or online to choose among heterogeneous experts under a latency or FLOP budget.

Useful7/10
Difficulty6/10
Novelty7/10
Paper: Which LLM for Which Work? Budgeted Model Allocation under Uncertain Evaluation arXiv:2608.29560
✗ Failed on benchmark 2026

Spectral-Gated Parallel Best Responses

Partition neural-network parameters into competing blocks, such as LoRA adapters, mixture-of-experts heads, or task-specific heads, and update each block by minimizing its local quadratic model while holding the other blocks fixed. Use the exact Jacobi coupling spectral radius to decide whether simultaneous updates are stable; near the boundary, apply damping or fall back to sequential Gauss-Seidel updates.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Competitive One-Step-Ahead Control of Friedkin--Johnsen Networks: Potential Games, Stability, and the Price of Competition arXiv:2608.27623