Unverified
2026
Add a consistency loss for a neural model processing irregularly sampled functions or sequences, comparing its latent covariance representation before and after a time-coordinate warp. The target discrepancy should be near zero for affine transformations but should grow with the normalized curvature, bandwidth, and local sparsity of a nonlinear warp.
Useful6/10
Difficulty5/10
Novelty6/10
Unverified
2026
Use a small set of known or trusted pairwise costs to estimate the effective entropic temperature of a Sinkhorn attention or mixture-of-experts routing layer directly from its observed transport plan. This provides a calibration controller that can detect over-concentrated routing and adjust epsilon without backpropagating through a costly temperature search.
Useful6/10
Difficulty4/10
Novelty8/10
Unverified
2026
Partition neural modules into two empirically identified reliability or noise classes and restrict their communication graph to a two-block stochastic block model. Allocate a fixed connectivity budget across within-class and cross-class edges using a water-filling update that favors block pairs producing the largest increase in validation utility. The resulting layer is sparse and modular, with a testable prediction that optimal connectivity concentrates on a few block pairs rather than…
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace an MoE router's single softmax distribution with a normalized coordinate-wise product of several simplex-valued routing factors. The product preserves positivity and normalization but, as depth grows, concentrates mass on a small subset of experts, creating a mathematically controlled heavy-tailed routing prior rather than relying only on an auxiliary load-balancing loss.
Useful6/10
Difficulty5/10
Novelty7/10
Unverified
2026
Construct training and evaluation examples whose every (d-1)-variable marginal is exactly independent, but whose full d-variable distribution contains a parity interaction. This isolates genuine high-order reasoning from shortcuts based on pairwise or lower-order statistics and can expose whether attention or MLP architectures learn the intended interaction.
Useful6/10
Difficulty3/10
Novelty6/10
Unverified
2026
Replace raw Pearson correlation when evaluating a temporal neural predictor with a score measuring how many null standard deviations its Fisher-transformed correlation exceeds. Estimate the null scale from a small set of time-misaligned predictions, then reuse it across context lengths or checkpoints. This prevents models from being rewarded for predicting statistically easy, low-information features and gives a more comparable validation signal across datasets and targets.
Useful6/10
Difficulty3/10
Novelty8/10
Unverified
2026
Replace a Gaussian or point-estimate regression head with a heteroscedastic Student-t head whose scale and degrees of freedom depend on the learned state. This gives the model a principled way to absorb abrupt, nonmonotone events and operating-condition shifts without forcing the central degradation trend toward rare extreme residuals.
Useful6/10
Difficulty3/10
Novelty4/10
Unverified
2026
Replace raw pairwise attention or graph-edge scores by exact U-centered residuals, removing additive effects attributable to either endpoint. The resulting scores represent interaction beyond independent source and destination biases and satisfy zero row sums, preventing a few high-degree or high-activation tokens from dominating relational aggregation.
Useful6/10
Difficulty3/10
Novelty6/10
Unverified
2026
Attach a predictor from each deeper representation to the representation immediately below it, and penalize the Gaussian KL divergence between the predicted lower-layer state and the actual lower-layer state. Learn or estimate one positive variance per layer so easy, low-noise layers receive high precision while intrinsically uncertain layers are not forced to fit their targets exactly.
Useful6/10
Difficulty4/10
Novelty5/10
Unverified
2026
Use the paper's independence condition as a design principle: prevent the gate's type-dependent admission behavior from being strongly correlated with downstream service cost. In an MoE or dynamic inference system, this discourages the gate from rejecting cheap requests and then preferentially admitting expensive requests when the queue happens to be shorter.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Use the paper's effective-rank and eigengap-dependent covariance estimation rates to construct a confidence-aware low-rank bottleneck for transformer activations or key/value tensors. The bottleneck is enabled only when the top-p empirical eigenspace can be estimated more accurately than the desired compression error; otherwise the layer remains full-rank.
Useful6/10
Difficulty5/10
Novelty5/10
Unverified
2026
Augment a causal sequence model with a small hierarchy of prefix summaries weighted by powers of the logarithmic rank of each preceding token. The summaries retain order-sensitive deviations from a baseline representation while costing O(KNd) for sequence length N, hierarchy width K, and hidden dimension d, instead of O(N^2d) dense attention.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Require the calibrated state posterior to remain unchanged when evidence is presented through information-equivalent prompt templates. Compare state distributions after semantic aggregation rather than raw token probabilities, and add a total-variation consistency penalty during calibration or fine-tuning.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Use the paper's attribution converse to calibrate watermark strength and sequence length for a registry of N users, rather than tuning detection and attribution thresholds independently. A dual controller allocates a per-token information and KL budget so that the learned key information approaches the minimum required for reliable attribution, avoiding both underpowered marks and unnecessarily visible perturbations.
Useful6/10
Difficulty4/10
Novelty6/10
Unverified
2026
Add an auxiliary loss that makes selected representation coordinates insensitive to all subsets of fewer than d variables while retaining a d-way parity statistic. The objective discourages the network from solving a task through pairwise shortcuts and explicitly rewards a controlled high-order interaction.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Adapt the slope of each spiking neuron's surrogate derivative using the normalized entropy of its block's attention distribution. High centered entropy uncertainty increases the slope, while low uncertainty decreases it, and a dead zone holds the default slope fixed for ordinary fluctuations. The adaptation exists only in backpropagation, so the forward spike function, parameter count, and inference cost remain unchanged.
Useful5/10
Difficulty4/10
Novelty5/10
Unverified
2026
Add a learnable orthogonal rotation to a hidden representation and train it to make every channel projection have a small ψ2/L2 ratio. Unlike variance normalization, this explicitly suppresses directions with unusually heavy empirical tails while preserving the total quadratic energy of the representation.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Apply a low-degree polynomial feature lift to normalized hidden representations and penalize degeneracy of the covariance in that lifted space. This can detect collapse in nonlinear combinations of features even when the raw hidden covariance appears healthy.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Replace a collection of dense task-specific linear layers with a common sparse structural matrix and task-specific edge strengths. All tasks share the same learned connectivity pattern, but retain independent values on active connections, allowing parameter sharing without forcing identical interactions.
Useful5/10
Difficulty5/10
Novelty5/10
Unverified
2026
Replace arbitrary learned thresholds in a binary MoE or hierarchical latent router with a threshold at the batch mean of a learned scalar projection. Add a penalty when the entropy of either routed subgroup falls too far below the parent entropy, using the paper's sharp constant as the target. This discourages routing branches from becoming nearly deterministic or semantically impoverished while retaining a simple, cheap gating operation.
Useful5/10
Difficulty5/10
Novelty7/10
Unverified
2026
Treat the maximum dependency distance faithfully modeled by a finite neural architecture as an emergent correlation length, and estimate how it grows with depth, state size, or attention span. Fit the exponent \(\kappa\) and use it as an architecture-selection signal: a model with larger \(\kappa\) should acquire long-range competence more efficiently at equal parameter or FLOP budget.
Useful5/10
Difficulty3/10
Novelty6/10
Unverified
2026
Normalize attention or router logits and control their upper tail using the paper's sharper Gaussian-maximum exponent rather than a correlation-blind sub-Gaussian bound. Use the resulting threshold to add a soft penalty or adaptive temperature whenever the observed maximum exceeds the calibrated level, reducing rare one-token or one-expert domination.
Useful5/10
Difficulty4/10
Novelty6/10
Unverified
2026
Replace softmax attention or dense MoE routing with a normalized Rayleigh–Jeans distribution over tokens or experts. If an item's energy is close to the chemical potential, its probability becomes disproportionately large, creating controllable low-energy condensation instead of the exponentially smooth allocation produced by softmax.
Useful5/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace a neural layer's dense parameters by a real parameter tensor and a binary activity mask, then update the mask using importance scores measured across multiple perturbation intervals. Gates that repeatedly occur in non-dominated accuracy–cost configurations receive larger scores and are more likely to be retained or activated, avoiding uniformly random sparse rewiring.
Useful5/10
Difficulty5/10
Novelty4/10