Architecture ideas

Attention variants, state-space and recurrent cells, normalization and token-mixing schemes — each tested against the standard block it replaces.

✗ Failed on benchmark 2026

Confidence-Tested LoRA Pruning

Replace deterministic LoRA importance scores with one-sided tests of whether each rank-one update has population contribution at least a user-selected threshold. Maintain empirical contribution samples during fine-tuning, estimate their uncertainty, and prune the components with the weakest statistical evidence while respecting the target rank budget. The method should avoid deleting components merely because their latest minibatch gradient was small.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Statistical Inference for Rank Allocation in Low-Rank Adaptation arXiv:2607.20205
✗ Failed on benchmark 2026

Volume-Mass Diffusion GNN

Replace ordinary graph propagation by diffusion with a positive node-dependent mass matrix \(\mathbf V\), so high-volume nodes update slowly and low-volume nodes update rapidly. Use node volumes as fixed metadata, a function of degree, or learned positive gates; this makes the architecture sensitive to dynamical localization that degree-normalized GCNs cannot represent.

Useful7/10
Difficulty5/10
Novelty7/10
Paper: Localization transitions of diffusion dynamics in physical networks arXiv:2607.19486
✗ Failed on benchmark 2026

ISS-Constrained Modular Recurrent Network

Replace an unconstrained recurrent block with two coupled modules: a contractive perceptual estimator and an input-to-state-stable cognitive state transition. Spectral normalization and a controlled Euler residual step enforce a quantitative gain condition, preventing hidden-state explosion while retaining long memory when the contraction factor is chosen close to one.

Useful7/10
Difficulty5/10
Novelty6/10
Paper: A modular state-space model of human perception, cognition, and decision dynamics arXiv:2607.14078