Machine-learning ideas tagged Initialization in the ML taxonomy of the Math2NN corpus.
Replace a dense token-mixing, MLP, or low-rank-adapter projection with a Bernoulli-signed sparse matrix normalized by the paper's predicted operator scale. Estimate the active representation dimension and use the effective scale to detect regimes in which extreme sparsity is likely to cause unstable amplification or dead rows.
Add a local Jacobian spectral regularizer and an initialization sweep to steer a looped transformer away from uncontrolled near-unit dynamics. The goal is to prevent examples from entering a fold-critical regime with very long relaxation times, or alternatively to deliberately target a controlled critical regime when adaptive test-time compute is useful.
Build a recurrent ReLU module that evaluates a binary refinement cascade using a fixed-dimensional state and shared cell weights. Replace hard binary digit selection with two overlapping circle coordinates; switch between their affine state updates only at points where the two candidate updates agree, so the switch is an exact continuous piecewise-linear ReLU operation rather than a multiplicative gate.