Emergent Models: Intelligence from Tiny Substrates
arXiv:2608.14019
2026
Architecture
2 ideas extracted · analyzed Sep 1, 2026
What the math gives to ML
The paper supplies a concrete alternative to feed-forward depth: a small latent workspace updated repeatedly by one shared local transition rule, with computation time represented by the number of microsteps rather than by parameter count. The transferable asset is the separation between an encoder/decoder interface and an autonomous recurrent substrate, together with adaptive halting and optional state retention across inputs. Its latent-universality result is primarily a design principle rather than a direct training guarantee: a fixed rule can express broad computations through suitable initial states, suggesting that optimization should focus on discovering useful encodings and compact transition rules. The most promising ML tests are a parameter-tied recurrent block with learned adaptive computation time and a stateful streaming variant, compared against ordinary deep residual blocks and SSMs at equal parameter and FLOP budgets.
Ideas from this paper
△ Mechanism confirmed, baseline not beaten
2026
Turn the latent substrate into a persistent computational workspace for sequential inputs: each new observation is written into a designated subspace, processed by the same local rule, decoded, and then selectively retained or reset. This creates a compact recurrent model whose state can accumulate algorithmic information across a stream without expanding the parameter count.
Useful7/10
Difficulty6/10
Novelty7/10
Unverified
2026
Replace a stack of independently parameterized residual or MLP blocks with a small latent grid or vector repeatedly updated by one shared transition rule. Let the number of updates depend on the current latent state, so easy examples terminate early while hard examples receive more computation, potentially improving parameter efficiency and extrapolation.
Useful7/10
Difficulty5/10
Novelty6/10