# Holonomy-designed recurrent memory

- ID: 2874
- Canonical URL: https://synthcore.org/idea/2874/holonomy-designed-recurrent-memory
- API JSON: https://synthcore.org/api/idea/2874.json
- API Markdown: https://synthcore.org/api/idea/2874.md
- Verification status: queued_mechanism
- Source: [arXiv:2608.30231](https://arxiv.org/abs/2608.30231)
- Category: architecture
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, architecture, training-dynamics
- Math tags: algebra, combinatorics, dynamical-systems
- Ratings: usefulness 7/10; difficulty 5/10; novelty 8/10

## Idea description

Build a recurrent module from two finite-state factors: a normalization state and a winner-selection state. Choose or learn their coupling so that the joint transition system contains a certified composite two-cycle, giving the network a small robust memory state, while every fixed-input generator still collapses most states toward attractors. The module can be embedded in a continuous RNN using soft state assignments during training and straight-through discretization for algebraic auditing.

## Mathematical statement

The paper represents each input-conditioned recurrent update as a transformation of a finite state set, and the generated transition monoid is M = <f_u : u in U>, consisting of all finite compositions of generators f_u. For the WTA example, repeated application of one generator is aperiodic: 3=(0,1,1) maps to 1=(0,0,1), then to 4=(1,0,0), then to 5=(1,0,1), so no cycle longer than one is produced by that frozen drive. After coupling normalization and competition, the joint state set contains C = {(D:0,W:4),(D:1,W:5)}. The paper reports that an element of the generated monoid exchanges these two states, so its restriction to C is the nontrivial permutation group Z_2. Here D is the discrete normalization state, W is the discrete WTA state, and C is an image set of the joint transition system. The important property is composite action: both coordinates change under the same monoid element, so the cycle is not inherited from either factor separately. In an ML adaptation, let q_t=(d_t,w_t) be a quantized recurrent state, let f_u(q) be the transition under input symbol or input bin u, and require a target word u_{1:L} to satisfy f_{u_L} compose ... compose f_{u_1}(q_0)=q_1 and the reverse word to map q_1 back to q_0, while a contraction penalty sends non-target states toward fixed points.

## Key formulas

- $$3=(0,1,1)\mapsto 1=(0,0,1)\mapsto 4=(1,0,0)\mapsto 5=(1,0,1),$$
- $$M=\langle f_u:u\in U\rangle=\{f_{u_k}\circ\cdots\circ f_{u_1}:k\geq 0,\ u_i\in U\},$$
- $$C=\{(D{:}0,W{:}4),(D{:}1,W{:}5)\},\qquad g|_C:(D{:}0,W{:}4)\leftrightarrow(D{:}1,W{:}5),\qquad \langle g|_C\rangle\cong\mathbb{Z}_2,$$
- $$\mathcal{L}_{\mathrm{cycle}}=\|F_{u_{1:L}}(q_0)-q_1\|_2^2+\|F_{v_{1:K}}(q_1)-q_0\|_2^2,\qquad F_{u_{1:L}}=f_{u_L}\circ\cdots\circ f_{u_1}.$$

## Implementation notes

Integrate this into a small GRU-like recurrent block or an SSM state update, not into the output head. Split the hidden state into two low-dimensional factors: d_t for normalization/gain and w_t for winner identity. Compute continuous updates h'_t = RNN(h_{t-1},x_t), then map them to soft discrete states p_t = softmax(A h'_t / tau), with p_t containing logits over the finite joint states q=(d,w). Implement each input bin u as a transition matrix T_u over joint states, or parameterize T_u through differentiable DN and WTA updates; use p_t = p_{t-1} T_u during the algebraic training branch. Pseudocode: initialize p0; for each symbol u, p <- p @ T[u]; retain the resulting p for the cycle loss; decode h_t from p or use the continuous hidden state as the task representation. Select a target pair q0,q1 and two short input words a,b, then add the displayed cycle loss so T_b T_a maps q0 to q1 and the reverse word maps q1 to q0. Also enumerate all states and generators after training, construct M by breadth-first closure under composition, and reject candidates with unwanted long cycles or excessive nonabsorbing states. The paper's exact algebraic contribution is the finite transformation-monoid closure and the Z2 restriction test; transition probabilities, temperature, and task loss are engineering choices. Estimate robustness by perturbing logits and measuring whether the decoded state remains in C. First experiment: train a 2-layer vanilla GRU and this 16- or 32-state algebraic cell on a delayed-copy task with distractor symbols. Compare accuracy, hidden-state corruption under noise, and number of training examples needed. Success means higher delayed-copy accuracy and a visibly stable two-state orbit, without increasing parameter count by more than 2x.

## Verification

- Status: queued_mechanism
- Mechanism evidence: no
- Mechanism confirmed: no

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.