# Asymptotically Commuting Recurrent Blocks

- ID: 2428
- Canonical URL: https://synthcore.org/idea/2428/asymptotically-commuting-recurrent-blocks
- API JSON: https://synthcore.org/api/idea/2428.json
- API Markdown: https://synthcore.org/api/idea/2428.md
- Verification status: unverified
- Source: [arXiv:2608.16917](https://arxiv.org/abs/2608.16917)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: rnn, ssm, training-dynamics, regularization
- Math tags: dynamical-systems, control-theory, geometry
- Ratings: usefulness 5/10; difficulty 5/10; novelty 7/10

## Idea description

Replace a time-homogeneous recurrent update by a sequence of parameterized maps f_t, and regularize late-time pairs of updates to approximately commute: applying block f_t followed by f_r should agree with applying f_r followed by f_t. This should make long-horizon predictions robust to local time-step reorderings and schedule perturbations, while proximal statistics provide a diagnostic for whether trajectories repeatedly approach one another rather than diverging permanently.

## Mathematical statement

Let (X,d) be the hidden-state metric space and f_t: X -> X the update at time t. Define the non-autonomous evolution map omega_n = f_n composed with f_(n-1) composed through f_1. The sequence is pointwise asymptotically commutative if, for every fixed r and x in X, d(f_n(f_r(x)), f_r(f_n(x))) tends to zero as n tends to infinity; a uniform version takes the supremum over x in a bounded hidden-state set. A pair x,y is proximal when the limit inferior of d(omega_n(x), omega_n(y)) is zero. Transfer these relations by minimizing a sampled late-time commutator loss while monitoring, but not blindly maximizing, the proximal recurrence statistic. The commutator controls order sensitivity; the proximal statistic detects recurrent near-collisions and must be paired with an anti-collapse constraint.

## Key formulas

- $$\omega_n(x)=f_n\circ f_{n-1}\circ\cdots\circ f_1(x)$$
- $$c_{n,r}=\sup_{x\in\mathcal{B}}d\left(f_n(f_r(x)),f_r(f_n(x))\right),\qquad c_{n,r}\to 0\text{ as }n\to\infty$$
- $$\mathcal{L}_{\mathrm{comm}}=\frac{1}{|\mathcal{S}|}\sum_{(n,r,x)\in\mathcal{S}}w_n\left\|f_n(f_r(x))-f_r(f_n(x))\right\|_2^2,\qquad w_n=\left(\frac{n}{T}\right)^p$$
- $$P_\varepsilon=\frac{1}{|\mathcal{P}|}\sum_{(x,y)\in\mathcal{P}}\mathbf{1}\left[\min_{n\in\{T-H+1,\ldots,T\}}d(\omega_n(x),\omega_n(y))<\varepsilon\right],\qquad \operatorname{Var}(h_T)\geq v_{\min}$$

## Implementation notes

Use a recurrent, state-space, or deep residual model with time-dependent blocks h_t = f_t(h_(t-1), u_t). For a first test, let f_t be a shared MLP with a low-rank time embedding or learned step-size modulation, making the model genuinely non-autonomous. Add lambda_c times L_comm to the task loss and compute it on hidden states from the current minibatch. Sample r from the first half of a sequence and n from the last quarter, with n different from r. Use normalized hidden states or a bounded projection when evaluating the commutator. Pseudocode is: sample hidden states x and indices n,r; compute a = f_n(f_r(x)); compute b = f_r(f_n(x)); set comm = mean squared error between a and b; optimize task_loss + lambda_c times (n/T)^p times comm. During evaluation, run the normal sequence and locally transposed sequences in which selected update pairs are swapped, then measure prediction disagreement. Compute the two compositions exactly; estimate the supremum c_(n,r) with a maximum or 95th percentile over sampled states. Estimate P_epsilon from validation pairs, but enforce a covariance-eigenvalue floor v_min to prevent trivial representation collapse. First experiment: train a GRU or residual SSM on sequential MNIST, permuted sequential MNIST, and a synthetic switching linear system. Compare the baseline, commutator regularization, and random-order augmentation. Sweep lambda_c. The prediction is that the late-time commutator curve decreases with n, approximately as a power law n^(-alpha) or toward a plateau, and that reordered-versus-normal prediction error decreases with the measured commutator. Require at least a twofold reduction in median late-time commutator and a 20 percent reduction in order-perturbation error without reducing hidden-state covariance rank. If P_epsilon rises while covariance collapses, the experiment falsifies the intended mechanism.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.