# Reversible Two-Relaxation Recurrent Block

- ID: 2877
- Canonical URL: https://synthcore.org/idea/2877/reversible-two-relaxation-recurrent-block
- API JSON: https://synthcore.org/api/idea/2877.json
- API Markdown: https://synthcore.org/api/idea/2877.md
- Verification status: unverified
- Source: [arXiv:2608.30253](https://arxiv.org/abs/2608.30253)
- Category: dynamics
- Solves: stability, generalization, scalability
- ML areas: rnn, ssm, training-dynamics, initialization
- Math tags: dynamical-systems, linear-algebra, geometry, numerical-analysis, spectral-theory
- Ratings: usefulness 6/10; difficulty 5/10; novelty 5/10

## Idea description

Replace the recurrent transition or state-space mixer with a reversible transport followed by complementary relaxation of symmetric and antisymmetric feature components. The construction preserves a weighted energy and damps both parity sectors, giving bounded long-horizon powers without requiring the learned transition matrix itself to be symmetric. A numerical-range ellipse can be used as a cheap training-time certificate against transient growth.

## Mathematical statement

Let f_t in R^(N x d x q) be populations at N positions, with q paired velocity or channel directions and feature dimension d. Let R be the involution that swaps every direction with its opposite, so R^2 = I. Define parity projectors P_+ = (I+R)/2 and P_- = (I-R)/2. The two-relaxation collision is C = I - s_+ P_+ - s_- P_-, where s_+ = 2 - s_- and 0 < s_- < 2; therefore both parity eigenvalues 1-s_+ and 1-s_- lie in (-1,1). Let U be the transport or mixing operator and H be a positive-definite equilibrium metric. Require metric unitarity U^T H U = H and reversibility R U R = U^(-1). The recurrent transition is A = U C. The paper reduces the dynamics to a two-step recurrence generated by a contraction and bounds the associated companion operator using its numerical range W(B) = {x* B x : ||x||_2 = 1}. If W(B) is contained in an ellipse E, the Crouzeix–Palencia inequality gives ||p(B)||_2 <= (1+sqrt(2)) max_{z in E} |p(z)| for every polynomial p. In the simplest neural implementation H = I; a diagonal H can model feature-dependent scales.

## Key formulas

- $$P_{+}=\frac{I+R}{2},\qquad P_{-}=\frac{I-R}{2},\qquad C=I-s_{+}P_{+}-s_{-}P_{-},\qquad s_{+}+s_{-}=2,\quad 0<s_{-}<2.$$
- $$U^{\mathsf T}HU=H,\qquad RUR=U^{-1},\qquad A=UC.$$
- $$W(B)=\{x^{*}Bx:\|x\|_{2}=1\}\subseteq E\quad\Longrightarrow\quad \|p(B)\|_{2}\le(1+\sqrt{2})\max_{z\in E}|p(z)|.$$
- $$\|x\|_{H}^{2}=x^{\mathsf T}Hx,\qquad \|A^{n}\|_{H}=\sup_{x\ne0}\frac{\|A^{n}x\|_{H}}{\|x\|_{H}}.$$

## Implementation notes

Integrate this as the recurrent transition in a small RNN, linear state-space model, or sequence mixer. Store the hidden state with shape [batch, positions, directions, features], and arrange directions in opposite pairs. Implement R as a fixed channel permutation. For every state f, compute f_plus = (f + R(f))/2 and f_minus = (f - R(f))/2, then apply complementary relaxation: g = f - s_plus*f_plus - s_minus*f_minus. Parameterize s_minus = 2*sigmoid(a) and set s_plus = 2 - s_minus, either globally or per feature. Apply U after relaxation. An exact low-cost U can be a fixed cyclic shift over positions followed by learned orthogonal feature mixing; parameterize the mixer with Householder reflections or obtain its orthogonal factor using QR retraction after each optimizer step. The recurrent update is f_next = U(g) + B x_t, followed by a nonlinear readout outside the recurrent transition. For diagnostics, estimate ||A^k|| by automatic-differentiation Jacobian-vector products and 10–20 power iterations. Approximate the numerical range by sampling normalized random vectors v_j and recording v_j^* A v_j; fit an empirical ellipse and penalize samples outside the unit disk or with ellipse radius above one. The exact mathematical ingredients are the parity split, complementary rates, orthogonality, and reversibility; ellipse parameters are estimated empirically. First test on a 1,000-step copy task and Tiny Shakespeare using a 2-layer, 128-dimensional model. Compare vanilla tanh RNN, orthogonal RNN, and this block at matched parameter count and FLOPs. Record gradient norm versus sequence length, maximum hidden-state amplification, validation loss, and time-to-target loss. Success means bounded amplification and non-diverging gradients at lengths where the vanilla RNN fails, with equal or lower perplexity and less than 10% extra compute.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.