# RG-Decaying Rotational Residual Blocks

- ID: 1353
- Canonical URL: https://synthcore.org/idea/1353/rg-decaying-rotational-residual-blocks
- API JSON: https://synthcore.org/api/idea/1353.json
- API Markdown: https://synthcore.org/api/idea/1353.md
- Verification status: unverified
- Source: [arXiv:2607.24252](https://arxiv.org/abs/2607.24252)
- Category: dynamics
- Solves: stability, accuracy, generalization
- ML areas: training-dynamics, optimizer, rnn, ssm
- Math tags: renormalization, statistical-mechanics, dynamical-systems, control-theory
- Ratings: usefulness 6/10; difficulty 5/10; novelty 8/10

## Idea description

Construct a residual network with two coupled feature streams and deliberately non-reciprocal cross-stream interactions represented by a skew-symmetric coupling matrix. Decay the coupling strength with depth according to the RG picture of an irrelevant perturbation, allowing early layers to exploit rotational mixing while forcing deep layers toward reciprocal equilibrium-like dynamics. This should preserve transient expressivity without producing depth-dependent amplification or oscillatory degradation.

## Mathematical statement

The paper studies a complex multi-component Model-A equation. ψₐ is component a, Γ is a relaxation rate, r and D are reciprocal mass and diffusion coefficients, r′ and D′ are non-reciprocal imaginary coefficients, u and u′ are reciprocal and non-reciprocal nonlinear couplings, and ξₐ is noise. The RG result is that a dimensionless non-reciprocal strength g is irrelevant near the equilibrium fixed point: βg = dg divided by d ln b = −yg g + O(g²), with coarse-graining factor b greater than 1 and yg greater than 0. Therefore g(b) = g₀ b raised to −yg. In a neural depth flow, zₗ is a paired-stream state, Sₗ(zₗ) is the reciprocal learned drift, and Aₗ is a skew-symmetric coupling satisfying Aₗᵀ = −Aₗ. The update is zₗ₊₁ = zₗ + h[Sₗ(zₗ) + gₗ Aₗzₗ], with gₗ = g₀((l + l₀)/l₀) raised to −yg. Since zₗᵀAₗzₗ = 0, the skew term does not change squared norm at first order; its role is rotational mixing rather than direct energy growth.

## Key formulas

- ∂ₜψₐ = −Γ[(r + i r′)ψₐ − (D + i D′)∇²ψₐ + (u + i u′)(ψ†ψ)ψₐ divided by 3] + ξₐ
- βg = dg divided by d ln b = −yg g + O(g²), with yg > 0, and therefore g(b) = g₀ b^(−yg)
- zₗ₊₁ = zₗ + h[Sθ(zₗ,l) + gₗ Aθ zₗ], with Aₗᵀ = −Aₗ and gₗ = g₀((l + l₀)/l₀)^(−yg)
- (||zₗ₊₁||² − ||zₗ||²) divided by h = 2 zₗᵀ Sₗ(zₗ) + 2 gₗ zₗᵀ Aₗ zₗ + O(h) = 2 zₗᵀ Sₗ(zₗ) + O(h)

## Implementation notes

1. Integration point: replace each residual block in a small ResNet, deep MLP, or state-space sequence model with a paired-stream block. Let zₗ = [xₗ; yₗ], where xₗ and yₗ have equal width. Use the ordinary learned residual transformation as Sθ(zₗ,l), and add a cross-stream term Aθ zₗ. Parameterize Aθ as Bθ − Bθᵀ, or use the block matrix [[0,Bθ], [−Bθᵀ,0]], guaranteeing skew symmetry. 2. Pseudocode: initialize g₀ > 0, l₀ = 1, and yg > 0. At layer l, compute g = g₀((l + l₀)/l₀)^(−yg), compute s = Sθ(z,l), compute a = Aθ(z)z, and update z ← z + h(s + g a). 3. Computed versus estimated: the complex non-reciprocal dynamics and the RG irrelevance direction yg > 0 are taken from the paper. The numerical exponent for a finite-width neural network is not supplied and should initially be treated as a hyperparameter; it can later be estimated by fitting the measured cross-stream Jacobian norm. 4. First experiment: train 20-layer and 100-layer paired-stream ResNets on CIFAR-10, comparing gₗ = 0, constant gₗ = g₀, and decaying schedules with yg in {0.1, 0.25, 0.5}. Include frozen random linear blocks to isolate dynamics from representation learning. Measure activation norms, Jacobian singular values, cross-stream Jacobian norms, and accuracy versus depth. The prediction is that log gₗ versus log((l + l₀)/l₀) has slope −yg, within roughly 20 percent after fitting. Deep-layer activation and Jacobian statistics should converge toward the gₗ = 0 baseline, whereas constant-g models should show larger oscillatory variance or instability as depth increases.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.