RG-Decaying Rotational Residual Blocks

1. Integration point: replace each residual block in a small ResNet, deep MLP, or state-space sequence model with a paired-stream block. Let zₗ = [xₗ; yₗ], where xₗ and yₗ have equal width. Use the ordinary learned residual transformation as Sθ(zₗ,l), and add a cross-stream term Aθ zₗ. Parameterize Aθ as Bθ − Bθᵀ, or use the block matrix [[0,Bθ], [−Bθᵀ,0]], guaranteeing skew symmetry. 2. Pseudocode: initialize g₀ > 0, l₀ = 1, and yg > 0. At layer l, compute g = g₀((l + l₀)/l₀)^(−yg), compute s = Sθ(z,l), compute a = Aθ(z)z, and update z ← z + h(s + g a). 3. Computed versus estimated: the complex non-reciprocal dynamics and the RG irrelevance direction yg > 0 are taken from the paper. The numerical exponent for a finite-width neural network is not supplied and should initially be treated as a hyperparameter; it can later be estimated by fitting the measured cross-stream Jacobian norm. 4. First experiment: train 20-layer and 100-layer paired-stream ResNets on CIFAR-10, comparing gₗ = 0, constant gₗ = g₀, and decaying schedules with yg in {0.1, 0.25, 0.5}. Include frozen random linear blocks to isolate dynamics from representation learning. Measure activation norms, Jacobian singular values, cross-stream Jacobian norms, and accuracy versus depth. The prediction is that log gₗ versus log((l + l₀)/l₀) has slope −yg, within roughly 20 percent after fitting. Deep-layer activation and Jacobian statistics should converge toward the gₗ = 0 baseline, whereas constant-g models should show larger oscillatory variance or instability as depth increases.