# Branched Rough Residual Block

- ID: 2997
- Canonical URL: https://synthcore.org/idea/2997/branched-rough-residual-block
- API JSON: https://synthcore.org/api/idea/2997.json
- API Markdown: https://synthcore.org/api/idea/2997.md
- Verification status: unverified
- Source: [arXiv:2609.01190](https://arxiv.org/abs/2609.01190)
- Category: dynamics
- Solves: accuracy, stability, sample-efficiency
- ML areas: ssm, rnn, training-dynamics
- Math tags: stochastic-processes, dynamical-systems, geometry, algebra, numerical-analysis
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Replace a standard recurrent or neural-CDE Euler transition with a second-order rough transition that receives both first-order increments of the input path and learned second-order branched increments. Unlike a geometric signature block, the second-order coefficients are independent learned maps rather than being forced to equal derivatives or shuffle-symmetric combinations of first-order vector fields, allowing the model to represent order-sensitive and non-geometric interactions in irregular data.

## Mathematical statement

The paper defines a branched rough path X with first-level increments X^\alpha(s,t) and second-level increments X^{\alpha\beta}(s,t), where \alpha,\beta\in\{1,\ldots,n\} index driver channels. A control function \omega satisfies bounds of the form |X^\alpha(s,t)|\leq C\omega(s,t)^{1/p} and |X^{\alpha\beta}(s,t)|\leq C\omega(s,t)^{2/p}, for roughness p\in[1,3). A Davie solution is characterized locally by Y(t)-Y(s)=f_\alpha(Y(s))X^\alpha(s,t)+f_{\alpha\beta}(Y(s))X^{\alpha\beta}(s,t)+R(s,t), with |R(s,t)|\leq C'\omega(s,t)^{3/p}. Here Y(t)\in\mathbb{R}^d is the hidden state, f_\alpha:\mathbb{R}^d    o\mathbb{R}^d are first-order rough velocity fields, f_{\alpha\beta}:\mathbb{R}^d    o\mathbb{R}^d are second-order branched coefficients, and R is the truncation error. For non-geometric X, f_{\alpha\beta} cannot generally be replaced by a derivative composition such as Df_\beta f_\alpha; this independent coefficient is the additional modeling capacity. The paper's associative jet-rough-velocity composition (ab)f=a(bf) motivates composing these coefficients consistently across coordinate transforms or stacked blocks.

## Key formulas

- $$\left|X^\alpha(s,t)\right|\leq C\,\omega(s,t)^{\frac{1}{p}},\qquad \left|X^{\alpha\beta}(s,t)\right|\leq C\,\omega(s,t)^{\frac{2}{p}},\qquad p\in[1,3).$$
- $$Y(t)-Y(s)=f_\alpha(Y(s))X^\alpha(s,t)+f_{\alpha\beta}(Y(s))X^{\alpha\beta}(s,t)+R(s,t),\qquad |R(s,t)|\leq C'\omega(s,t)^{\frac{3}{p}}.$$
- $$y_{k+1}=y_k+F_1(y_k)\Delta x_k+F_2(y_k):\Delta x_k^{(2)},\qquad \Delta x_k^{(2),\alpha\beta}=\Delta x_k^\alpha\Delta x_k^\beta+A_k^{\alpha\beta},$$
- $$(ab)f=a(bf).$$

## Implementation notes

Integrate the block as a replacement for the recurrent update in a neural CDE, continuous-time RNN, or lightweight state-space model. The driver is an input sequence x_k\in\mathbb{R}^n observed at times t_k, and the hidden state is y_k\in\mathbb{R}^d. First normalize each channel and compute \Delta x_k=x_{k+1}-x_k. Construct a second-order tensor with symmetric products \Delta x_k^\alpha\Delta x_k^\beta and an optional order-sensitive area proxy A_k^{\alpha\beta}=0.5(\Delta x_k^\alpha\Delta x_{k+1}^\beta-\Delta x_k^\beta\Delta x_{k+1}^\alpha); set A_k=0 for the simplest baseline. Let F_1 be an MLP producing a d    imes n matrix and F_2 an MLP producing a d    imes n    imes n tensor. Apply the update y_{k+1}=y_k+F_1(y_k)\Delta x_k+F_2(y_k):\Delta x_k^{(2)}, where ':' contracts the two driver-channel indices. Normalize \Delta x_k^{(2)} by its batch RMS to prevent second-order terms from dominating. Estimate the local control as \widehat{\omega}_k=\|\Delta x_k\|^p+\|\Delta x_k^{(2)}\|^{p/2}; if \widehat{\omega}_k exceeds a threshold, subdivide the interval or apply two half-steps, implementing the paper's local-expansion regime. Stack the block with shared transformation rules if using coordinate changes. The cheap first experiment should use d=64 on PhysioNet irregular time-series or synthetic fractional-Brownian/noisy streams, comparing a GRU and neural-CDE Euler block at matched parameters and FLOPs. Measure validation NLL or accuracy, gradient-norm variance, and sensitivity to random timestamp thinning. Success means lower error at equal compute, fewer exploding trajectories, and better degradation under irregular sampling.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.