# Residual-Gated Lift Depth

- ID: 1085
- Canonical URL: https://synthcore.org/idea/1085/residual-gated-lift-depth
- API JSON: https://synthcore.org/api/idea/1085.json
- API Markdown: https://synthcore.org/api/idea/1085.md
- Verification status: mech_ok_no_baseline
- Source: [arXiv:2607.17664](https://arxiv.org/abs/2607.17664)
- Category: dynamics
- Solves: scalability, stability, accuracy
- ML areas: graph-nn, training-dynamics, scheduler, world-model
- Math tags: dynamical-systems, graph-theory, control-theory, numerical-analysis, linear-algebra
- Ratings: usefulness 7/10; difficulty 4/10; novelty 8/10

## Idea description

Use the paper's localized truncation residual as an online certificate for whether the current polynomial lift is expressive enough. Start with a low-degree edge lift and activate additional degree blocks or a learned closure only when the residual exceeds a calibrated threshold, avoiding the cost and instability of always using a large polynomial dictionary.

## Mathematical statement

Let $z_d$ be the order-$d$ graph-induced lift and let $K_d$ be its retained linear operator. The exact lifted derivative is $\dot z_d=K_dz_d+r_d$, with $r_d$ supported only in the final block and equal to $\mathcal L_{f_Q}\Phi_d$. During training or inference, estimate the residual using finite differences, $\widehat r_t=(z_{t+1}-z_t)/\Delta t-K_dz_t$, or by comparing degree-$d$ and degree-$(d+1)$ models. A normalized residual indicator is $\rho_t=\|\widehat r_t\|_2/(\|K_dz_t\|_2+\varepsilon)$. Under bounded state norm $\|x\|\le M$ and quadratic-field norm $\|f_Q(x)\|\le qM^2$, homogeneity gives the scaling bound $\|r_d(x)\|\le c_dqM^{d+1}$, where $c_d$ depends on the selected tensor dictionary and derivative multiplicities. This predicts that closure error grows as degree $d+1$ in state amplitude and is amplified by graph weight accumulation through $q$, motivating a threshold controller for lift depth.

## Key formulas

- $$\dot z_d=K_dz_d+r_d,\qquad r_d=\operatorname{col}(0,\ldots,0,D\Phi_d(x)f_Q(x)).$$
- $$\widehat r_t=\frac{z_{t+1}-z_t}{\Delta t}-K_dz_t,\qquad \rho_t=\frac{\|\widehat r_t\|_2}{\|K_dz_t\|_2+\varepsilon}.$$
- $$\|r_d(x)\|_2\le c_dqM^{d+1},\qquad \|x\|_2\le M,\quad \|f_Q(x)\|_2\le q\|x\|_2^2.$$
- $$d_{t+1}=\begin{cases}d_t+1,&\rho_t>\tau_{\mathrm{up}}\text{ for }m\text{ consecutive steps},\\d_t-1,&\rho_t<\tau_{\mathrm{down}}\text{ for }m\text{ steps},\\d_t,&\text{otherwise}.\end{cases}$$

## Implementation notes

1. Integration point: place the controller around a graph-lifted state-space model, latent ODE, or recurrent world model. Maintain degree-2 features initially, and pre-build sparse degree-3 and degree-4 feature maps so activation does not require recompiling the network. 2. Pseudocode: at each window, compute `z_d`; predict `dz_pred=K_d@z_d+closure(z_d)`; from the next observed state compute `r_hat=(z_next-z)/dt-K_d@z`; calculate `rho=norm(r_hat)/(norm(K_d@z)+eps)`; update an exponential moving average; if it exceeds `tau_up`, activate the next degree or increase closure width; if it remains below `tau_down`, prune the highest degree. Train with `L=L_prediction+lambda_r*mean(rho^2)` and optionally stop gradients through the discrete gate. 3. $K_d$ and the residual support pattern come from the lifting construction; $c_d$, $q$, measurement-noise variance, and thresholds are estimated empirically. Use hysteresis and a minimum dwell time to prevent degree chattering. 4. First cheap experiment: train on short trajectories from a 16-node SEIR graph and a randomly generated quadratic graph system, comparing fixed degree-2, fixed degree-3, and residual-gated models. Use identical decoders and optimizer settings. Test one-step error, 50-step rollout error, active feature count, and residual ratio. Calibrate $\tau_{\mathrm{up}}$ to the 95th percentile of normalized training residuals. The falsifiable signature is a sharp increase in rollout error when $\rho$ crosses the threshold, followed by recovery after degree activation; across state rescalings by factor $a$, the unnormalized residual should scale approximately as $a^{d+1}$, and across graph weight rescalings by factor $b$, it should scale approximately as $b$ in the weak-nonlinearity regime.

## Verification

- Status: mech_ok_no_baseline
- Mechanism evidence: yes
- Mechanism confirmed: yes
- Practical verdict: harms
- Verdict: Built a readable scalar quadratic lift MVP with exact truncated operators, residual certificate, hysteretic degree-2-to-3 gating, scaling checks, finite-difference verification, and fixed-degree rollout baselines. The mechanism manifested: residual norms scaled as a^(d+1) with observed exponents 3.000, 4.000, and 5.000 for degrees 2–4, scaled linearly with q with slope 1.000, and vanished at q=0; finite-difference relative error was below 3e-7. In the toy rollout, gating achieved RMSE 0.006175 versus 0.009882 for fixed degree 2 and 0.006287 for fixed degree 3 while averaging degree 2.98.

### Mechanism check

- Verdict: Built a readable scalar quadratic lift MVP with exact truncated operators, residual certificate, hysteretic degree-2-to-3 gating, scaling checks, finite-difference verification, and fixed-degree rollout baselines. The mechanism manifested: residual norms scaled as a^(d+1) with observed exponents 3.000, 4.000, and 5.000 for degrees 2–4, scaled linearly with q with slope 1.000, and vanished at q=0; finite-difference relative error was below 3e-7. In the toy rollout, gating achieved RMSE 0.006175 versus 0.009882 for fixed degree 2 and 0.006287 for fixed degree 3 while averaging degree 2.98.
- Confidence: 8/10
- Limitations: Only a one-dimensional quadratic dynamical system was tested, not the paper's 16-node SEIR graph or a learned neural closure. The certificate used noiseless Euler next states, thresholds were calibrated in the same toy regime, and no wall-clock, FLOP, or noisy-measurement robustness study was performed.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0324
- Paired wins: 2/8
- Benchmark verdict: idea worse (significant)

## Artifacts

- [bench_report.json](https://synthcore.org/code/435/bench_report.json)
- [bench_residual_gated.py](https://synthcore.org/code/435/bench_residual_gated.py)
- [report.md](https://synthcore.org/code/435/report.md)
- [report_bench_2026-08-31T121041.md](https://synthcore.org/code/435/report_bench_2026-08-31T121041.md)
- [residual_gated_lift.py](https://synthcore.org/code/435/residual_gated_lift.py)
- [Download all files as ZIP](https://synthcore.org/download/435)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.