# Online Taylor Residual World Model

- ID: 3100
- Canonical URL: https://synthcore.org/idea/3100/online-taylor-residual-world-model
- API JSON: https://synthcore.org/api/idea/3100.json
- API Markdown: https://synthcore.org/api/idea/3100.md
- Verification status: unverified
- Source: [arXiv:2609.03351](https://arxiv.org/abs/2609.03351)
- Category: dynamics
- Solves: stability, accuracy, sample-efficiency
- ML areas: world-model, rl, optimizer
- Math tags: dynamical-systems, control-theory, optimization, linear-algebra
- Ratings: usefulness 8/10; difficulty 5/10; novelty 6/10

## Idea description

Augment a neural dynamics model with a sparse local Taylor residual whose coefficients are updated online by recursive least squares. Use the neural model for global behavior and the Taylor model for short-horizon prediction, where local adaptation can correct payload, friction, actuator, or environment changes without retraining the network.

## Mathematical statement

For observed transitions $x_{k+1}=F(x_k,u_k)+\epsilon_k$, define deviations $\delta x=x-x_0$ and $\delta u=u-u_0$ around an operating point $(x_0,u_0)$. Let $z=[\delta x^\top,\delta u^\top]^\top$ and let $\phi_r(z)$ contain every monomial of total degree at most $r$. The local residual model is $\hat{x}_{k+1}=\hat{F}_{\theta}(x_k,u_k)+W_k\phi_r(z_k)$, where $\hat{F}_{\theta}$ is a neural predictor and $W_k$ is an online coefficient matrix. For each output row, recursive least squares uses covariance $P_k$, coefficient row $w_k$, forgetting factor $\lambda_k\in(0,1]$, gain $K_k$, and residual target $y_k=x_{k+1}-\hat{F}_{\theta}(x_k,u_k)$. The paper's horizon approximation freezes the local Jacobian $A_k$ and input matrix $B_k$ during an $H$-step rollout. The mechanism is time-scale separation: fast local adaptation handles mismatch while the neural model remains a slow global prior.

## Key formulas

- $$\dot p=v,\qquad m\dot v=-mge_3+fRe_3,\qquad \dot R=R\omega^{\times},\qquad J\dot\omega=-\omega^{\times}J\omega+\tau.$$
- $$\phi_r(z)=\left[1,\{z_i\}_{|\alpha|=1},\{z^{\alpha}\}_{2\leq |\alpha|\leq r}\right]^{\top},\qquad z=\begin{bmatrix}x-x_0\\u-u_0\end{bmatrix}.$$
- $$\hat{x}_{k+1}=\hat{F}_{\theta}(x_k,u_k)+W_k\phi_r(z_k),\qquad A_k=\left.\frac{\partial\hat{x}_{k+1}}{\partial x_k}\right|_{(x_0,u_0)},\qquad B_k=\left.\frac{\partial\hat{x}_{k+1}}{\partial u_k}\right|_{(x_0,u_0)}.$$
- $$K_k=\frac{P_{k-1}\phi_k}{\lambda_k+\phi_k^{\top}P_{k-1}\phi_k},\qquad W_k=W_{k-1}+K_k\left(y_k-W_{k-1}\phi_k\right)^{\top},\qquad P_k=\frac{P_{k-1}-K_k\phi_k^{\top}P_{k-1}}{\lambda_k}.$$

## Implementation notes

1. Integration point: attach this module to a pretrained neural dynamics model used by a Dyna agent, MPC planner, or trajectory optimizer. Initially freeze the neural parameters. Maintain an operating point $(x_0,u_0)$, construct monomials through degree $r=1$, $2$, or $3$, and maintain one RLS covariance and coefficient matrix per output dimension. 2. Pseudocode: observe $(x_k,u_k,x_{k+1})$; compute $z_k=[x_k-x_0,u_k-u_0]$ and $\phi_k=\phi_r(z_k)$; calculate residual target $y_k=x_{k+1}-\hat{F}_{\theta}(x_k,u_k)$; apply the displayed RLS update; predict with $\hat{F}_{\theta}+W_k\phi_r$; calculate $A_k$ and $B_k$ by automatic differentiation; freeze them for the next $H$ planning steps; recenter $(x_0,u_0)$ when $\|z_k\|$ exceeds a threshold. 3. Computed directly: monomials, RLS gains, covariance, and Jacobians. Estimated empirically: the forgetting schedule, covariance initialization, recentering threshold, and useful Taylor order. 4. First cheap experiment: train a small MLP world model on CartPole or Pendulum, then inject an abrupt mass or actuator-scale change. Compare neural-only, order-1, order-2, and order-3 residual models inside an MPC controller with horizon $H=10$. Measure one-step error, ten-step error, rollout divergence, and recovery time. The quantitative prediction is that effective adaptation memory is approximately $N_{\mathrm{eff}}\approx1/(1-\lambda)$: recovery time should scale with this quantity, while stationary variance should increase as $\lambda$ decreases. Degree two should reduce local multi-step error over degree one only within a neighborhood where its quadratic term exceeds measurement noise; degree three should help only in a smaller, sufficiently excited neighborhood.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.