# Local Hermite Action Surrogate

- ID: 2666
- Canonical URL: https://synthcore.org/idea/2666/local-hermite-action-surrogate
- API JSON: https://synthcore.org/api/idea/2666.json
- API Markdown: https://synthcore.org/api/idea/2666.md
- Verification status: unverified
- Source: [arXiv:2608.21467](https://arxiv.org/abs/2608.21467)
- Category: optimization
- Solves: speedup, accuracy, sample-efficiency
- ML areas: rl, world-model, optimizer, diffusion-sampling
- Math tags: approximation-theory, numerical-analysis, optimization, linear-algebra
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Approximate an expensive neural objective as a local second-order Hermite polynomial over a symmetric action stencil, then optimize the fitted polynomial rather than repeatedly evaluating the original objective. Unlike a Taylor model, the coefficients are obtained from function values and do not require reliable action derivatives through a simulator or learned environment.

## Mathematical statement

Let $J(a)$ be a scalar objective for a continuous action $a\in\mathbb{R}^{m}$, and let $a_0$ be the current nominal action. Choose a spacing $h_r>0$ for each coordinate and evaluate $J$ on a local stencil. A second-order Hermite surrogate has the form $\widehat J(a_0+u)=\alpha+\sum_r\beta_ru_r+\sum_r\gamma_ru_r^2+\sum_{r<s}\delta_{rs}u_ru_s$, where $u=a-a_0$. The coefficients are fitted by weighted least squares from direct objective evaluations: $\theta=(\alpha,\beta,\gamma,\delta)=\arg\min_\theta\sum_q\rho_q(\phi(u_q)^\top\theta-J(a_0+u_q))^2$. Here $\phi(u)$ contains one constant, $m$ linear features, $m$ diagonal quadratic features, and $m(m-1)/2$ cross features; $\rho_q$ are optional observation weights. In one dimension, symmetric evaluations give $\beta\approx[J(a_0+h)-J(a_0-h)]/(2h)$ and $\gamma\approx[J(a_0+h)-2J(a_0)+J(a_0-h)]/(2h^2)$. The transferable property is local polynomial interpolation from function values, which can be more robust than derivative-based Taylor expansion for noisy, nonsmooth, or black-box rollout objectives.

## Key formulas

- $$\widehat J(a_0+u)=\alpha+\sum_{r=1}^{m}\beta_ru_r+\sum_{r=1}^{m}\gamma_ru_r^2+\sum_{1\le r<s\le m}\delta_{rs}u_ru_s,\qquad u=a-a_0.$$
- $$\theta^{\star}=\arg\min_{\theta}\sum_{q=1}^{Q}\rho_q\left(\phi(u_q)^{\top}\theta-J(a_0+u_q)\right)^2.$$
- $$\beta_r\approx\frac{J(a_0+h_re_r)-J(a_0-h_re_r)}{2h_r},\qquad \gamma_r\approx\frac{J(a_0+h_re_r)-2J(a_0)+J(a_0-h_re_r)}{2h_r^2}.$$
- $$a_{\mathrm{new}}=\operatorname{clip}_{\mathcal A}\!\left(a_0+\arg\min_{u\in\mathcal U}\widehat J(a_0+u)\right).$$

## Implementation notes

Use this as a derivative-free local planner around a policy action or as an inner optimizer for a learned world-model objective. At each replanning step, obtain the nominal action a0 from the policy. Select coordinate radii h_r and evaluate the expensive objective J at a0, at a0 plus and minus h_r e_r, and, when cross terms are required, at a0 plus signed combinations of two coordinate directions. Build each feature vector phi(u)=[1,u_1,...,u_m,u_1^2,...,u_m^2,u_1u_2,...,u_{m-1}u_m]. Solve the small weighted least-squares system (Phi^T R Phi+lambda I)theta=Phi^T R y, where y contains direct objective values, R contains confidence or recency weights, and lambda is a ridge stabilizer. Optimize the fitted quadratic over a bounded trust region U using projected gradient descent, enumeration for one or two dimensions, or a box-constrained quadratic solver. If the fitted Hessian is indefinite, retain the trust-region minimizer or add diagonal damping eta I. The formulas determine coefficient estimation; model mismatch, trust-region radius, and damping are tuned empirically. First test on Pendulum or a 2D synthetic control task with a neural dynamics model, comparing nine objective evaluations against finite-difference Taylor optimization and random shooting. Measure objective regret after replanning, action quality, and wall-clock time. Success is lower regret at equal evaluations, particularly when rollout noise or imperfect gradients destabilizes the Taylor baseline.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.