# Offline quadratic latent operator

- ID: 3066
- Canonical URL: https://synthcore.org/idea/3066/offline-quadratic-latent-operator
- API JSON: https://synthcore.org/api/idea/3066.json
- API Markdown: https://synthcore.org/api/idea/3066.md
- Verification status: unverified
- Source: [arXiv:2609.02578](https://arxiv.org/abs/2609.02578)
- Category: architecture
- Solves: speedup, memory, scalability
- ML areas: world-model, graph-nn, inference-speedup
- Math tags: tensor-decomposition, linear-algebra, pde, approximation-theory
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Replace repeated high-dimensional quadratic interactions in a neural operator or world model with a reduced latent quadratic map whose basis-pair interactions are precomputed offline. The online computation becomes a small polynomial in latent coefficients, preserving quadratic interaction structure while avoiding repeated full-resolution contractions.

## Mathematical statement

The paper's collision operator Q(f,f) is quadratic in f. For a reduced representation f_rb=sum_{i=1}^n c_i u_i, where u_i are fixed basis functions and c_i are latent coefficients, bilinearity gives Q(f_rb,f_rb)=sum_{i,j} c_i c_j Q(u_i,u_j). Define G_{kij}=<u_k,Q(u_i,u_j)>, where k is the reduced output coordinate and <.,.> is the selected inner product. The reduced output is q_k(c)=sum_{i,j}G_{kij}c_i c_j. All expensive pairwise operator evaluations occur offline. Online inference only contracts the stored tensor G with c tensor c. A separable or low-rank approximation G_{kij} approximately equal to sum_{r=1}^R A_{kir}B_{rij} can reduce storage and computation; R is the retained factorization rank.

## Key formulas

- $$Q(f,f)(\mathbf{x},\mathbf{v})=\int_{\mathbb{R}^{d}}\int_{\mathbb{S}^{d-1}}B(\mathbf{v}-\mathbf{v}_{*},\sigma)[f(\mathbf{x},\mathbf{v}_{*}^{\prime})f(\mathbf{x},\mathbf{v}^{\prime})-f(\mathbf{x},\mathbf{v}_{*})f(\mathbf{x},\mathbf{v})]\\,\mathrm{d}{\sigma}\\,\mathrm{d}{\mathbf{v}_{*}}.$$
- $$f_{\mathrm{rb}}=\sum_{i=1}^{n}c_i u_i,\qquad Q(f_{\mathrm{rb}},f_{\mathrm{rb}})=\sum_{i=1}^{n}\sum_{j=1}^{n}c_i c_j Q(u_i,u_j).$$
- $$G_{kij}=\langle u_k,Q(u_i,u_j)\rangle,\qquad q_k(c)=\sum_{i=1}^{n}\sum_{j=1}^{n}G_{kij}c_i c_j.$$
- $$G_{kij}\approx\sum_{r=1}^{R}A_{kir}B_{rij},\qquad q_k(c)\approx\sum_{r=1}^{R}\sum_{i,j}A_{kir}B_{rij}c_i c_j.$$

## Implementation notes

(1) Integration point: add a quadratic latent operator after the encoder of a neural operator, graph simulator, or world model. The encoder maps an input field or graph state x to c in R^n. Replace a dense high-resolution pairwise interaction or MLP block with q(c), concatenate [c,q(c)], and pass the result to the decoder or next residual block. This changes both training and inference computation.

(2) Pseudocode:
```text
# Offline
for i in range(n):
  for j in range(n):
    h = expensive_quadratic_operator(u[i], u[j])
    for k in range(n):
      G[k,i,j] = inner_product(u[k], h)
optionally fit low-rank factors A,B to G

# Online
c = encoder(x)
if low_rank:
  t[r] = sum(i,j, B[r,i,j] * c[i] * c[j])
  q[k] = sum(r, A[k,r] * t[r])
else:
  q[k] = sum(i,j, G[k,i,j] * c[i] * c[j])
y = decoder(concat(c, q))
```
Use symmetry when justified, storing only i<=j and doubling off-diagonal terms. Initialize the quadratic block and its decoder projection near zero so a pretrained baseline is preserved initially. (3) The basis-pair expansion and offline interaction tensor are taken from the paper; the learned encoder, decoder, basis, and factor rank R are engineering choices. Estimate R by randomized SVD or Tucker decomposition of G, retaining 99% of interaction energy. (4) First experiment: train a small Fourier neural operator or graph simulator on a 2D parametric advection-reaction benchmark. Compare a standard MLP interaction, a dense quadratic interaction, and the precomputed reduced operator at matched latent width. Measure forward latency, peak memory, rollout error, and inverse-problem optimization time. Success is at least 2x lower inference cost or memory at matched rollout error, with stable gradients.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.