Unverified 2026

Analytic KL spatial adapter

Implementation & benchmark of arXiv:2607.12387 — Hierarchical Bayesian inversion using the Karhunen-Loève expansion with analytical eigenpairs of the squared exponential kernel

Usefulness6/10
Difficulty5/10
Novelty6/10

Source paper: Hierarchical Bayesian inversion using the Karhunen-Loève expansion with analytical eigenpairs of the squared exponential kernel arXiv:2607.12387 ⓘ · analyzed Aug 30, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Replace a dense spatial parameter field in a neural field or convolutional adapter by a truncated squared-exponential KL expansion with analytic Gaussian-Hermite modes. The amplitude and correlation length remain trainable, but changing them only rescales coefficients and basis parameters instead of triggering a numerical eigensolve.

Formulas

$$(T_K f)(\mathbf{x})=\int_X K(\mathbf{x},\mathbf{x}')f(\mathbf{x}')\,\mathrm d\nu(\mathbf{x}'),$$
$$T_K\phi_i=\lambda_i\phi_i,\qquad \lambda_1\geq\lambda_2\geq\cdots\geq0,$$
$$K_{\sigma,\ell}(x,x')=\sigma^2\exp\!\left(-\frac{\|x-x'\|^2}{2\ell^2}\right),\qquad w(x)=m(x)+\sum_{\alpha\in A_M}\sqrt{\lambda_\alpha(\sigma,\ell,s)}\,z_\alpha\phi_\alpha(x;s,\ell),\quad z_\alpha\sim\mathcal N(0,1),$$
$$\widehat E_M(s)=\frac{1}{|B|}\sum_{x\in B}\left[K_{\sigma,\ell}(x,x)-\sum_{\alpha\in A_M}\lambda_\alpha(\sigma,\ell,s)\phi_\alpha(x;s,\ell)^2\right].$$

Mathematical statement

Mercer's construction defines the covariance operator $(T_K f)(x)=\int_X K(x,x')f(x')\,d\nu(x')$ for a continuous symmetric positive-semidefinite kernel $K$ satisfying $\int_X\int_X|K(x,x')|^2d\nu(x)d\nu(x')<\infty$. Its eigenpairs satisfy $T_K\phi_i=\lambda_i\phi_i$, with orthogonal eigenfunctions and non-increasing nonnegative eigenvalues. For the squared-exponential kernel $K_{\sigma,\ell}(x,x')=\sigma^2\exp[-\|x-x'\|^2/(2\ell^2)]$ under a Gaussian weight $\nu_s=\mathcal N(0,s^2I)$, use the paper's analytic eigenpair family in the implementation form $\phi_{\alpha}(x;s,\ell)=\prod_{j=1}^d h_{\alpha_j}(x_j;\rho)$ and $\lambda_{\alpha}(\sigma,\ell,s)=\sigma^2\prod_{j=1}^d c(\ell,s)q(\ell,s)^{\alpha_j}$, where $\alpha\in\mathbb N^d$ is a multi-index, $h_n(\cdot;\rho)$ is the normalized Gaussian-Hermite function of width $\rho$, and $0\le q(\ell,s)<1$ is the analytic geometric eigenvalue-decay factor of the squared-exponential/Gaussian-weight eigenproblem. The adapted field is $w(x)=m(x)+\sum_{\alpha\in A_M}\sqrt{\lambda_\alpha}z_\alpha\phi_\alpha(x)$ with independent $z_\alpha\sim\mathcal N(0,1)$. Select the Gaussian weight width $s$ by minimizing the empirical truncation proxy $\widehat E_M(s)=|B|^{-1}\sum_{x\in B}[K(x,x)-\sum_{\alpha\in A_M}\lambda_\alpha\phi_\alpha(x)^2]$.

Implementation notes

(1) Integration point: implement a spatially varying additive adapter inside a neural field, CNN, or coordinate MLP. For an input coordinate $x\in\mathbb R^d$, compute an adapter scalar or channel vector $a(x)=W_zf_M(x)$, where $f_M(x)$ contains the first $M$ tensor-product Gaussian-Hermite modes; add $a(x)$ to the feature map or use it to modulate a convolutional or MLP weight. Store only $M\times C$ coefficients rather than one parameter per pixel or voxel. (2) Pseudocode: choose multi-indices $A_M$ by increasing total degree; initialize trainable $\log\sigma$, $\log\ell$, and optionally $\log s$; for every batch coordinate $x$, compute normalized Hermite functions using $h_0(u)=\pi^{-1/4}e^{-u^2/2}$, $h_1(u)=\sqrt2u h_0(u)$, and $h_{n+1}(u)=\sqrt{2/(n+1)}u h_n(u)-\sqrt{n/(n+1)}h_{n-1}(u)$; form $\phi_\alpha(x)=\prod_jh_{\alpha_j}(x_j;\rho)$; evaluate the analytic paper eigenvalue routine $\lambda_\alpha(\sigma,\ell,s)$; and return $a(x)=\sum_\alpha\sqrt{\lambda_\alpha}z_\alpha\phi_\alpha(x)$. (3) The analytic eigenpair expressions and derivatives with respect to $\sigma,\ell,s$ are computed by automatic differentiation; estimate $s$ by minimizing $\widehat E_M$ on a fixed coordinate batch, and enforce positivity with softplus. Do not numerically diagonalize a kernel matrix during training. (4) First experiment: fit a $64\times64$ image-coordinate MLP and a small CIFAR-10 convolutional model with a spatial bias field. Compare a dense per-pixel bias, learned Fourier features, and this KL adapter at equal trainable parameter count. Measure validation loss, high-frequency energy, parameter memory, and wall-clock time while sweeping $M\in\{8,16,32,64\}$. Success is lower validation error or comparable error with at least 4x fewer spatial parameters, smooth degradation as $M$ decreases, and no eigensolve cost when $\ell$ changes.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.