Unverified 2026

Strang-Split Anisotropic Kernel Layer

Implementation & benchmark of arXiv:2608.18019 — Ordered Diffusion Kernels

Usefulness6/10
Difficulty6/10
Novelty8/10

Source paper: Ordered Diffusion Kernels arXiv:2608.18019 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Approximate anisotropic diffusion in a neural operator by composing several ordered local propagation steps rather than learning one unrestricted dense attention matrix. Each directional step uses its own ordering function and bandwidth, and symmetric composition reduces the leading splitting error.

Formulas

$$\mathcal{L}=\mathcal{L}_1+\mathcal{L}_2$$
$$P_{\mathrm{split}}=P_1(\delta/2)\,P_2(\delta)\,P_1(\delta/2)\ \approx\ e^{\delta(\mathcal{L}_1+\mathcal{L}_2)}$$
$$K^{(r)}_{ij}=\exp\!\left(-\frac{\|x_j-x_i\|^{2}}{4\varepsilon_r}+\beta_r[s_r(x_j)-s_r(x_i)]\right),\qquad P^{(r)}_{ij}=\frac{K^{(r)}_{ij}}{\sum_{k\in\mathcal{N}(i)}K^{(r)}_{ik}}$$
$$P_1(\delta/2)P_2(\delta)P_1(\delta/2)=e^{\delta(\mathcal{L}_1+\mathcal{L}_2)}+O(\delta^3)$$

Mathematical statement

The paper extends ordered diffusion kernels from gradient-flow SDEs with state-dependent isotropic diffusion to anisotropic diffusions through Strang splitting. Let $\mathcal{L}=\mathcal{L}_1+\mathcal{L}_2$ be two generator components, such as directional drift-diffusion operators associated with two ordering functions $s_1$ and $s_2$. If $P_r(\delta)$ approximates $e^{\delta\mathcal{L}_r}$, the symmetric composition $P_1(\delta/2)P_2(\delta)P_1(\delta/2)$ approximates $e^{\delta(\mathcal{L}_1+\mathcal{L}_2)}$ with local error $O(\delta^3)$ when the operators are sufficiently smooth and their commutators are bounded. For component $r$, use bandwidth $\varepsilon_r$, order strength $\beta_r$, and scalar ordering function $s_r(x)$ to define $K^{(r)}_{ij}=\exp[-\|x_j-x_i\|^2/(4\varepsilon_r)+\beta_r(s_r(x_j)-s_r(x_i))]$, followed by independent row normalization. Every factor satisfies $P_r\mathbf{1}=\mathbf{1}$, preserving constant features while allowing different components to represent distinct local transport and diffusion directions.

Implementation notes

Integrate this as a propagation block in a graph neural operator, point-cloud encoder, or latent world model. Replace one unrestricted attention layer by three sparse propagation sublayers: direction 1 half-step, direction 2 full-step, and direction 1 half-step. Each direction has a scalar MLP $s_r(x)$, order coefficient $\beta_r$, and bandwidth $\varepsilon_r$; value projections and feed-forward layers may be shared to keep parameter counts comparable. For each direction compute $\ell^{(r)}_{ij}=-\|x_j-x_i\|^2/(4\varepsilon_r)+\beta_r(s_r(x_j)-s_r(x_i))$, apply a neighbor-wise softmax to obtain $P_r$, and update with $h\leftarrow h+\alpha_r(P_rV(h)-h)$. Use $\alpha_1=\delta/2$, then $\alpha_2=\delta$, then repeat direction 1 with $\alpha_1$. For a direct semigroup interpretation, keep $\delta$ small; for ordinary deep-network training, tune residual scales and use normalization after the three-step block. The paper determines the symmetric composition and its $O(\delta^3)$ local error; training estimates $s_r$, $\beta_r$, and bandwidths. Measure noncommutativity through $\|P_1P_2-P_2P_1\|_F$ and clip logits before softmax. First test on a 2D advection-diffusion field with diffusion matrix $A=R(\phi)\operatorname{diag}(0.05,0.5)R(\phi)^T$, training a small point-cloud next-step predictor. Compare one isotropic ordered layer, sequential composition, and Strang composition at matched FLOPs. Evaluate one-step MSE, 50-step rollout error, and feature-norm growth. The expected signal is lower rollout error and reduced depth sensitivity for Strang splitting, especially when the principal diffusion direction changes spatially.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.