# Separable Tensor-Product Spline Trial Layer

- ID: 1189
- Canonical URL: https://synthcore.org/idea/1189/separable-tensor-product-spline-trial-layer
- API JSON: https://synthcore.org/api/idea/1189.json
- API Markdown: https://synthcore.org/api/idea/1189.md
- Verification status: unverified
- Source: [arXiv:2607.20378](https://arxiv.org/abs/2607.20378)
- Category: architecture
- Solves: scalability, memory, accuracy
- ML areas: mlp, embedding, memory, initialization
- Math tags: approximation-theory, tensor-decomposition, linear-algebra
- Ratings: usefulness 6/10; difficulty 5/10; novelty 6/10

## Idea description

Use the paper's correspondence between KAN splines and finite-element or isogeometric shape functions to build coordinate-separable tensor-product trial layers. Replace additive coordinate aggregation with a multiplicative contraction of one-dimensional spline expansions, yielding an explicit tensor-product basis without storing a dense multidimensional grid.

## Mathematical statement

A KAN scalar map uses the univariate B-spline expansion \(\phi_{ij}(x_j)=\sum_{m=1}^{G+r}c_m^{(i,j)}B_m(x_j)\), where \(B_m\) is the order-\(r\) spline basis, \(G\) is grid size, and \(c_m^{(i,j)}\) are trainable coefficients. The paper explains that in two dimensions the additive construction \(K(x,y)=\sum_iN_i^x(x)c_i^x+\sum_jN_j^y(y)c_j^y\) does not reproduce a tensor-product finite-element field. The multiplicative construction does: \(K_{mult}(x,y)=(\sum_iN_i^x(x)c_i^x)(\sum_jN_j^y(y)c_j^y)=\sum_{i,j}N_i^x(x)N_j^y(y)c_i^xc_j^y\). Here \(N_i^x,N_j^y\) are one-dimensional shape functions and \(c_i^x,c_j^y\) are factor coefficients. A sum of \(R\) such products gives a low-rank approximation to a general non-separable field, with parameter cost proportional to \(R D G\) rather than \(G^D\).

## Key formulas

- $$\phi_{ij}(x_j)=\sum_{m=1}^{G+r}c^{(i,j)}_{m}\,B_{m}(x_{j}).$$
- $$K_{\mathrm{mult}}(\bm{x})=\Big(\sum_{i}N_{i}^{x}(x)c_{i}^{x}\Big)\Big(\sum_{j}N_{j}^{y}(y)c_{j}^{y}\Big)=\sum_{i,j}N_{i}^{x}(x)N_{j}^{y}(y)\,c_{i}^{x}c_{j}^{y}.$$
- $$u_\theta(x_1,\ldots,x_D)=\sum_{k=1}^{R}\prod_{d=1}^{D}\left(\sum_{m=1}^{G+r}c_{k,d,m}B_{d,m}(x_d)\right).$$

## Implementation notes

1. Integration point: replace the first coordinate-mixing layer of a KAN, coordinate MLP, or implicit neural representation. For each rank \(k\) and coordinate \(d\), store a vector of spline coefficients \(c_{k,d,m}\); evaluate one-dimensional B-spline bases locally and combine their outputs multiplicatively. Use \(R\) separable products so expressivity is controlled by rank instead of a full tensor grid. 2. Pseudocode: `u=zeros(batch); for k in range(R): z=ones(batch); for d in range(D): B=bspline_basis(x[:,d], knots[d], order=r); s=B @ coeff[k,d]; z=z*s; u=u+z`; optionally add a residual additive KAN or MLP path. For two-dimensional PDEs this is `u=sum_k (Nx @ a[k])*(Ny @ b[k])`. Enforce known boundary conditions by multiplying the output by a boundary mask or retain a boundary penalty. 3. Compute spline bases and derivatives analytically or with a differentiable B-spline routine. The tensor-product representation is exact for rank-one separable fields; approximation quality for general fields is controlled empirically by rank \(R\). Products may cause exploding or vanishing activations, so initialize factor coefficients with scale \(R^{-1/(2D)}\), normalize each factor output, and use gradient clipping. 4. First experiment: fit a 2D manufactured PDE solution and compare a dense tensor-grid spline layer, additive KAN, and rank-\(R\) multiplicative KAN at matched parameter counts. Sweep \(R\in\{1,2,4,8\}\) and grid sizes, measuring relative field error, parameter memory, forward latency, and weak-residual error. The expected signal is lower memory and faster evaluation than a dense tensor-product basis for moderate rank, with accuracy improving as rank increases. Reject the method if the rank required to match additive-KAN accuracy removes the memory or speed advantage.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.