Sobolev-Calibrated Frozen Sigmoid Features

(1) Integration point: use this as the first hidden layer of a shallow MLP, a frozen coordinate encoder for a PINN, or a frozen feature map before a small trainable linear head. Normalize every input coordinate to a bounded domain \(\Omega\), sample or deterministically construct \((w_j,b_j)\), compute \(\phi_j(x)=\sigma(w_j^\top x+b_j)\), and train only coefficients \(a_j\). For a PINN, differentiate \(\phi_j\) analytically or with autodiff and include PDE residuals in the head-training loss. (2) Pseudocode: x <- normalized inputs; for j=1..M: w[j],b[j] <- sample q(w,b); Phi[j] <- tanh(w[j]^T x+b[j]); yhat <- Phi @ a; loss <- mean((yhat-y)^2) + lambda * Sobolev_or_PDE_residual(Phi @ a); update a only. If targeting an \(H^m\) metric, add sampled derivative terms \(\sum_{|\alpha|\le m}\|D^\alpha yhat-D^\alpha y\|_2^2\). (3) Mathematics versus estimates: the exponent \((k-m)/d\) and random-feature logarithmic factor come from the paper. Estimate unknown target smoothness \(k\) by fitting validation-error slopes versus \(M\), or use known PDE regularity. The sampling density must be nonzero throughout the chosen direction-offset box; uniform sampling is the simplest valid choice. For a deterministic baseline, place directions on a spherical grid and offsets on a uniform grid, then compare against i.i.d. parameters. (4) First cheap experiment: fit smooth 2D functions such as \(u(x,y)=\sin(3x)\cos(2y)+e^{x+y}\) and a 2D Poisson solution using frozen tanh features, with \(M\in\{64,128,256,512,1024\}\). Compare against a trainable one-hidden-layer tanh MLP and ordinary random Fourier features at equal feature counts and optimizer steps. Plot \(L^2\), gradient, and Hessian errors against \(M\) on log-log axes. Success means a slope near \(-(k-m)/d\), stable performance across random seeds, and lower wall-clock optimization cost because only the output head is trained; for a PINN, additionally require lower PDE residual at equal training time.