Stieltjes-Newton Feature Head

Integrate this as a small feature head before an MLP, or as the scalar-coordinate branch of a coordinate network. For each scalar input x, first fit or learn a monotone coordinate g. A cheap fixed choice is the clipped empirical CDF g(x)=rank(x)/(N+1); a learnable choice is a monotone rational-quadratic spline with positive bin widths and heights, initialized to the identity. Let z=g(x), choose ordered knots z_1<...<z_K, and compute generalized Newton features recursively. In the continuous case use B_0(z)=1 and B_{j+1}(z)=(z-z_{j+1})B_j(z). For a genuinely Stieltjes version, approximate the integral recurrence by quadrature over sorted training values and include jump increments Δg(t). Concatenate B_0,...,B_K with the original hidden representation and apply a learned linear projection. Initialize the constant coefficient and add one feature every few epochs using the Newton update h ← h+c_j B_j, freezing old coefficients during the first validation interval. Normalize every feature by its training-set RMS and regularize the spline derivative toward a positive bounded range. The paper supplies the basis construction and lower-triangular incremental update; estimate g, knot locations, feature scales, and coefficients empirically. First test on one-dimensional Friedman regression and a UCI tabular regression dataset with a two-layer MLP baseline at equal parameter count. Compare the raw MLP, ordinary polynomial features, Fourier features, and Stieltjes-Newton features. Success means lower validation MSE at equal parameters, faster loss reduction at equal FLOPs, and an advantage when samples are highly nonuniform or contain sharp distributional jumps.