Sparse Learnable Power-Law Head

1. Integration point: use a small encoder h_phi(x)=softplus(g_phi(x)) followed by a symbolic head. Start with M=32 candidates: h_k^a for exponents initialized from {-2,-1,-0.5,0.5,1,2}, pairwise products h_k^a h_l^b, and optionally log(h_k+epsilon_h). Parameterize exponents as a_max tanh(u) to avoid numerical explosions, and use yhat=softplus(sum_j xi_j theta_j+b) for positive outputs. 2. Pseudocode: compute h=softplus(g_phi(x)); construct every theta_j(h;lambda_j); compute yhat; backpropagate the hybrid log-MAPE loss plus lambda_s times the L1 norm of xi; every 10 to 50 epochs apply soft-thresholding xi_j <- sign(xi_j) max(|xi_j|-alpha lambda_s,0); prune terms with |xi_j|<tau and fine-tune. 3. Taken from the paper: the sparse linear library form and hybrid logarithmic/MAPE objective. Estimated empirically: encoder features, exponent values, sparsity strength, pruning threshold, and whether the chosen library spans the task. 4. First cheap experiment: train on a synthetic positive regression problem with targets spanning 10^4, comparing a same-width dense MLP head, a fixed-power library, and this learnable sparse head. Sweep lambda_s and report test MAPE, parameter count, and active terms. The falsifiable prediction is a sparsity transition: as lambda_s increases, the active-term fraction should collapse sharply while MAPE remains within 5 percent of the dense head over an intermediate interval. On y=2 x_1^1.7 x_2^-0.6+0.3 x_3^0.5, recovered exponents should approach 1.7, -0.6, and 0.5, with approximately three retained terms.