Bohnenblust–Hille coefficient regularization

1. Integration point: implement this in a polynomial feature layer whose output is y=Σ_{|α|≤d}c_αφ_α(x), or in a tensorized MLP layer represented explicitly by coefficient tensors. Start with d=2, where p=4/3, and d=3, where p=3/2. Apply the regularizer to each output channel’s coefficient vector; do not apply it to ordinary dense layers. 2. Pseudocode: initialize coefficient tensor c; on every minibatch compute y=poly_forward(x,c) and task loss; draw K random phase vectors θ_k, set z_{k,i}=exp(iθ_{k,i}), evaluate f_k=Σ_αc_αΠ_i z_{k,i}^{α_i}; set S=max_k|f_k|; compute R=(Σ_α(|c_α|²+ε)^(p/2))^(1/p); optimize L_task+λ_BH R^p+λ_inf S². Use complex arithmetic or represent each complex coefficient and phase with two real channels. 3. The paper supplies the exponent p and the dimension-free inequality; C(d) need not be known for optimization. Estimate the torus supremum empirically with K=16–64 probes, resampling probes each step or each epoch. Track the empirical ratio R/S and increase K at evaluation time to detect probe underestimation. 4. First experiment: compare a degree-2 polynomial MLP against the same model with Frobenius, L1, and no coefficient regularization on CIFAR-10 or a synthetic parity/regression task, at fixed parameter count and dimensions N=32,128,512. Sweep λ_BH and λ_inf. Success means lower test error or better corruption stability at equal training loss, with the advantage persisting as N grows; also measure coefficient concentration, gradient spikes, and whether the empirical R/S ratio remains bounded instead of increasing with dimension.