# Sparse-interaction Bohnenblust–Hille regularizer

- ID: 1202
- Canonical URL: https://synthcore.org/idea/1202/sparse-interaction-bohnenblust-hille-regularizer
- API JSON: https://synthcore.org/api/idea/1202.json
- API Markdown: https://synthcore.org/api/idea/1202.md
- Verification status: unverified
- Source: [arXiv:2607.20847](https://arxiv.org/abs/2607.20847)
- Category: regularization
- Solves: stability, generalization, accuracy
- ML areas: mlp, regularization, architecture
- Math tags: functional-analysis, harmonic-analysis, probability, approximation-theory
- Ratings: usefulness 5/10; difficulty 5/10; novelty 7/10

## Idea description

Add a support-sensitive coefficient regularizer to a high-order polynomial or Volterra layer whose monomials involve at most M input features. The regularizer penalizes the gap between the layer's coefficient ℓ_{2m/(m+1)} norm and its empirical worst-case response on random unit-modulus inputs, exploiting the fact that the theoretical gap constant approaches 1 for fixed M and large degree m.

## Mathematical statement

For an m-homogeneous polynomial P(z)=Σ_{|α|=m} c_α(P)z^α on the complex polydisc ᵓ^n, define the support size w(α)=|supp(α)|=card{j:α_j≠0}. The paper considers polynomials with w(α)≤M for every nonzero coefficient and proves the polynomial Bohnenblust–Hille inequality ( Σ_{|α|=m}|c_α(P)|^{2m/(m+1)} )^{(m+1)/(2m)}≤K_{m,M}||P||_∞, where ||P||_∞=sup_{z∈ᵓ^n}|P(z)|. Its main bound is 1≤K_{m,M}≤A_M^{M/m}m^{(M^2−1)/(2m)}, with A_M depending only on M, so K_{m,M}→1 for fixed M and m→∞. In the neural adaptation, c_α are trainable interaction coefficients, p=2m/(m+1), and ||P||_∞ is approximated by the maximum response over random phase probes z^(r)∈ℂ^n with |z_j^(r)|=1.

## Key formulas

- $$P(z)=\sum_{|\alpha|=m}c_{\alpha}(P)z^{\alpha},\qquad \|P\|_{\infty}:=\sup_{z\in\mathbb{D}^{n}}|P(z)|.$$
- $$w(\alpha):=|\operatorname{supp}(\alpha)|=\operatorname{card}\{j:\alpha_{j}\neq 0\}.$$
- $$\left(\sum_{|\alpha|=m}|c_{\alpha}(P)|^{\frac{2m}{m+1}}\right)^{\frac{m+1}{2m}}\leq K_{m,M}\|P\|_{\infty},\qquad 1\leq K_{m,M}\leq A_{M}^{M/m}m^{(M^{2}-1)/(2m)}.$$
- $$\mathcal{R}_{\mathrm{BH}}(c)=\operatorname{softplus}\!\left(\frac{\|c\|_{p}}{\max_{1\leq r\leq R}|P(z^{(r)})|+\varepsilon}-\widehat K_{m,M}\right)^2,\qquad p=\frac{2m}{m+1},\quad \widehat K_{m,M}=A_M^{M/m}m^{(M^2-1)/(2m)}.$$

## Implementation notes

(1) Integration point: use an explicit degree-m interaction layer after an embedding h∈R^n. Store only coefficients c_α whose support size w(α)≤M; the layer output is P(h)=Σ c_α∏_j h_j^{α_j}. This fits a polynomial network, factorization-machine layer, or Volterra block, rather than a standard dense MLP without explicit interaction coefficients. Apply the regularizer once per minibatch or every K optimizer steps.

(2) Pseudocode:
```
p = 2*m/(m+1)
coeff_norm = (sum(abs(c_alpha)**p for alpha in active_terms))**(1/p)
for r in range(R):
    phase = exp(1j * 2*pi * Uniform(0,1,n))
    probe_value[r] = sum(c_alpha * prod(phase[j]**alpha[j]
                          for j in support(alpha)) for alpha in active_terms)
emp_sup = max(abs(probe_value[r]) for r in range(R))
Khat = A_M**(M/m) * m**((M*M-1)/(2*m))
loss = task_loss + lambda_bh * softplus(coeff_norm/(emp_sup+eps)-Khat)**2
```
Use complex phases directly, or evaluate real and imaginary parts with paired cosine/sine probes. Normalize h before the interaction layer so the unit-polydisc interpretation is meaningful; alternatively use h/(||h||_∞+ε) per sample.

(3) Computed from the paper: the exponent p, the support restriction w(α)≤M, and the explicit upper-bound schedule Khat. Estimated empirically: the supremum norm, using R random probes. Increase R during validation to test whether the regularizer is being fooled by undersampling. A conservative variant maintains an exponential moving maximum of probe responses. The extracted text does not provide a numerical value for A_M, so initialize A_M=2 or use a fixed safety multiplier q≥1 and report sensitivity to q and R.

(4) First experiment: train a degree-8 or degree-16 polynomial regression/classification network on a synthetic teacher containing sparse interactions of support at most M=2 or 3. Compare an unconstrained coefficient ℓ_2 baseline, ordinary ℓ_p coefficient weight decay, and this BH regularizer at matched parameter count and FLOPs. Measure validation error, worst-case response on 10,000 fresh random phase probes, coefficient-to-response ratio, and gradient-norm variance. Success means lower held-out error or improved phase robustness at equal training loss, with the empirical ratio approaching the predicted Khat rather than growing with degree.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.