# Pfaffian activation budget

- ID: 1587
- Canonical URL: https://synthcore.org/idea/1587/pfaffian-activation-budget
- API JSON: https://synthcore.org/api/idea/1587.json
- API Markdown: https://synthcore.org/api/idea/1587.md
- Verification status: unverified
- Source: [arXiv:2607.29267](https://arxiv.org/abs/2607.29267)
- Category: regularization
- Solves: generalization, stability
- ML areas: mlp, regularization, training-dynamics
- Math tags: geometry, algebra, approximation-theory
- Ratings: usefulness 5/10; difficulty 4/10; novelty 8/10

## Idea description

Use a tanh MLP with an explicitly tracked Pfaffian-chain complexity and select its width and input sparsity using the paper's zero-count bound. The bound limits the number of regular decision-boundary crossings along one-dimensional data-space restrictions, so it provides a principled way to discourage excessively oscillatory fits beyond ordinary weight decay.

## Mathematical statement

The paper's theorem states that if f_1,...,f_n are Pfaffian functions on R^n sharing a chain q=(q_1,...,q_s), each chain differential equation has degree at most alpha, each f_i has Pfaffian degree beta_i, and every chain function depends on only k input variables, then the number of regular common real zeros is bounded by B=2^{s(s-1)/2}(product_i beta_i)(sum_i beta_i-n+min{k,s}alpha+1)^s. Here s is the chain length, alpha is the maximum polynomial degree in the chain differential equations, beta_i is the Pfaffian degree of equation f_i, and k is the number of active input coordinates. For tanh, q_j(t)=tanh(a_j t+b_j) satisfies q_j'(t)=a_j(1-q_j(t)^2), so alpha=2. A scalar affine readout F(t)-c has n=1 and beta_1=1, giving B=3^s for k=1 and s>=1. The theorem counts regular isolated roots; degenerate tangencies should therefore be handled separately when measuring empirical crossings.

## Key formulas

- $$B=2^{\frac{s(s-1)}{2}}\beta_{1}\dotsm\beta_{n}\left(\beta_{1}+\dotsb+\beta_{n}-n+\min\{k,s\}\alpha+1\right)^{s}.$$
- $$q_j'(t)=a_j\bigl(1-q_j(t)^2\bigr),\qquad q_j(t)=\tanh(a_jt+b_j).$$
- $$B_{\mathrm{1D}}=2^{\frac{s(s-1)}{2}}\beta\left(\beta-1+\min\{1,s\}\alpha+1\right)^s=3^s\quad\text{for }\beta=1,\ \alpha=2,\ s\geq1.$$
- $$\mathcal{L}=\mathcal{L}_{\mathrm{CE}}+\lambda\log\!\left(1+B(s,\alpha,\boldsymbol{\beta},k)\right),\qquad k=\#\{r:\|W_{:,r}\|_2>\tau\}.$$

## Implementation notes

(1) Exact integration point: use a one-hidden-layer tanh classifier, f(x)=b_0+sum_{j=1}^s v_j tanh(w_j^T x+b_j), and apply the certificate to restrictions x=x_0+t u along random lines through the training and validation data. Treat the hidden width s as the Pfaffian-chain length. Add group sparsity over input coordinates, with g_r=||W_{:,r}||_2, so that k is the number of coordinates whose group norm exceeds a threshold tau. (2) Pseudocode: for each candidate width s, initialize the MLP; on every batch compute logits and cross-entropy; compute g_r for every input coordinate; set k=sum_r 1[g_r>tau]; compute B=2^{s(s-1)/2} prod_i beta_i (sum_i beta_i-n+min(k,s) alpha+1)^s with alpha=2, n=1, and beta_1=1; add lambda*log1p(B) for architecture scoring. Because the exact k and B are discrete and astronomically large, do not backpropagate through the hard expression in the first implementation. Instead, use it after each epoch for pruning/model selection, or use k_soft=sum_r sigmoid((g_r-tau)/epsilon) during a differentiable architecture-search phase. Prune input groups and retrain after selection. (3) The paper supplies the analytic zero-count certificate. The implementation estimates active-variable count from trained weights and independently measures actual oscillation by evaluating f(x_0+t u)-c on a dense grid, removing values with absolute magnitude below a tolerance and counting sign changes. Also record the number of near-tangent stationary events using finite differences, since those are not regular roots covered by the theorem. (4) First cheap experiment: compare tanh MLP widths 4, 8, 16, and 32 on two-moons and a noisy one-dimensional sinusoid classification dataset against ordinary tanh training and tanh plus L2 weight decay, using matched optimization budgets. Evaluate test accuracy, calibration, adversarial perturbation stability, training-seed variance, and sign changes along 100 random lines. The expected signal is fewer boundary oscillations and lower seed variance at equal training accuracy, or equal test accuracy with a smaller selected width; the theoretical B itself need not be numerically tight.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.