Pfaffian activation budget
Implementation & benchmark of arXiv:2607.29267 — Khovanskii's Bezout-type Theorem for Pfaffian Functions: A Self-Contained Proof, and Applications
Source paper: Khovanskii's Bezout-type Theorem for Pfaffian Functions: A Self-Contained Proof, and Applications arXiv:2607.29267 ⓘ · analyzed Aug 31, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use a tanh MLP with an explicitly tracked Pfaffian-chain complexity and select its width and input sparsity using the paper's zero-count bound. The bound limits the number of regular decision-boundary crossings along one-dimensional data-space restrictions, so it provides a principled way to discourage excessively oscillatory fits beyond ordinary weight decay.
Formulas
Mathematical statement
The paper's theorem states that if f_1,...,f_n are Pfaffian functions on R^n sharing a chain q=(q_1,...,q_s), each chain differential equation has degree at most alpha, each f_i has Pfaffian degree beta_i, and every chain function depends on only k input variables, then the number of regular common real zeros is bounded by B=2^{s(s-1)/2}(product_i beta_i)(sum_i beta_i-n+min{k,s}alpha+1)^s. Here s is the chain length, alpha is the maximum polynomial degree in the chain differential equations, beta_i is the Pfaffian degree of equation f_i, and k is the number of active input coordinates. For tanh, q_j(t)=tanh(a_j t+b_j) satisfies q_j'(t)=a_j(1-q_j(t)^2), so alpha=2. A scalar affine readout F(t)-c has n=1 and beta_1=1, giving B=3^s for k=1 and s>=1. The theorem counts regular isolated roots; degenerate tangencies should therefore be handled separately when measuring empirical crossings.
Implementation notes
(1) Exact integration point: use a one-hidden-layer tanh classifier, f(x)=b_0+sum_{j=1}^s v_j tanh(w_j^T x+b_j), and apply the certificate to restrictions x=x_0+t u along random lines through the training and validation data. Treat the hidden width s as the Pfaffian-chain length. Add group sparsity over input coordinates, with g_r=||W_{:,r}||_2, so that k is the number of coordinates whose group norm exceeds a threshold tau. (2) Pseudocode: for each candidate width s, initialize the MLP; on every batch compute logits and cross-entropy; compute g_r for every input coordinate; set k=sum_r 1[g_r>tau]; compute B=2^{s(s-1)/2} prod_i beta_i (sum_i beta_i-n+min(k,s) alpha+1)^s with alpha=2, n=1, and beta_1=1; add lambda*log1p(B) for architecture scoring. Because the exact k and B are discrete and astronomically large, do not backpropagate through the hard expression in the first implementation. Instead, use it after each epoch for pruning/model selection, or use k_soft=sum_r sigmoid((g_r-tau)/epsilon) during a differentiable architecture-search phase. Prune input groups and retrain after selection. (3) The paper supplies the analytic zero-count certificate. The implementation estimates active-variable count from trained weights and independently measures actual oscillation by evaluating f(x_0+t u)-c on a dense grid, removing values with absolute magnitude below a tolerance and counting sign changes. Also record the number of near-tangent stationary events using finite differences, since those are not regular roots covered by the theorem. (4) First cheap experiment: compare tanh MLP widths 4, 8, 16, and 32 on two-moons and a noisy one-dimensional sinusoid classification dataset against ordinary tanh training and tanh plus L2 weight decay, using matched optimization budgets. Evaluate test accuracy, calibration, adversarial perturbation stability, training-seed variance, and sign changes along 100 random lines. The expected signal is fewer boundary oscillations and lower seed variance at equal training accuracy, or equal test accuracy with a smaller selected width; the theoretical B itself need not be numerically tight.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.