{
 "artifacts": null,
 "category": "regularization",
 "description": "Replace two unconstrained scalar quadratic feature heads with a pair whose quadratic forms admit a positive-definite linear combination. This prevents the two heads from simultaneously vanishing on any nonzero hidden vector, which can reduce representation collapse and improve the conditioning of downstream gates or auxiliary objectives. The constraint can be implemented softly with a spectral-margin penalty, or exactly by parameterizing one learned pencil as positive definite.",
 "formulas_latex": [
  "$$\\{x\\in\\mathbb{R}^{n}\\mid x^{T}Ax=0\\}\\cap\\{x\\in\\mathbb{R}^{n}\\mid x^{T}Bx=0\\}=\\{0\\}\\ \\Longleftrightarrow\\ \\exists(\\lambda,\\mu)\\neq(0,0)\\ \\text{s.t.}\\ \\lambda A+\\mu B\\succ0,\\qquad n\\geq3.$$",
  "$$\\gamma(A,B,\\theta)=\\lambda_{\\min}\\!\\left(\\cos(\\theta)A+\\sin(\\theta)B\\right),\\qquad \\mathcal{L}_{\\mathrm{pencil}}=\\operatorname{softplus}(m-\\gamma(A,B,\\theta)),$$",
  "$$q_A(h)=h^{T}Ah,\\qquad q_B(h)=h^{T}Bh,\\qquad h\\neq0\\ \\Longrightarrow\\ \\big(q_A(h),q_B(h)\\big)\\neq(0,0)\\ \\text{when }\\gamma\u003e0.$$"
 ],
 "id": 2896,
 "implementation": "Integrate this into an MLP block after a hidden vector h in R^n, using two symmetric quadratic heads q_A(h)=h^T A h and q_B(h)=h^T B h. Parameterize A=(U+U^T)/2 and B=(V+V^T)/2, and add a scalar pencil angle theta with lambda=cos(theta), mu=sin(theta). At every training step: (1) compute q_A and q_B for the minibatch; (2) form P=cos(theta)A+sin(theta)B; (3) estimate gamma=lambda_min(P) using an exact symmetric eigendecomposition for small n or 3-5 Lanczos or power iterations for larger n; (4) add alpha*softplus(m-gamma) to the task loss, where m is a target positive margin; (5) backpropagate through the eigensolver or use the Rayleigh-quotient estimate gamma_hat=v^T P v with v updated by normalized power iteration on -P. Optionally make the guarantee exact by parameterizing P=LL^T+epsilon I and setting A=P cos(theta)-C sin(theta), B=P sin(theta)+C cos(theta), where C is an unconstrained symmetric matrix; this makes the chosen pencil positive definite by construction. The mathematics supplies the certificate gamma\u003e0; empirical quantities are the hidden-vector norm distribution, the frequency of simultaneous near-zero outputs, and downstream conditioning. First run a cheap ablation on MNIST or CIFAR-10 with a 64-dimensional MLP, comparing two unconstrained quadratic heads against the pencil-regularized version at matched parameter count and FLOPs. Measure training stability, the minimum observed sqrt(q_A(h)^2+q_B(h)^2), gradient norms, and test accuracy. Success is a substantial reduction in near-simultaneous head collapse without slower loss descent or a meaningful parameter or FLOP increase.",
 "math_summary": "Calabi's theorem states that for symmetric matrices A,B in R^{n x n}, with n \u003e= 3, the condition {x in R^n : x^T A x = 0} intersect {x in R^n : x^T B x = 0} = {0} is equivalent to the existence of (lambda,mu) != (0,0) such that lambda A + mu B is positive definite, written lambda A + mu B ≻ 0. Here x is the hidden feature vector, A and B are the two learned symmetric quadratic-head matrices, and lambda,mu are scalar pencil coefficients. We exploit the certificate by maximizing the smallest eigenvalue gamma = lambda_min(lambda A + mu B); gamma \u003e 0 certifies that no nonzero hidden vector can make both quadratic heads exactly zero. To avoid scale ambiguity, constrain lambda^2 + mu^2 = 1, for example by setting (lambda,mu) = (cos(theta), sin(theta)).",
 "math_tags": [
  "linear-algebra",
  "geometry",
  "optimization"
 ],
 "ml_areas": [
  "mlp",
  "regularization",
  "training-dynamics"
 ],
 "paper": {
  "arxiv_id": "2608.30571",
  "arxiv_url": "https://arxiv.org/abs/2608.30571",
  "summary_what_math_gives_to_ml": "The paper turns exclusion of simultaneous zeros of two quadratic forms into a constructive positive-definite pencil certificate: for n \u003e= 3, two homogeneous equations have no common nonzero solution exactly when some nontrivial linear combination of their matrices is positive definite. This gives neural networks a way to design pairs of quadratic feature constraints that cannot become jointly degenerate, rather than merely penalizing observed violations. The most direct transfer is a regularized quadratic feature block whose matrices are trained together with an explicitly optimized separating pencil and a spectral-margin loss. The guarantee is structural and applies to every hidden representation, while the engineering tradeoff is the cost of maintaining or estimating the smallest eigenvalue of the pencil.",
  "title": "Last two pieces of the puzzle for unsolvability of a system of two quadratic (in)equalities",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 8,
  "usefulness": 5
 },
 "solves": [
  "stability",
  "generalization"
 ],
 "title": "Positive-Definite Quadratic Feature Pair",
 "url": "https://synthcore.org/idea/2896/positive-definite-quadratic-feature-pair",
 "verification": {
  "peer_reviewed": false,
  "status": "unverified",
  "status_label": "Unverified",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)"
 }
}