Polynomial Jacobian Non-Collapse

(1) Integration point: apply this to a two-dimensional continuous output head, such as a VAE decoder coordinate head, a diffusion model prediction head for two selected channels, or a two-dimensional bottleneck in an MLP. To expose top-degree coefficients, use a small polynomial or Hermite-feature adapter before the output head: f_i(x) = Σ_{|α|≤d} a_{i,α} φ_α(x), with tensor-product Hermite features for approximately Gaussian inputs or centered monomial features for general inputs. The vectors a₁ and a₂ contain coefficients with |α| = d. (2) Pseudocode: sample a minibatch x_b; compute y_b = f(x_b); obtain g₁b = ∇x f₁(x_b) and g₂b = ∇x f₂(x_b) with autodiff; calculate q₁ = g₁b·g₁b, q₂ = g₂b·g₂b, q₁₂ = g₁b·g₂b, and Δ_b = q₁q₂ − q₁₂². Add −λΔ mean_b log(Δ_b + ε) to the task loss. Normalize a₁ and a₂, calculate ρ = sqrt(max(ε, 1 − (a₁ᵀa₂)²)), and add λρ max(0, ρ₀ − ρ)². (3) The paper supplies the anti-concentration and expected-determinant relationships; the minibatch determinant, coefficient score, and output occupancy are empirical estimates. Use gradient clipping and a warm-up schedule for λΔ because the log determinant has large gradients near singular Jacobians. (4) First experiment: train a small VAE with a two-dimensional decoder output on a synthetic mixture of 2D Gaussians and MNIST-derived two-dimensional projections. Compare the baseline, determinant-only regularization, and coefficient-plus-determinant regularization at equal parameter count. Measure reconstruction loss, held-out likelihood, mean and tenth-percentile Δ_b, ρ, fraction of occupied equal-area output grid cells, and maximum probability in a thin strip. The method succeeds if it increases occupied-area coverage and reduces thin-strip concentration without materially degrading reconstruction or likelihood.