Web-Constrained Product Flow

Integrate this as a two-dimensional invertible coupling block inside a VAE encoder or an invertible latent model, operating on two latent channels z_1 and z_2 at a time. Start with an affine coupling transform u=z_1 and v=s(z_1)z_2+t(z_1), where s(z_1)=softplus(a(z_1))+epsilon ensures a nonzero Jacobian; alternate channel order across blocks and optionally insert an invertible 2-by-2 linear mixing layer. Construct K=4 to 8 probe product densities q_k(z_1,z_2)=q^k_1(z_1)q^k_2(z_2), using one-dimensional Gaussians with different means and variances. For each probe, sample z, compute (u,v) and log_abs_det_J, and use small one-dimensional density heads r^k_1(u), r^k_2(v) to model the output marginals. Compute e_k=log q^k_1(z_1)+log q^k_2(z_2)-r^k_1(u)-r^k_2(v)-log_abs_det_J and add L_web=(1/K) sum_k mean(e_k^2) to the task, reconstruction, or negative-log-likelihood objective. To encode the rank bound, parameterize the probe-dependent corrections as alpha^k_T(t)=sum_{j=1}^3 c^k_{Tj}b_{Tj}(t), where T is one of X,Y,U,V and b_{Tj} are learned one-dimensional basis functions; penalize alpha^k_X(z_1)+alpha^k_Y(z_2)+alpha^k_U(u)+alpha^k_V(v) squared. The paper-derived quantities are the change-of-variables residual and the three-relation capacity limit; output marginals, basis functions, and coefficients are estimated during training. Pseudocode: sample k and z1,z2; compute u,v,logdet; evaluate e_k; compute the Abelian residual; backpropagate task_loss+lambda_web*mean(e_k^2)+lambda_abelian*mean(residual^2). First run a 2D toy benchmark where independent non-Gaussian inputs are transformed by a known nonlinear bijection, then test a small MNIST VAE with paired latent channels. Compare against an unconstrained RealNVP and a standard total-correlation penalty at matched parameter count and training FLOPs. Track held-out mutual information between u and v, reconstruction or likelihood, Jacobian singular-value condition numbers, and variance across random seeds. A successful result is lower held-out mutual information and better Jacobian conditioning without reducing reconstruction quality.