Observable-Probe Distribution Matching

(1) Integration point: add an auxiliary distribution-matching loss to a diffusion model, VAE decoder, or implicit generator. Choose a shared finite basis \(\phi_1,\ldots,\phi_m\) on a low-dimensional projection of samples, such as random Fourier features or Gaussian RBFs, and evaluate an antisymmetric interaction field at \(N\) probe locations. Draw target samples \(z\sim p\) and model samples \(y\sim q_\theta\). Estimate basis coefficients \(a\) and \(b_\theta\) by least squares or batch averages, then form the antisymmetric mismatch vector \(c_\theta=a\wedge b_\theta\). Construct \(M\) by evaluating the selected kernel, basis functions, and probes, and compute \(v=M c_\theta\). (2) Pseudocode: sample batches; estimate \(b_\theta\); compute \(c_\theta=a\wedge b_\theta\); compute \(v=M c_\theta\); minimize \(L=L_{\rm task}+\lambda\|v\|_2^2\). Every \(K\) steps, update probe locations or probe weights to maximize \(\log(\sigma_{\min}(M)+\delta)\), with probes constrained to the target support; use softmax weights for differentiable weighting. Estimate \(\sigma_{\min}(M)\) using SVD or randomized SVD. (3) Exact versus empirical quantities: the linear identity and singular-value inequality are exact for the chosen finite basis; coefficients, held-out field noise \(\varepsilon_V\), and operator error \(\varepsilon_M\) are estimated from independent batches. Use bootstrap or concentration radii for \(\varepsilon_V\), and rebuild \(M\) on independent probes to estimate \(\varepsilon_M\). If \(\sigma_{\min}(M)\leq\varepsilon_M\), report abstention rather than a small mismatch. (4) First experiment: train a small MLP generator or score-based diffusion model on a 2D eight-Gaussian-mixture target. Compare fixed random probes, uniform probes, and adaptive smallest-singular-value probes at equal batch size and FLOPs. Measure held-out sliced Wasserstein distance, dense-grid TV, coefficient error, \(\sigma_{\min}(M)\), and the frequency of false low-drift cases caused by rank deficiency. Success means lower held-out distribution error at equal training steps, fewer blind-direction failures, and a calibrated relationship between observable loss and true coefficient mismatch.