# Kac-Ward Exact Teacher for Autoregressive Samplers - ID: 2721 - Canonical URL: https://synthcore.org/idea/2721/kac-ward-exact-teacher-for-autoregressive-samplers - API JSON: https://synthcore.org/api/idea/2721.json - API Markdown: https://synthcore.org/api/idea/2721.md - Verification status: mech_ok_no_baseline - Source: [arXiv:2608.24382](https://arxiv.org/abs/2608.24382) - Category: training - Solves: accuracy, sample-efficiency, generalization - ML areas: optimizer, training-dynamics, loss, sampling - Math tags: statistical-mechanics, graph-theory, linear-algebra, probability - Ratings: usefulness 8/10; difficulty 6/10; novelty 7/10 ## Idea description Use the exact Kac–Ward conditional sampler as an oracle teacher for a neural autoregressive distribution over planar Ising configurations. At each prefix, supervise the network with the exact next-spin probability rather than only a sampled next spin, then retain the oracle as an evaluation and active-correction mechanism for prefixes where the student is inaccurate. This converts an approximate variational sampler into a calibrated amortized approximation with an exact, independently sampled reference distribution. ## Mathematical statement For spins $s_i\in\{-1,+1\}$, couplings $J_{ij}$, inverse temperature $\beta$, and energy $E(\mathbf{s})=-\sum_{\langle i,j\rangle}J_{ij}s_is_j$, the target distribution is $P(\mathbf{s})=e^{-\beta E(\mathbf{s})}/Z$. Under an ordering of the $N$ vertices, the exact chain rule is $P(\mathbf{s})=\prod_{i=1}^N P_i(s_i\mid\mathbf{s}_{i}}\exp[-\beta E(x,a,\mathbf{s}_{>i})]$; then the oracle label is $q_i(a\mid x)=Z_i(a\mid x)/(Z_i(+1\mid x)+Z_i(-1\mid x))$. The paper evaluates each $Z_i(a\mid x)$ by adding auxiliary spins and edges so that the boundary fields induced by $x$ become a planar zero-field Ising model. For a zero-field planar graph $G=(V,E)$ with edge weights $K_e=\beta J_e$, the Kac–Ward expression has the form $Z_G=2^{|V|}\prod_{e\in E}\cosh(K_e)\sqrt{\det(I-T_G)}$, where $T_G$ is indexed by directed edges and $(T_G)_{(u\to v),(v\to w)}=\tanh(K_{uv})e^{i\theta(v\to w,u\to v)}$ when $w\neq u$, with $\theta$ the signed turning angle. The student minimizes exact conditional cross-entropy, whose excess over its optimum is the expected conditional KL divergence. ## Key formulas - $$P(\mathbf{s})=\prod_{i=1}^{N}P_i(s_i\mid\mathbf{s}_{i}}e^{-\beta E(x,a,\mathbf{s}_{>i})},\qquad E(\mathbf{s})=-\sum_{\langle i,j\rangle}J_{ij}s_is_j.$$ - $$Z_G=2^{|V|}\left(\prod_{e\in E}\cosh K_e\right)\sqrt{\det(I-T_G)},\quad (T_G)_{(u\to v),(v\to w)}=\tanh(K_{uv})e^{i\theta(v\to w,u\to v)}.$$ - $$\mathcal{L}(\theta)=\mathbb{E}_{x}\left[-\sum_{i=1}^{N}\sum_{a=\pm1}q_i(a\mid x)\log p_\theta(a\mid x)\right],\qquad \mathcal{L}(\theta)-\mathcal{L}_{\min}=\mathbb{E}_{x,i}\,\mathrm{KL}\big(q_i(\cdot\mid x)\,\|\,p_\theta(\cdot\mid x)\big).$$ ## Implementation notes 1. Integration point: implement a binary autoregressive model, such as a masked MLP, PixelCNN-style network, or Transformer over a raster ordering of the planar lattice. Its output at position i is a logit l_i(x), giving p_theta(s_i=+1|x)=sigmoid(l_i(x)). Build a separate Kac–Ward oracle that accepts a prefix x, constructs the paper's auxiliary planar zero-field graph for the two candidate values s_i=+1 and -1, and returns log Z_i(+1|x) and log Z_i(-1|x). 2. Pseudocode: for each minibatch prefix x and position i, compute q_plus=softmax([logZ_plus,logZ_minus])[0]; obtain p_plus from the network; add -(q_plus*log(p_plus)+(1-q_plus)*log(1-p_plus)) to the loss; periodically sample the next spin from p_theta, query the oracle on the resulting student-generated prefix, and add these off-policy prefixes to a replay buffer. Use stable complex slogdet evaluations rather than explicitly forming determinants. 3. Computed from the paper: the two exact conditional partition functions, auxiliary graph construction, and Kac–Ward matrix. Estimated empirically: student conditional KL, held-out log-likelihood, energy and correlation errors, and sample autocorrelation. 4. First cheap experiment: use 4x4 and 6x6 square lattices with random Gaussian or ±1 couplings at several temperatures, and compare ordinary maximum-likelihood training, sampled-label training, and Kac–Ward soft-label distillation. Draw 10,000 exact oracle configurations for validation and independently generate 10,000 student configurations. The quantitative prediction is that oracle-generated samples have zero lag-k autocorrelation up to finite-sample error, while the distilled model's average conditional KL decreases with training and its held-out log-likelihood gap tracks the accumulated conditional-KL gap. Require mean conditional-probability error below 1e-2 on 4x4 instances before comparing observables. Also measure oracle runtime versus lattice size and test the predicted approximately O(N^{5/2}) scaling; after amortization, student generation should require O(N) sequential outputs. ## Verification - Status: mech_ok_no_baseline - Mechanism evidence: yes - Mechanism confirmed: yes - Practical verdict: inconclusive - Verdict: Built a tiny planar-Ising autoregressive distillation MVP with exact conditional probabilities by enumeration and a Kac–Ward determinant sanity check. The CE-minus-entropy equals conditional-KL identity held with absolute error below 7e-17, and the half-angle Kac–Ward formula matched enumeration to 1.9e-15 relative error on 3x3. Soft oracle supervision consistently improved conditional KL over sampled-label training, confirming the proposed mechanism in this toy setting. ### Mechanism check - Verdict: Built a tiny planar-Ising autoregressive distillation MVP with exact conditional probabilities by enumeration and a Kac–Ward determinant sanity check. The CE-minus-entropy equals conditional-KL identity held with absolute error below 7e-17, and the half-angle Kac–Ward formula matched enumeration to 1.9e-15 relative error on 3x3. Soft oracle supervision consistently improved conditional KL over sampled-label training, confirming the proposed mechanism in this toy setting. - Confidence: 8/10 - Limitations: The conditional oracle uses brute-force enumeration rather than the paper’s auxiliary-graph conditional Kac–Ward construction, so this is only a small-lattice MVP. It does not test 6x6 or larger instances, held-out generalization, replay-based active correction, observables, autocorrelation, or the claimed O(N^{5/2}) Kac–Ward scaling. ### Practical benchmark - Paired seeds: 8 - Baseline mean: 0 - Idea mean: 0 - p-value: 0.23005 - Paired wins: 5/8 - Benchmark verdict: no significant win ## Artifacts - [bench_report.json](https://synthcore.org/code/1052/bench_report.json) - [experiment.py](https://synthcore.org/code/1052/experiment.py) - [ising_teacher_track.py](https://synthcore.org/code/1052/ising_teacher_track.py) - [kw_test.py](https://synthcore.org/code/1052/kw_test.py) - [report.md](https://synthcore.org/code/1052/report.md) - [report_bench_2026-09-01T232411.md](https://synthcore.org/code/1052/report_bench_2026-09-01T232411.md) - [results.json](https://synthcore.org/code/1052/results.json) - [stage2_bench.py](https://synthcore.org/code/1052/stage2_bench.py) - [Download all files as ZIP](https://synthcore.org/download/1052) ## Disclaimer AI-generated research hypothesis, automatically tested. Not peer-reviewed.