△ Mechanism confirmed, baseline not beaten 2026

Kac-Ward Exact Teacher for Autoregressive Samplers

Implementation & benchmark of arXiv:2608.24382 — Exact autoregressive sampling of planar Ising spin glasses via the Kac--Ward theory

Usefulness8/10

Difficulty6/10

Novelty7/10

Training Accuracy Sample efficiency Generalization Optimizer Training dynamics Loss Sampling Statistical mechanics Graph theory Linear algebra Probability

Source paper: Exact autoregressive sampling of planar Ising spin glasses via the Kac--Ward theory arXiv:2608.24382 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Use the exact Kac–Ward conditional sampler as an oracle teacher for a neural autoregressive distribution over planar Ising configurations. At each prefix, supervise the network with the exact next-spin probability rather than only a sampled next spin, then retain the oracle as an evaluation and active-correction mechanism for prefixes where the student is inaccurate. This converts an approximate variational sampler into a calibrated amortized approximation with an exact, independently sampled reference distribution.

Formulas

$$P(\mathbf{s})=\prod_{i=1}^{N}P_i(s_i\mid\mathbf{s}_{<i}),\qquad q_i(a\mid x)=\frac{Z_i(a\mid x)}{Z_i(+1\mid x)+Z_i(-1\mid x)}.$$

$$Z_i(a\mid x)=\sum_{\mathbf{s}_{>i}}e^{-\beta E(x,a,\mathbf{s}_{>i})},\qquad E(\mathbf{s})=-\sum_{\langle i,j\rangle}J_{ij}s_is_j.$$

$$Z_G=2^{|V|}\left(\prod_{e\in E}\cosh K_e\right)\sqrt{\det(I-T_G)},\quad (T_G)_{(u\to v),(v\to w)}=\tanh(K_{uv})e^{i\theta(v\to w,u\to v)}.$$

$$\mathcal{L}(\theta)=\mathbb{E}_{x}\left[-\sum_{i=1}^{N}\sum_{a=\pm1}q_i(a\mid x)\log p_\theta(a\mid x)\right],\qquad \mathcal{L}(\theta)-\mathcal{L}_{\min}=\mathbb{E}_{x,i}\,\mathrm{KL}\big(q_i(\cdot\mid x)\,\|\,p_\theta(\cdot\mid x)\big).$$

Mathematical statement

For spins $s_i\in\{-1,+1\}$, couplings $J_{ij}$, inverse temperature $\beta$, and energy $E(\mathbf{s})=-\sum_{\langle i,j\rangle}J_{ij}s_is_j$, the target distribution is $P(\mathbf{s})=e^{-\beta E(\mathbf{s})}/Z$. Under an ordering of the $N$ vertices, the exact chain rule is $P(\mathbf{s})=\prod_{i=1}^N P_i(s_i\mid\mathbf{s}_{<i})$. For a prefix $x=\mathbf{s}_{<i}$, define $Z_i(a\mid x)=\sum_{\mathbf{s}_{>i}}\exp[-\beta E(x,a,\mathbf{s}_{>i})]$; then the oracle label is $q_i(a\mid x)=Z_i(a\mid x)/(Z_i(+1\mid x)+Z_i(-1\mid x))$. The paper evaluates each $Z_i(a\mid x)$ by adding auxiliary spins and edges so that the boundary fields induced by $x$ become a planar zero-field Ising model. For a zero-field planar graph $G=(V,E)$ with edge weights $K_e=\beta J_e$, the Kac–Ward expression has the form $Z_G=2^{|V|}\prod_{e\in E}\cosh(K_e)\sqrt{\det(I-T_G)}$, where $T_G$ is indexed by directed edges and $(T_G)_{(u\to v),(v\to w)}=\tanh(K_{uv})e^{i\theta(v\to w,u\to v)}$ when $w\neq u$, with $\theta$ the signed turning angle. The student minimizes exact conditional cross-entropy, whose excess over its optimum is the expected conditional KL divergence.

Implementation notes

1. Integration point: implement a binary autoregressive model, such as a masked MLP, PixelCNN-style network, or Transformer over a raster ordering of the planar lattice. Its output at position i is a logit l_i(x), giving p_theta(s_i=+1|x)=sigmoid(l_i(x)). Build a separate Kac–Ward oracle that accepts a prefix x, constructs the paper's auxiliary planar zero-field graph for the two candidate values s_i=+1 and -1, and returns log Z_i(+1|x) and log Z_i(-1|x). 2. Pseudocode: for each minibatch prefix x and position i, compute q_plus=softmax([logZ_plus,logZ_minus])[0]; obtain p_plus from the network; add -(q_plus*log(p_plus)+(1-q_plus)*log(1-p_plus)) to the loss; periodically sample the next spin from p_theta, query the oracle on the resulting student-generated prefix, and add these off-policy prefixes to a replay buffer. Use stable complex slogdet evaluations rather than explicitly forming determinants. 3. Computed from the paper: the two exact conditional partition functions, auxiliary graph construction, and Kac–Ward matrix. Estimated empirically: student conditional KL, held-out log-likelihood, energy and correlation errors, and sample autocorrelation. 4. First cheap experiment: use 4x4 and 6x6 square lattices with random Gaussian or ±1 couplings at several temperatures, and compare ordinary maximum-likelihood training, sampled-label training, and Kac–Ward soft-label distillation. Draw 10,000 exact oracle configurations for validation and independently generate 10,000 student configurations. The quantitative prediction is that oracle-generated samples have zero lag-k autocorrelation up to finite-sample error, while the distilled model's average conditional KL decreases with training and its held-out log-likelihood gap tracks the accumulated conditional-KL gap. Require mean conditional-probability error below 1e-2 on 4x4 instances before comparing observables. Also measure oracle runtime versus lattice size and test the predicted approximately O(N^{5/2}) scaling; after amortization, student generation should require O(N) sequential outputs.

Verification

Mechanism confirmed, baseline not beaten

Stage 1 · Toy mechanism gate: Passed ✓

Stage 2 · Mechanism transferred to benchmark: Confirmed ✓

Stage 2 · Practical benchmark result: Inconclusive

Methodology: 8 paired seeds, permutation test (p<0.05) against a learning-rate-tuned baseline. How verification works

Stage 1 — Mechanism check agent confidence 8/10

Built a tiny planar-Ising autoregressive distillation MVP with exact conditional probabilities by enumeration and a Kac–Ward determinant sanity check. The CE-minus-entropy equals conditional-KL identity held with absolute error below 7e-17, and the half-angle Kac–Ward formula matched enumeration to 1.9e-15 relative error on 3x3. Soft oracle supervision consistently improved conditional KL over sampled-label training, confirming the proposed mechanism in this toy setting.

Agent confidence: 8/10

Baseline: Sampled-label conditional KL: 0.02115, 0.03466, 0.03366 at beta 0.2, 0.7, 1.3.
Idea: Soft-oracle conditional KL: 0.0000680, 0.000333, 0.00138 at beta 0.2, 0.7, 1.3; approximately 25x–311x lower than baseline.

Limitations: The conditional oracle uses brute-force enumeration rather than the paper’s auxiliary-graph conditional Kac–Ward construction, so this is only a small-lattice MVP. It does not test 6x6 or larger instances, held-out generalization, replay-based active correction, observables, autocorrelation, or the claimed O(N^{5/2}) Kac–Ward scaling.

How to run: python3 experiment.py

Stage 2 — Benchmark vs. tuned baseline (arXiv:2608.24382) Sep 2, 2026

Ahead of the baseline (−1.8%) but not statistically significant (p=0.2301, wins 5 / 8 paired seeds).

Benchmark: Planar_ising_autoregressive
Model: mlp_tiny
Paired seeds: 8
Baseline mean: 0.1179
Idea mean: 0.1157
Effect (Δ): -0.0021 (−1.8%; negative = idea better)
Wins: 5 / 8 paired seeds
p-value: 0.2301 (permutation test, 20 000 shuffles)
Smallest detectable effect: ±3.2%
Mechanism: Confirmed ✓
Practical effect: Inconclusive
Baseline tuning: swept over 3 configs

Limitations:

The exact oracle uses cached brute-force enumeration on a fixed 4x4 ferromagnetic lattice rather than the full auxiliary-graph Kac-Ward conditional construction. Observables, long-run sample autocorrelation, larger lattices, replay-based active correction, and runtime scaling were not tested.

How to run: python3 stage2_bench.py

Verdict computed by deterministic test code from paired-seed statistics — not by the language model.

Stage 2 — Benchmark Sep 1, 2026 latest Failed ✗

Agent confidence: 9/10

Implemented and registered the planar Ising autoregressive track with a shared mlp_tiny system; only the training target changed from sampled next-spin labels to exact soft conditional probabilities. Test MSE improved from 0.117852 to 0.115734, but the paired permutation test gave p=0.23005, so this is no significant win. The NN-scale mechanism prediction was confirmed on seed 0 because conditional-probability MSE decreased from 0.0357703 to 0.0323526.

Baseline: Test MSE mean 0.1178522473, std 0.0178481495; best lr=0.01, epochs=5, batch=128.
Idea: Test MSE mean 0.1157343183, std 0.0174895135; best lr=0.01, epochs=5, batch=128; paired delta=-0.0021179290, p=0.23005; 5/8 wins.

Benchmark result