# ZCA In-Context Output Transport

- ID: 794
- Canonical URL: https://synthcore.org/idea/794/zca-in-context-output-transport
- API JSON: https://synthcore.org/api/idea/794.json
- API Markdown: https://synthcore.org/api/idea/794.md
- Verification status: failed_benchmark
- Source: [arXiv:2607.12241](https://arxiv.org/abs/2607.12241)
- Category: architecture
- Solves: accuracy, sample-efficiency, stability
- ML areas: graph-nn, mlp, training-dynamics, sample-efficiency
- Math tags: statistics, linear-algebra, geometry, dynamical-systems
- Ratings: usefulness 7/10; difficulty 4/10; novelty 5/10

## Idea description

Train a neural surrogate to predict outputs in a source-domain ZCA-whitened space, then adapt to a shifted domain using only the shifted domain's output mean and covariance. At inference, transport the network prediction through the target covariance square root, yielding a weight-free correction that preserves output-coordinate semantics and can be applied to MLP, CNN, graph-NN, or transformer regressors.

## Mathematical statement

Let $y\in\mathbb{R}^{d}$ be the physical output, with source-domain mean $\mu_s$ and covariance $C_s\succeq 0$. Define the regularized ZCA whitener $W_s=(C_s+\varepsilon I)^{-1/2}$ and the training target $z_s=W_s(y-\mu_s)$. Train a network $f_\theta(x)\approx z_s$. For a target domain, estimate $\mu_t$ and $C_t$ from $m$ calibration outputs and define $S_t=(C_t+\varepsilon I)^{1/2}$. The adapted prediction is $\hat y_t=\mu_t+S_t f_\theta(x)$. ZCA is computed from $C=U\operatorname{diag}(\lambda_i)U^\top$ as $C^{\pm1/2}=U\operatorname{diag}((\lambda_i+\varepsilon)^{\pm1/2})U^\top$. The mechanism assumes that the conditional operator is more invariant after removing domain-dependent first and second output moments.

## Key formulas

- $$\mu_s=\frac{1}{n_s}\sum_{i=1}^{n_s}y_i,\qquad C_s=\frac{1}{n_s-1}\sum_{i=1}^{n_s}(y_i-\mu_s)(y_i-\mu_s)^\top,$$
- $$z_s=W_s(y-\mu_s),\qquad W_s=(C_s+\varepsilon I)^{-1/2},\qquad f_\theta(x)\approx z_s,$$
- $$\hat y_t=\mu_t+(C_t+\varepsilon I)^{1/2}f_\theta(x),\qquad C_t=\frac{1}{m-1}\sum_{j=1}^{m}(y_j^t-\mu_t)(y_j^t-\mu_t)^\top,$$
- $$C=U\operatorname{diag}(\lambda_1,\ldots,\lambda_d)U^\top\Longrightarrow C^{\pm1/2}=U\operatorname{diag}\left((\lambda_i+\varepsilon)^{\pm1/2}\right)U^\top.$$

## Implementation notes

1. Exact integration point: use this as a replacement for the final regression normalization and denormalization layer of a supervised MLP or graph-NN. Compute source statistics once from training targets. Transform each source target with $z_s=(C_s+\varepsilon I)^{-1/2}(y-\mu_s)$ and train the network with MSE in whitened coordinates. At deployment, collect solved calibration examples from the shifted domain, estimate its mean and covariance, and cache the target square-root matrix. 2. Pseudocode: `mu_s,cov_s=stats(Y_source); W_s=inv_sqrt(cov_s+eps*I); train f(x) against W_s@(y-mu_s)`; then `mu_t,cov_t=stats(Y_calib); S_t=sqrt(cov_t+eps*I); y_hat=mu_t+S_t@f(x)`. If the calibration count is smaller than output dimension, use shrinkage `cov_t=(1-alpha)*cov_t+alpha*trace(cov_t)/d*I`. 3. Computed directly from the mechanism: the affine ZCA maps and eigendecompositions. Estimated empirically: source and target moments, shrinkage coefficient, calibration size, and residual conditional mismatch. 4. First cheap experiment: train a 3-layer MLP on a synthetic shifted regression operator and a graph-NN on IEEE-30 power-flow data with one line outage. Compare a frozen model, diagonal standardization, full ZCA transport, and gradient adaptation under equal calibration budgets. Sweep $m\in\{16,32,64,128,256,512\}$. The quantitative prediction is that target output mean and covariance errors will decrease after transport and that ZCA error will be much less sensitive to the moment mismatch $\|\mu_t-\mu_s\|_2+\|C_t-C_s\|_F$. Test the numerical stability boundary: when the smallest covariance eigenvalue approaches zero, the unregularized gain diverges as $\lambda_{\min}^{-1/2}$, while regularization bounds it by $(\lambda_{\min}+\varepsilon)^{-1/2}$.

## Verification

- Status: failed_benchmark
- Mechanism evidence: yes
- Mechanism confirmed: no
- Practical verdict: inconclusive
- Verdict: Built and ran a reproducible ZCA in-context output transport MVP with a source-trained MLP and shifted synthetic regression domain. The three required math predictions manifested: maximum inverse-square-root gain was 31.622775 versus the predicted bound 31.622777; exact affine target transport had 0.0052 relative covariance error and 0.0067 mean error; calibration covariance error followed an observed log-log slope of -0.429 versus the predicted -0.5. In the regression comparison, ZCA transport achieved RMSE 0.354/0.329/0.292/0.282 for m=16/32/128/256, versus diagonal transport 0.485/0.453/0.446/0.446 and frozen 1.888, indicating a meaningful toy-domain benefit.

### Mechanism check

- Verdict: Built and ran a reproducible ZCA in-context output transport MVP with a source-trained MLP and shifted synthetic regression domain. The three required math predictions manifested: maximum inverse-square-root gain was 31.622775 versus the predicted bound 31.622777; exact affine target transport had 0.0052 relative covariance error and 0.0067 mean error; calibration covariance error followed an observed log-log slope of -0.429 versus the predicted -0.5. In the regression comparison, ZCA transport achieved RMSE 0.354/0.329/0.292/0.282 for m=16/32/128/256, versus diagonal transport 0.485/0.453/0.446/0.446 and frozen 1.888, indicating a meaningful toy-domain benefit.
- Confidence: 8/10
- Limitations: Only a synthetic MLP regression task was tested; no CNN, graph-NN, transformer, IEEE-30 power-flow, gradient adaptation, repeated independent training seeds, FLOP/speed accounting, or calibration shrinkage sweep was evaluated. The affine sanity check uses finite samples and epsilon regularization, so its covariance error is not exactly zero.

### Practical benchmark

- Paired seeds: 8
- Baseline mean: 0
- Idea mean: 0
- p-value: 0.0976
- Paired wins: 6/8
- Benchmark verdict: no significant win

## Artifacts

- [bench_report.json](https://synthcore.org/code/325/bench_report.json)
- [experiment.py](https://synthcore.org/code/325/experiment.py)
- [report.md](https://synthcore.org/code/325/report.md)
- [report_bench_2026-08-31T011435.md](https://synthcore.org/code/325/report_bench_2026-08-31T011435.md)
- [results.json](https://synthcore.org/code/325/results.json)
- [run_bench.py](https://synthcore.org/code/325/run_bench.py)
- [zca_track.py](https://synthcore.org/code/325/zca_track.py)
- [Download all files as ZIP](https://synthcore.org/download/325)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.