# Scalar Residual Thermodynamic Head

- ID: 2299
- Canonical URL: https://synthcore.org/idea/2299/scalar-residual-thermodynamic-head
- API JSON: https://synthcore.org/api/idea/2299.json
- API Markdown: https://synthcore.org/api/idea/2299.md
- Verification status: unverified
- Source: [arXiv:2608.14502](https://arxiv.org/abs/2608.14502)
- Category: architecture
- Solves: accuracy, sample-efficiency, stability
- ML areas: graph-nn, fine-tuning, loss, training-dynamics
- Math tags: optimization, approximation-theory, differential-geometry
- Ratings: usefulness 6/10; difficulty 4/10; novelty 6/10

## Idea description

Attach a small temperature-pressure residual head to a pretrained structural encoder instead of relearning the full free-energy surface. Predict one scalar Gibbs free energy and obtain entropy, volume, and other thermodynamic responses by automatic differentiation, enforcing that all outputs derive from a common potential.

## Mathematical statement

The paper decomposes the Gibbs free energy as \(\hat{G}(\bm{x}^{\circ};T,P)=U^{\circ}+\hat{G}^{\delta}(\bm{x}^{\circ};T,P)\), where \(\bm{x}^{\circ}\) is a relaxed crystal structure, \(U^{\circ}\) is the pretrained static 0 K energy, and \(\hat{G}^{\delta}\) is a learned thermodynamic residual depending on temperature \(T\), pressure \(P\), and structure. The residual has smaller structure-dependent dynamic range than \(G\), so it can be represented by a lower-capacity adapter. For a Gibbs potential, the thermodynamic identities are \(S=-\partial G/\partial T\) and \(V=\partial G/\partial P\), where \(S\) is entropy and \(V\) is volume; these derivatives are computed through the residual because \(U^{\circ}\) is independent of \(T,P\). The adapted learning objective combines free-energy error with derivative supervision: \(\mathcal{L}=\operatorname{Huber}(\hat{G}-G)+\lambda_S\operatorname{Huber}(-\partial_T\hat{G}-S)+\lambda_V\operatorname{Huber}(\partial_P\hat{G}-V)+\lambda_{\mathrm{sm}}\|\nabla^2_{T,P}\hat{G}\|^2\), with optional Hessian terms used only when convexity or smoothness labels are available.

## Key formulas

- $$\hat{G}(\bm{x}^{\circ};T,P)=U^{\circ}+\hat{G}^{\delta}(\bm{x}^{\circ};T,P).$$
- $$S(\bm{x}^{\circ};T,P)=-\frac{\partial \hat{G}}{\partial T}=-\frac{\partial \hat{G}^{\delta}}{\partial T},\qquad V(\bm{x}^{\circ};T,P)=\frac{\partial \hat{G}}{\partial P}=\frac{\partial \hat{G}^{\delta}}{\partial P}.$$
- $$\mathcal{L}=\operatorname{Huber}(\hat{G}-G)+\lambda_S\operatorname{Huber}\left(-\partial_T\hat{G}-S\right)+\lambda_V\operatorname{Huber}\left(\partial_P\hat{G}-V\right)+\lambda_{\mathrm{sm}}\left\|\nabla^2_{T,P}\hat{G}\right\|_F^2.$$

## Implementation notes

Integrate this at the scalar prediction stage of a crystal graph neural network or atomistic transformer. First run each relaxed structure through the pretrained encoder to obtain a structure embedding \(h=\operatorname{Encoder}(\bm{x}^{\circ})\) and obtain its frozen 0 K energy \(U^{\circ}\) from the base potential. Concatenate \(h\), normalized temperature \(\tilde T\), and normalized pressure \(\tilde P\) in a two- or three-layer MLP residual head, producing \(g_\delta=\operatorname{MLP}_\delta(h,\tilde T,\tilde P)\); return \(\hat G=U^{\circ}+g_\delta\). During each batch, mark \(\tilde T\) and \(\tilde P\) as differentiable, compute \(\partial_T\hat G\) and \(\partial_P\hat G\) with reverse-mode autodiff, and optimize the loss above. Use per-dataset normalization for energy, entropy, and volume so derivative terms have comparable scale. Start with the encoder frozen, then unfreeze only the final encoder block if residual fine-tuning underfits. Estimate no mathematical quantities: \(U^{\circ}\) is directly supplied by the pretrained model, while all derivatives are exact autodiff derivatives of the learned scalar. The first cheap experiment should use a small pretrained crystal GNN, a few hundred structures sampled over several \((T,P)\) points, and compare against (a) direct \(G\) prediction with an identical MLP and (b) independent \(G,S,V\) heads. Evaluate held-out free-energy MAE, derivative MAE, Maxwell-consistency error from mixed derivatives, and phase-crossing temperature error for pairs of structures. Success is lower error at the same number of expensive labels, smoother phase diagrams, and substantially smaller violation of derivative identities.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.