Orbit-Consistent Equivariant Distillation

1. Integration point: insert a symmetry-aware student policy and critic into the existing VLA distillation loop. Keep the vision-language encoder unchanged initially; apply known transformations to the spatial state representation and use an action head that predicts equivariant quantities such as planar translation, rotation, and gripper state. For a transformer, either rotate or reflect image tokens before the shared encoder or add transformed-token consistency losses. 2. Pseudocode: sample a minibatch $(s,a_T)$; sample one or more $g$; compute $a=f_\theta(s)$ and $a_g=f_\theta(L_gs)$; compute the teacher loss against $a_T$; add $\lambda_{eq}\|a_g-R_g(a)\|^2$; compute $V(s)$ and $V(L_gs)$ and add $\lambda_V(V(L_gs)-V(s))^2$; backpropagate the sum. If transformations are cheap, average over all elements of a small finite group; otherwise sample two transformations per batch. 3. Computed quantities: $L_g$ and $R_g$ come from the task geometry; teacher actions are computed by the frozen teacher; the equivariance error is measured directly, not estimated by a separate model. The coefficients $\lambda_{eq}$ and $\lambda_V$ are hyperparameters. 4. First cheap experiment: use a small CNN/MLP policy on a planar reaching or rotated-MNIST-style manipulation proxy, comparing ordinary distillation with orbit-consistent distillation at equal data budgets. Hold out rotations and reflections during training. The quantitative prediction is that $\epsilon_{eq}$ will decrease approximately monotonically with $\lambda_{eq}$ and reach at least a 5-fold reduction over baseline; transformed-task success should remain within 10% of canonical-task success, while the baseline gap should be substantially larger. Sweep the number of training orbits and test whether performance saturates after roughly one representative per group orbit rather than scaling linearly with the number of transformations.