Parity-block curvature preconditioner

Choose a concrete involution with a known network meaning. The cheapest test is a two-branch MLP or CNN with branch-swap symmetry: $S$ swaps the branch parameters, so $P_+$ gives shared or even parameters and $P_-$ gives antisymmetric or odd parameters. A second test is a transformer with paired attention heads, where $S$ exchanges each head pair. At every step compute $g_+=P_+g$ and $g_-=P_-g$, maintain separate RMS or diagonal curvature estimates $A_+$ and $A_-$, and update $\theta\leftarrow\theta-\eta_+A_+g_+-\eta_-A_-g_-$. Estimate each sector's top generalized curvature by one Hutchinson or Lanczos pass restricted to that sector. If the implemented symmetry is only a coordinate relabeling with a non-Euclidean parameter metric, construct a basis $Q=[B_+|B_-]$ by eigendecomposition of $S$ and solve $H_\pm v=\lambda G_\pm v$ with $G_\pm=B_\pm^TB_\pm$ rather than treating the basis as orthonormal. The paper supplies the parity-sector and non-orthogonal congruence reduction; curvature estimates and optimizer states are empirical. First experiment: train a two-branch CIFAR-10 CNN with tied initialization and branch-swap-invariant loss, comparing AdamW against sector-wise AdamW. Log $\lambda_{\max,+}$, $\lambda_{\max,-}$, gradient norms, and validation accuracy. The prediction is that the measured stable step-size boundary for independently scaled sectors follows $\eta_{\pm,c}=2/\lambda_{\max,\pm}$ within 20%, and that one sector can tolerate at least a 1.5-fold different learning rate whenever its top curvature differs by that factor.