Unverified 2026

Hamiltonian Shape-Attractor Optimizer

Implementation & benchmark of arXiv:2608.07672 — Classical fractons with cosmological fixed points

Usefulness6/10

Difficulty6/10

Novelty8/10

Dynamics Stability Generalization Optimizer Training dynamics Mlp Initialization Dynamical systems Control theory Geometry Statistical mechanics

Source paper: Classical fractons with cosmological fixed points arXiv:2608.07672 ⓘ · analyzed Sep 1, 2026

AI-generated research hypothesis, automatically tested. Not peer-reviewed.

Idea description

Represent each trainable parameter block as a global scale multiplied by a normalized shape, and evolve the shape through a projected Hamiltonian optimizer. The optimizer is designed so that normalized weights can approach a stable central configuration while auxiliary momenta retain phase-space volume that prevents ordinary Hamiltonian dynamics from having a full-space attractor.

Formulas

$$\theta=Rs,\qquad R=\|\theta\|_2,\qquad s=\frac{\theta}{\|\theta\|_2},\qquad \Pi_s=I-ss^\top.$$

$$U_q(s)=\sum_{i<j}w_{ij}\|s_i-s_j\|^q,\qquad \Pi_s\nabla_sU_q(s)=0\iff\nabla_sU_q(s)=\lambda s.$$

$$H_T=\Pi_s\nabla_s^2U_q(s)\Pi_s,\qquad \delta s_{k+1}\approx(I-\eta H_T)\delta s_k.$$

$$R(t)\propto|t|^{\alpha/(\alpha-\beta)},\qquad (\alpha,\beta)=(-2,1)\Rightarrow R(t)\propto|t|^{2/3}.$$

Mathematical statement

For a parameter vector $\theta\in\mathbb{R}^d$, write $\theta=Rs$, where $R=\|\theta\|_2>0$ is the scale and $s\in\mathbb{S}^{d-1}$ satisfies $s^\top s=1$. For pairwise shape energy $U_q(s)=\sum_{i<j}w_{ij}\|s_i-s_j\|^q$, a central configuration is a constrained stationary point satisfying $\Pi_s\nabla_sU_q(s)=0$, equivalently $\nabla_sU_q(s)=\lambda s$, where $\Pi_s=I-ss^\top$ projects onto the tangent space of the sphere and $\lambda$ is a Lagrange multiplier. The paper's scale-invariant Hamiltonian family has pair terms $\|p_i-p_j\|^{\alpha}\|x_i-x_j\|^{\beta}$, and homothetic solutions obey $R(t)\propto|t|^{\alpha/(\alpha-\beta)}$. For the distinguished exponents $(\alpha,\beta)=(-2,1)$, this becomes $R(t)\propto|t|^{2/3}$. Stability is tested by the tangent Hessian $H_T=\Pi_s\nabla_s^2U_q(s)\Pi_s$: positive eigenvalues at a local minimum predict contraction of shape perturbations under projected gradient dynamics.

Implementation notes

1. Integration point: replace ordinary SGD or Adam updates for each weight matrix $W_l$ by an augmented update on a normalized shape $S_l=W_l/(\|W_l\|_F+\epsilon)$, a scalar scale $R_l$, and an auxiliary momentum $P_l$. Construct a cheap graph over neurons or output channels; use pair weights $w_{ij}=1$ within each layer or k-nearest-neighbor weights to avoid quadratic cost. 2. Pseudocode: compute the task gradient $G_l=\nabla_{W_l}L$; compute the tangent gradient $G_l^T=G_l-S_l\langle S_l,G_l\rangle_F$; calculate the central-shape force $F_l=\Pi_{S_l}\nabla_{S_l}U_q$; update $P_l\leftarrow P_l-\eta_p(R_lG_l^T+\gamma F_l)$, then update $S_l\leftarrow\operatorname{Normalize}(S_l+\eta_sP_l)$, and finally set $W_l=R_lS_l$. Update $R_l$ using the task-gradient component parallel to $S_l$, or test the prescribed scale schedule $R_l(t)=R_l(1)(t/t_0)^{2/3}$. 3. Computed from the paper: scale-shape decomposition, projected central-configuration force, and the distinguished $2/3$ exponent. Estimated empirically: tangent-Hessian eigenvalues, shape convergence, and useful values of $q$, $\gamma$, and step sizes. 4. First cheap experiment: train a two-layer MLP on MNIST using SGD, Adam, and this optimizer with matched parameter counts and learning-rate sweeps. Record loss, each layer's scale, cosine distance between shapes at successive checkpoints, and the largest tangent-Hessian eigenvalue. The quantitative prediction is a stability boundary near $\eta_s\lambda_{\max}(H_T)=2$: below it, perturbations decay with factor approximately $1-\eta_s\lambda$; above it, normalized weights oscillate or diverge. In a task-free control experiment, fit log-log scale trajectories and test whether the imposed schedule has slope $2/3$ within 10 percent, while shape distances decay according to the dominant tangent-Hessian mode.

Verification

This idea has not been verified yet.

Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.

Artifacts

Artifacts unavailable.