Dynamical-Degree Expansion Scheduler
Implementation & benchmark of arXiv:2607.08125 — Complex dynamics perspective for birational maps of the plane arising from cluster algebra mutations
Source paper: Complex dynamics perspective for birational maps of the plane arising from cluster algebra mutations arXiv:2607.08125 ⓘ · analyzed Aug 30, 2026
AI-generated research hypothesis, automatically tested. Not peer-reviewed.
Idea description
Use the tropical dynamical degree as an analytic expansion budget for repeated neural blocks. Layers with $pq>4$ deliberately expand along a known tropical eigendirection, while layers with $pq\leq4$ avoid exponential asymptotic growth; a schedule can therefore increase representational mixing without allowing hidden-state norms to explode.
Formulas
Mathematical statement
The tropicalized mutation $A=A_2\circ A_1$ is piecewise linear and has largest eigenvalue $\lambda_+=(pq-2+\sqrt{pq(pq-4)})/2$ when $pq>4$. The paper identifies this largest eigenvalue with the birational map's dynamical degree, meaning that it controls asymptotic degree or orbit growth. For $pq\leq4$, the dynamical degree is $1$, while for $pq>4$ it exceeds $1$. In a neural stack, let $h_{\ell+1}=N_\ell(A_{p_\ell,q_\ell}(h_\ell))$, where $N_\ell$ is a norm-control operation and $p_\ell,q_\ell$ are layer parameters. The proposed engineering rule is to choose a target per-layer growth budget $\rho_\ell$ and require $\lambda_+(p_\ell,q_\ell)\leq\rho_\ell$ for expansive layers. Here $h_\ell$ is the hidden state at depth $\ell$, $\rho_\ell\geq1$ is a prescribed expansion ceiling, and $\lambda_+$ is computed exactly rather than estimated from noisy minibatch Hessians or Jacobians.
Implementation notes
Integrate the rule at initialization and during architecture scheduling for a deep stack of tropical mutation blocks or any repeated piecewise-linear mixer. Partition channels into pairs and assign each layer an integer pair $(p_\ell,q_\ell)$. Before training, compute $\lambda_+(p_\ell,q_\ell)$ from the closed-form formula; begin with all layers in the non-expansive regime $pq\leq4$, then progressively replace selected layers with pairs satisfying $pq>4$ as training proceeds. A simple schedule is rho = 1 + (rho_max-1)*min(step/warmup,1); at each milestone choose the largest candidate $(p,q)$ whose exact $\lambda_+$ is at most rho. After each tropical block, apply the norm controller $N_\ell$: calculate the current pairwise or full-token norm, and rescale only when it exceeds the layer budget $c_\ell$ times the incoming norm. The mathematical quantities are the exact branch map and exact $\lambda_+$; empirical quantities are hidden-state norms, gradient norms, and the observed multi-step amplification $g_k=\|h_{\ell+k}\|/\|h_\ell\|$. Validate first on a 12-layer CIFAR-10 MLP-Mixer or small language-model residual stack. Compare fixed $(2,2)$, fixed expansive $(3,2)$, and the scheduled rule at equal FLOPs. Record loss smoothness, gradient clipping frequency, activation overflow, and final accuracy. The hypothesis is that scheduling reaches the accuracy of the expansive model while reducing exploding-gradient events and improving early loss descent versus the always-conservative baseline.
Verification
This idea has not been verified yet.
Verification happens in two stages: Stage 1 — a mechanism check on a toy system confirms the claimed mathematical phenomenon reproduces; Stage 2 — a benchmark implements the idea on a real (small) neural network task and compares it against a tuned baseline over 8 paired seeds with a permutation test.
Artifacts
Artifacts unavailable.