# Coarsening-Aware Global-Consensus Scheduler - ID: 636 - Canonical URL: https://synthcore.org/idea/636/coarsening-aware-global-consensus-scheduler - API JSON: https://synthcore.org/api/idea/636.json - API Markdown: https://synthcore.org/api/idea/636.md - Verification status: mechanism_failed - Source: [arXiv:2607.09411](https://arxiv.org/abs/2607.09411) - Category: dynamics - Solves: stability, generalization, accuracy - ML areas: optimizer, scheduler, training-dynamics, regularization - Math tags: statistical-mechanics, dynamical-systems, optimization, graph-theory - Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10 ## Idea description Modify learning-rate or annealing schedules so that local improvement is not mistaken for convergence when different parameter blocks occupy incompatible global modes. Measure a local-consistency score and a global-coherence score separately; slow training whenever local consistency is high but global coherence remains low, allowing competing parameter domains to merge before cooling further. ## Mathematical statement The source model is an Ising Hamiltonian $H=J_1\sum_{\langle i,j\rangle}s_i s_j+J_2\sum_{\langle\langle i,j\rangle\rangle}s_i s_j$, where $s_i\in\{-1,+1\}$, $J_1>0$ is the nearest-neighbor antiferromagnetic coupling, and $J_2>0$ selects stripe order. Let $P_{\mathrm{stripe}}(n,L)$ be the probability of a globally stripe-ordered final state after $n$ local-update sweeps on a system of linear size $L$. The kinetic time $n^*$ is defined by $P_{\mathrm{stripe}}(n^*,L)=p_0$ for a fixed threshold $p_0$, and the paper reports effective coarsening compatible with $n^*(L)\propto L^z$ with $z\gtrsim2$ over the simulated range. For a neural network partitioned into blocks $b=1,\ldots,B$, define local consistency $Q_t=B^{-1}\sum_b q_b(t)$, where $q_b$ is the mean cosine similarity of neighboring parameter-update vectors inside block $b$, and define global coherence $C_t=\lambda_1(G_t)/\operatorname{tr}(G_t)$, where $G_t=B^{-1}\sum_b(u_b-\bar u)(u_b-\bar u)^\top$ is the covariance of normalized block update directions $u_b$, and $\lambda_1$ is its largest eigenvalue. The adapted controller slows the schedule when $Q_t$ is high but $C_t$ is low: local consistency has been restored, but competing global update sectors remain. ## Key formulas - $$H=J_1\sum_{\langle i,j\rangle}s_i s_j+J_2\sum_{\langle\langle i,j\rangle\rangle}s_i s_j,\qquad s_i\in\{-1,+1\},\quad J_1>0,\ J_2>0.$$ - $$P_{\mathrm{stripe}}(n^*,L)=p_0,\qquad n^*(L)\propto L^z,\quad z\gtrsim2.$$ - $$Q_t=\frac{1}{B}\sum_{b=1}^{B}q_b(t),\qquad q_b(t)=\frac{1}{|E_b|}\sum_{(i,j)\in E_b}\frac{\Delta\theta_i\cdot\Delta\theta_j}{\|\Delta\theta_i\|\,\|\Delta\theta_j\|},$$ - $$C_t=\frac{\lambda_1(G_t)}{\operatorname{tr}(G_t)},\quad G_t=\frac{1}{B}\sum_{b=1}^{B}(u_b-\bar u)(u_b-\bar u)^\top,\quad \eta_{t+1}=\eta_t\exp\{-\kappa[1-Q_t][C_0-C_t]_+\},$$ ## Implementation notes 1. Integration point: partition a small MLP, CNN, or transformer into blocks such as layers, attention heads, convolutional channels, or parameter shards. Every $K$ optimizer steps, collect the recent update vector $\Delta\theta_b$ for each block and normalize it to $u_b$. Define the block adjacency graph $E_b$ using neighboring layers, adjacent channels, or attention heads in the same layer. 2. Pseudocode: initialize learning rate $\eta$ and target global coherence $C_0$. At each diagnostic interval, compute pairwise cosine similarities on adjacent update vectors to obtain $Q_t$; compute the covariance $G_t$ of block-level directions and its leading-eigenvalue ratio $C_t$ using a low-rank eigensolver; if $Q_t>q_0$ and $C_t