Cycle-aware softmax temperature control

Integrate this into the logits-to-probabilities operation of a recurrent attention block, iterative token mixer, or MoE router. For each monitored layer, define logits $z\in\mathbb{R}^m$, probabilities $p=\operatorname{softmax}(z/\tau)$, and a small nonnegative feedback proxy $K\in\mathbb{R}_+^{m\times m}$. A practical construction is to retain the previous-step activation $h_{t-1}$, compute the absolute Jacobian-vector effect from probability perturbations to the next logits, and approximate it with $K_{ij}=|\partial z_{t+1,i}/\partial p_{t,j}|$; estimate this using one-sided finite differences on a detached minibatch or one reverse-mode Jacobian-vector product per column block. Every 20-100 optimizer steps, run 3-5 power iterations: initialize positive $q$, repeatedly set $q\leftarrow Kq/(\|Kq\|_1+\epsilon)$, and estimate $\rho(K)=\|Kq\|_1/\|q\|_1$. Set $\tau$ with an EMA controller using $\tau_{target}=\max(\tau_{min},v\rho(K)/(\nu+g_{margin}))$, then update slowly via $\tau\leftarrow(1-\alpha)\tau+\alpha\tau_{target}$, stopping gradients through the controller. Here $v$ is a calibrated response scale, $\nu$ is an explicit damping proxy estimated from residual decay, and $g_{margin}>0$ prevents operation at the boundary. The matrix-exponential stability principle and spectral estimate come from the paper; $K$, $v$, and $\nu$ are empirical engineering proxies. First test on a 6-layer recurrent attention toy transformer and a Switch-style MoE on WikiText-2 or C4, comparing fixed temperatures and standard annealing at equal FLOPs. Track training loss variance, maximum router probability, expert-load entropy, estimated $\rho(K)/\tau$, gradient norm spikes, and validation perplexity. Success means fewer collapse events and lower loss at equal final temperature, with the estimated gain remaining below threshold.