# Entropy-Feedback Zeroth-Order Cooling - ID: 896 - Canonical URL: https://synthcore.org/idea/896/entropy-feedback-zeroth-order-cooling - API JSON: https://synthcore.org/api/idea/896.json - API Markdown: https://synthcore.org/api/idea/896.md - Verification status: mech_ok_no_baseline - Source: [arXiv:2607.14245](https://arxiv.org/abs/2607.14245) - Category: dynamics - Solves: stability, speedup, sample-efficiency - ML areas: optimizer, training-dynamics, fine-tuning - Math tags: information-theory, optimization, stochastic-processes, control-theory, dynamical-systems - Ratings: usefulness 7/10; difficulty 5/10; novelty 6/10 ## Idea description Replace a fixed temperature schedule in a population-based, derivative-free neural-network optimizer with a feedback controller driven by the entropy of candidate importance weights. When candidate losses are diffuse, the optimizer cools rapidly to exploit progress; when one or a few candidates dominate, cooling slows to prevent irreversible population collapse and loss of exploration. ## Mathematical statement At optimization iteration t, sample N candidate parameter vectors theta_i(t)=theta_t+sigma_t epsilon_i, where epsilon_i is drawn from a standard Gaussian and J_i is the candidate minibatch loss. Define temperature-tau importance weights w_i=exp(-(J_i-J_min)/tau_t) divided by the sum over candidates, Shannon entropy H_t=-sum_i w_i log(w_i), and normalized entropy h_t=H_t/log(N). The effective sample size is ESS_t=exp(H_t), so entropy directly measures population collapse. Let h_c be the critical normalized entropy and delta a smoothing width. Define b(h)=sigmoid((h-h_c)/delta) and update tau_{t+1}=tau_t exp(-alpha[epsilon+(1-epsilon)b(h_t)]), with cooling gain alpha>0 and small mandatory rate epsilon>0. Cooling is fast for h_t well above h_c and slow but nonzero below h_c. Update the parameter center with the weighted candidate mean and optionally damping gamma. ## Key formulas - $$w_i(t)=\frac{\exp\left(-(J_i-J_{\min})/\tau_t\right)}{\sum_{j=1}^{N}\exp\left(-(J_j-J_{\min})/\tau_t\right)},\qquad H_t=-\sum_{i=1}^{N}w_i(t)\log w_i(t),\qquad h_t=\frac{H_t}{\log N}.$$ - $$\mathrm{ESS}_t=\exp(H_t),\qquad \frac{\mathrm{ESS}_t}{N}=\exp\left(-(1-h_t)\log N\right).$$ - $$b(h_t)=\frac{1}{1+\exp(-(h_t-h_c)/\delta)},\qquad \tau_{t+1}=\tau_t\exp\left[-\alpha\left(\varepsilon+(1-\varepsilon)b(h_t)\right)\right].$$ - $$\theta_{t+1}=(1-\gamma)\theta_t+\gamma\sum_{i=1}^{N}w_i(t)\theta_t^{(i)},\qquad \theta_t^{(i)}=\theta_t+\sigma_t\epsilon_i,\quad \epsilon_i\sim\mathcal{N}(0,I).$$ ## Implementation notes 1. Integration point: implement this as a zeroth-order optimizer for a small neural network, derivative-free fine-tuning of a pretrained model, or optimization of prompts or low-rank adapter parameters. At each optimizer step, draw N perturbations of the current trainable parameter vector, evaluate every candidate on the same minibatch, and replace the ordinary fixed-temperature softmax weighting and hand-designed cooling schedule with the entropy controller above. Use antithetic perturbations when memory permits and keep a separate validation minibatch for monitoring collapse. 2. Pseudocode: ```text initialize theta, tau=tau0 for t in range(T): eps = Normal(0,I,N) candidates = theta + sigma * eps losses = [loss(model(candidate), minibatch) for candidate in candidates] z = -(losses - min(losses)) / tau w = softmax(z) H = -sum(w * log(w + 1e-12)) h = H / log(N) b = sigmoid((h - hcrit) / delta) theta_mean = sum_i(w[i] * candidates[i]) theta = (1-gamma)*theta + gamma*theta_mean tau = tau * exp(-alpha*(epsilon_floor + (1-epsilon_floor)*b)) ``` 3. Computed directly from the mechanism: normalized importance-weight entropy, effective sample size, and the thresholded temperature update. Estimated empirically: threshold h_c, cooling gain alpha, perturbation scale sigma, and damping gamma. Start with h_c=0.5, delta=0.05, alpha=0.05, epsilon=0.02, and gamma=1; tune only h_c in the first ablation. Clip the logits before softmax for numerical stability. 4. First cheap experiment: optimize a two-layer ReLU MLP on MNIST using N=64 or 128 Gaussian candidates per step and compare against the same zeroth-order optimizer with exponential cooling, constant temperature, and a manually tuned ESS floor. Run five seeds on a 10,000-example subset. The quantitative prediction is that normalized entropy will repeatedly approach h_c from above rather than falling immediately toward zero; the slope of log(tau_t) should change from approximately -alpha when h_t>h_c+2delta to approximately -alpha epsilon when h_t