# Basin-Entropy Threshold Tuning

- ID: 94
- Canonical URL: https://synthcore.org/idea/94/basin-entropy-threshold-tuning
- API JSON: https://synthcore.org/api/idea/94.json
- API Markdown: https://synthcore.org/api/idea/94.md
- Verification status: unverified
- Source: [arXiv:2608.23225](https://arxiv.org/abs/2608.23225)
- Category: regularization
- Solves: generalization, stability, accuracy
- ML areas: rnn, ssm, regularization, loss
- Math tags: information-theory, dynamical-systems, combinatorics
- Ratings: usefulness 5/10; difficulty 4/10; novelty 8/10

## Idea description

Use the hysteresis threshold as a regularizer for attractor diversity. Estimate how many initial states converge to each fixed point and select thresholds that maximize basin entropy or penalize domination by one attractor, reducing attractor collapse in discrete recurrent classifiers and memory modules.

## Mathematical statement

Let the deterministic hysteretic dynamics map an initial binary state $x(0)$ to its eventual fixed point $z_k$, when convergence occurs. If $B_k$ is the basin of fixed point $z_k$ and $M$ sampled initial states are used, estimate $p_k=|B_k|/M$ by $\hat p_k=n_k/M$, where $n_k$ is the number of samples converging to attractor $k$. The basin entropy is $H(Th)=-\sum_{k=1}^{K}\hat p_k\log(\hat p_k)$, with $0\log 0=0$; for $K$ observed attractors, $H(Th)\le\log K$, with equality for uniform basin sizes. The threshold changes the deadband in the update $h$: increasing $Th$ causes more coordinates to retain their previous state whenever $|y_i|<Th$, thereby changing the number and relative sizes of basins. Since weights and fields are integer while candidate thresholds are half-integers, threshold selection should be performed by a discrete sweep. A training objective can combine task loss with negative basin entropy and an explicit largest-basin collapse penalty.

## Key formulas

- $$\hat p_k(Th)=\frac{n_k(Th)}{M},\qquad n_k(Th)=\sum_{m=1}^{M}\mathbf 1\{\operatorname{limit}(x^{(m)};Th)=z_k\},$$
- $$H(Th)=-\sum_{k=1}^{K(Th)}\hat p_k(Th)\log\hat p_k(Th),\qquad 0\log 0:=0,$$
- $$0\le H(Th)\le\log K(Th),$$
- $$\mathcal L_{\mathrm{total}}=\mathcal L_{\mathrm{task}}-\lambda H(Th)+\beta\max\left(0,\frac{n_{\max}}{M}-q\right)^2.$$

## Implementation notes

(1) Integration point: apply this to the hysteretic recurrent layer's threshold-selection and regularization loop, not to the ordinary classifier output. Candidate thresholds can be global, per layer, or grouped by channel. (2) Pseudocode: `for Th in {0.5,1.5,2.5,3.5}: sample M initial states; for each x0 run K hysteresis updates; if x[t+1]==x[t], hash x as attractor id; count n_k; p_k=n_k/M; H=-sum(p_k*log(p_k)); score=validation_loss-lambda*H`. Select the threshold with the best validation score subject to a convergence-rate constraint. During training, compute the detached scalar penalty `-lambda*H + beta*max(0,n_max/M-q)^2` and add it to task loss. (3) Basin counts, convergence times, attractor hashes, and entropy are estimated empirically. The paper's mathematical contribution supplies the basin interpretation and entropy criterion; no gradient through the discrete threshold is required. Use random Rademacher initial states for global basin coverage and noisy encoded examples for task-relevant basin coverage. (4) First experiment: train a 32-bit attractor classifier on MNIST with 10 classes, comparing a fixed threshold, a validation-selected threshold, and entropy-regularized thresholding against a sign-RNN. Log the number of attractors, normalized entropy $H/\log K$, largest-basin fraction, class-balanced accuracy, convergence rate, and robustness from noisy initial states. The hypothesis is reduced largest-basin collapse and improved minority-class or corruption performance, with a measurable tradeoff if high entropy produces slow convergence.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.