Entropy-to-Contraction Attractor Regularization

Use a contractive branch module as the transition map of an RNN or world model, or as the decoder dynamics of a generative model. Define each branch as f_i(x) = s_i tanh(W_i x + b_i), with spectral normalization or power iteration enforcing ||Df_i(x)||_op <= s_i < 1. At every training step, sample branch indices from p and propagate states. Estimate H_hat from the branch probabilities. Estimate chi_hat by averaging minus log2 of the estimated Jacobian operator norm along sampled trajectories; one or two power iterations per Jacobian are sufficient for the first prototype. Sample length-n branch words, apply their composed maps to several probe states, and penalize pairs whose average endpoint distance is below c exp(-a n). Add the ratio penalty lambda_R times positive_part(r0 - H_hat divided by chi_hat) squared, plus lambda_S times L_sep, to the task loss. The entropy-to-contraction bound and separation mechanism come from the paper; Jacobian norms, separation distances, and attractor dimension are estimated empirically. The first cheap experiment should use a two-dimensional contractive RNN with four branches trained on a mixture of spirals or a chaotic-map trajectory. Compare spectral normalization, entropy regularization, separation regularization, and the combined method. Sweep contraction scale and branch-logit temperature. The predicted signature is that measured box or correlation dimension increases approximately linearly with H_hat divided by chi_hat below d, then saturates near d. If minimum length-n separation falls below c exp(-a n), measured dimension should fall below the ratio. Perturbation norms should decay as approximately 2 to the power of minus chi_hat times t, with measured log-decay slope within 20 percent of minus chi_hat times ln(2).