# SRB Entropy-Lyapunov Regularizer

- ID: 867
- Canonical URL: https://synthcore.org/idea/867/srb-entropy-lyapunov-regularizer
- API JSON: https://synthcore.org/api/idea/867.json
- API Markdown: https://synthcore.org/api/idea/867.md
- Verification status: unverified
- Source: [arXiv:2607.13530](https://arxiv.org/abs/2607.13530)
- Category: regularization
- Solves: stability, accuracy, generalization
- ML areas: rnn, ssm, world-model, regularization, training-dynamics
- Math tags: dynamical-systems, information-theory, differential-geometry, linear-algebra, statistics
- Ratings: usefulness 6/10; difficulty 6/10; novelty 8/10

## Idea description

Add an entropy-Lyapunov consistency term to a recurrent or state-space model whose learned dynamics are intended to reproduce a chaotic invariant distribution. The regularizer targets the equality condition h_mu(f) = sum_i max(lambda_i, 0), while a dominated-splitting diagnostic determines whether the theorem assumptions are approximately plausible instead of blindly forcing equality.

## Mathematical statement

Let f_theta: R^d -> R^d be the learned one-step map, mu_theta its empirical invariant measure, h_mu_theta(f_theta) its metric entropy rate, and lambda_i its Lyapunov exponents counted with multiplicity. The Margulis-Ruelle inequality gives h_mu_theta(f_theta) <= sum_i max(lambda_i, 0); the SRB or Pesin condition is equality when the map has sufficient regularity and a dominated splitting. Estimate the positive-exponent sum with QR-based tangent propagation and estimate entropy rate from conditional next-state log likelihood or a nearest-neighbor entropy estimator. Use the nonnegative gap G = sum_i max(lambda_i, 0) - h_mu_theta(f_theta) as a diagnostic or soft penalty. A candidate splitting E direct-sum F is dominated when ||Df_theta^n restricted to E|| times ||(Df_theta^n restricted to F)^(-1)|| <= C rho^n for C > 0 and 0 < rho < 1. The regularity requirement can be approximated by checking whether the Jacobian modulus has finite Dini integral integral from 0 to 1 of omega_Df(r)/r dr.

## Key formulas

- $$h_{\mu}(f)\leq \sum_{i:\lambda_i(\mu,f)>0}\lambda_i(\mu,f),\qquad G(\theta)=\sum_{i=1}^{d}\max\{\lambda_i(\theta),0\}-h_{\mu_\theta}(f_\theta)\geq 0.$$
- $$\mathcal{L}_{\mathrm{total}}=\mathcal{L}_{\mathrm{data}}+\beta\,\mathrm{softplus}\!\left(G(\theta)-\tau\right)^2+\gamma\,\mathcal{L}_{\mathrm{dom}}.$$
- $$\mathcal{L}_{\mathrm{dom}}=\frac{1}{K}\sum_{n\in\mathcal{N}}\left[\log\left(\|Df_\theta^n|_E\|\,\|(Df_\theta^n|_F)^{-1}\|\right)-\log C-n\log\rho\right]_+.$$

## Implementation notes

Integrate this after the ordinary prediction loss in a differentiable GRU, neural ODE discretization, or state-space world model. First train with the data loss alone for a warm-up period, then activate the regularizer after trajectories approach their empirical invariant distribution. Maintain long trajectory batches with states sampled from the model. Every q steps, propagate r random tangent vectors with Jacobian-vector products, v_(t+1) = Df_theta(x_t)v_t, and apply QR orthogonalization. Accumulate the logarithms of the QR diagonal factors; their averages estimate the Lyapunov exponents. For the first MVP, stop gradients through these estimates and update the penalty periodically. Estimate entropy rate using the model conditional Gaussian negative log likelihood, h_hat = T^(-1) sum_t[-log p_theta(x_(t+1)|x_t)], after calibrating the additive noise-volume constant. Alternatively use a k-nearest-neighbor entropy-rate estimator on saved states. For a candidate split of the largest k and smallest d-k singular directions, evaluate products of restricted Jacobian norms over several horizons n. Fit log products against n to estimate C and rho, and apply the dominated-splitting penalty when the observed decay is slower than log(C) + n log(rho). The paper supplies the inequality and equality criterion; entropy, exponents, and splitting parameters are estimated empirically. First experiment: train 2D and 4D GRUs on noisy Lorenz-63 trajectories, comparing prediction loss alone, Lyapunov-spectrum matching, and this gap penalty. Evaluate long-horizon invariant-measure statistics, forecast error, and estimated entropy. The predicted signature is that the empirical gap G decreases toward zero only in runs with an approximately dominated unstable/stable split, while the entropy estimate remains below the positive-exponent sum. Increasing beta should eventually collapse positive exponents toward zero, producing a measurable transition from chaotic to fixed-point or periodic behavior.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.