# Davis–Wielandt Shell Constraint for Heterogeneous SSMs

- ID: 2682
- Canonical URL: https://synthcore.org/idea/2682/davis-wielandt-shell-constraint-for-heterogeneous-ssms
- API JSON: https://synthcore.org/api/idea/2682.json
- API Markdown: https://synthcore.org/api/idea/2682.md
- Verification status: mechanism_failed
- Source: [arXiv:2608.21984](https://arxiv.org/abs/2608.21984)
- Category: dynamics
- Solves: stability, accuracy
- ML areas: rnn, ssm, architecture, training-dynamics
- Math tags: control-theory, dynamical-systems, linear-algebra, spectral-theory, numerical-analysis
- Ratings: usefulness 7/10; difficulty 6/10; novelty 7/10

## Idea description

Build a recurrent or state-space network from heterogeneous dynamical modules and characterize each module through sampled frequency-response passivity and Davis–Wielandt shell bounds. Constrain inter-module coupling so that the composed frequency response retains a positive passivity margin, providing a model-based alternative to blindly shrinking all recurrent weights.

## Mathematical statement

For an operator or frequency-response matrix G(jω), its Davis–Wielandt shell is the set of pairs (z,q) generated by unit vectors x, where z=x*Gx is the numerical-range response and q=||Gx||² is squared amplification. Define the local passivity index p(G) as the minimum Hermitian-part eigenvalue over frequencies and the gain index g(G) as the maximum singular value over frequencies. For feedback coupling u=-Ky, use the conservative compositional certificate p_comp ≥ p_blocks - ||K||₂ g_blocks²; enforce p_comp>0. Here G is a module transfer matrix, ω is angular frequency, x is a unit complex vector, K is the inter-module coupling matrix, and ||·||₂ is the spectral norm.

## Key formulas

- $$\mathcal{DW}(G)=\{(z,q)\in\mathbb{C}\times\mathbb{R}_{+}:z=x^{\ast}Gx,\ q=x^{\ast}G^{\ast}Gx=\|Gx\|_{2}^{2},\ \|x\|_{2}=1\}.$$
- $$p(G)=\inf_{\omega\in[0,\omega_{\max}]}\lambda_{\min}\!\left(\frac{G(j\omega)+G(j\omega)^{\ast}}{2}\right),\qquad g(G)=\sup_{\omega\in[0,\omega_{\max}]}\|G(j\omega)\|_{2}.$$
- $$p_{\mathrm{comp}}\ge p_{\mathrm{blocks}}-\|K\|_{2}g_{\mathrm{blocks}}^{2}>0.$$
- $$G_r(e^{j\omega})=C_r(e^{j\omega}I-A_r)^{-1}B_r+D_r,$$

## Implementation notes

1. Integration point: use a heterogeneous recurrent or state-space model with modules indexed by r. Each module has local dynamics x⁽ʳ⁾_{t+1}=A_r x⁽ʳ⁾_t+B_r u⁽ʳ⁾_t and output y⁽ʳ⁾_t=C_r x⁽ʳ⁾_t+D_r u⁽ʳ⁾_t; couple outputs through u=-Ky+B_u v_t. Apply the constraint during optimizer steps or as a periodic projection of K. 2. Pseudocode: every M updates, freeze parameters and collect representative hidden states; linearize each nonlinear module around those states; evaluate G_r(e^{jω}) on a logarithmic grid using the displayed transfer formula; compute p_r=minω λ_min((G_r+G_r*)/2) and g_r=maxω σ_max(G_r); set p_blocks=min_r p_r and g_blocks=max_r g_r; calculate p_bound=p_blocks-||K||₂g_blocks². If p_bound<ε, scale K by α=min(1,(p_blocks-ε)/(||K||₂g_blocks²)) or reject the update. 3. The paper-derived quantities are shell coordinates, passivity margins, gain estimates, and the compositional inequality. The minibatch linearization, finite frequency grid, and frequency cutoff are empirical approximations. 4. First cheap experiment: compare a 2–4 block gated SSM or GRU on sequential MNIST and copy-memory against unconstrained coupling and spectral normalization. Sweep coupling scale α and record p_bound, hidden-state norm, Jacobian amplification, and long-horizon error. The quantitative prediction is a degradation boundary near p_bound=0: positive-margin runs should have bounded rollout amplification, while negative-margin runs should exhibit rapidly increasing hidden-state norms or prediction error. The measured critical coupling should be within 20% of ||K||₂=p_blocks/g_blocks².

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built a heterogeneous two-module discrete-time SSM toy with frequency-response indices, Davis–Wielandt shell verification, conservative coupling projection, and rollout sweeps. The shell identity error was 2.7e-15 and the certificate scaled exactly with slope -g^2 (relative error 1.8e-16); the predicted certificate crossing was observed at 0.00347. However, actual dynamical instability occurred near coupling 0.240, about 69x larger than the certified boundary, so the claimed degradation boundary within 20% was not confirmed. The mechanism is mathematically valid but substantially conservative in this setup.

### Mechanism check

- Verdict: Built a heterogeneous two-module discrete-time SSM toy with frequency-response indices, Davis–Wielandt shell verification, conservative coupling projection, and rollout sweeps. The shell identity error was 2.7e-15 and the certificate scaled exactly with slope -g^2 (relative error 1.8e-16); the predicted certificate crossing was observed at 0.00347. However, actual dynamical instability occurred near coupling 0.240, about 69x larger than the certified boundary, so the claimed degradation boundary within 20% was not confirmed. The mechanism is mathematically valid but substantially conservative in this setup.
- Confidence: 9/10
- Limitations: Only a small linear two-module toy was tested; there was no GRU/SSM training task, sequential MNIST or copy-memory evaluation, optimizer integration, nonlinear linearization, FLOP/speed measurement, or comparison against spectral normalization. The finite frequency grid and scalar/diagonal module construction also do not establish usefulness for general heterogeneous neural SSMs.

## Artifacts

- [dw_experiment.py](https://synthcore.org/code/1036/dw_experiment.py)
- [report.md](https://synthcore.org/code/1036/report.md)
- [results.json](https://synthcore.org/code/1036/results.json)
- [Download all files as ZIP](https://synthcore.org/download/1036)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.