# Contractive projected residual dynamics - ID: 575 - Canonical URL: https://synthcore.org/idea/575/contractive-projected-residual-dynamics - API JSON: https://synthcore.org/api/idea/575.json - API Markdown: https://synthcore.org/api/idea/575.md - Verification status: beats_baseline - Source: [arXiv:2607.08301](https://arxiv.org/abs/2607.08301) - Category: dynamics - Solves: stability, generalization, accuracy - ML areas: rnn, ssm, training-dynamics, regularization - Math tags: dynamical-systems, convex-analysis, geometry, functional-analysis - Ratings: usefulness 6/10; difficulty 5/10; novelty 5/10 ## Idea description Build a recurrent or continuous-depth block from a dissipative vector field and project every state derivative onto the tangent cone of a closed convex hidden-state set. Unlike ordinary clipping, tangent-cone projection removes only the outward component at the boundary and preserves admissible motion. Under the paper's maximal-dissipativity result, the continuous flow is nonexpansive in its initial state. ## Mathematical statement Let X be a Hilbert space, K a closed convex subset, and A_theta the learned drift. The tangent cone at x is T_K(x)=closure of {alpha(z-x): alpha>=0,z in K}, and the normal cone is N_K(x)={n: inner_product(n,z-x)<=0 for every z in K}. The constrained dynamics are dx/dt in A_theta(x,v)-N_K(x), equivalently dx/dt=Pi_{T_K(x)} A_theta(x,v) when the projection is single-valued. If A_theta is maximal dissipative, meaning inner_product(A_theta(x,v)-A_theta(z,v),x-z)<=0 for all x,z in K, the paper proves that the constrained operator remains maximal dissipative and generates a contraction semigroup. Thus two states satisfy ||x_1(t)-x_2(t)||<=||x_1(0)-x_2(0)||. The paper's equation (31) gives ||[y_tau1,x_1(tau),u_tau1]^T-[y_tau2,x_2(tau),u_tau2]^T|| <= ||[y_01,x_01,u_01]^T-[y_02,x_02,u_02]^T||, while the expanded energy inequality bounds output discrepancy, terminal state discrepancy, and future input discrepancy by initial state and total input discrepancy. In the neural adaptation, x is the hidden state, v is the input, K is a box or Euclidean ball, and A_theta is a parameterized dissipative drift. ## Key formulas - $$N_K(x)=\{n\in X:\langle n,z-x\rangle\le 0\ \forall z\in K\},\qquad T_K(x)=\overline{\{\alpha(z-x):\alpha\ge 0,\ z\in K\}}$$ - $$\frac{d^+x(t)}{dt}\in A_\theta(x(t),v(t))-N_K(x(t)),\qquad \frac{dx(t)}{dt}=\Pi_{T_K(x(t))}A_\theta(x(t),v(t))$$ - $$\langle A_\theta(x,v)-A_\theta(z,v),x-z\rangle\le 0\quad\Longrightarrow\quad \|x_1(\tau)-x_2(\tau)\|^2\le \|x_1(0)-x_2(0)\|^2$$ - $$\int_0^\tau\|y_1(t)-y_2(t)\|_Y^2dt+\|x_1(\tau)-x_2(\tau)\|_X^2+\int_\tau^\infty\|u_1(t)-u_2(t)\|_U^2dt\le \|x_1(0)-x_2(0)\|_X^2+\int_0^\infty\|u_1(t)-u_2(t)\|_U^2dt$$ ## Implementation notes Integrate this into the hidden-state update of a recurrent model or an explicit Euler continuous-depth block. Let x_t in R^d be the hidden state and v_t the input embedding. Use K=[-1,1]^d initially. Parameterize the drift as A_theta(x_t,v_t)=-(L_theta^T L_theta+lambda I)x_t+B_theta v_t+b_theta, where L_theta and B_theta are learned matrices and lambda>=0; the negative semidefinite term enforces dissipativity in x. Compute q=A_theta(x_t,v_t). For each coordinate i, set p_i=q_i when -1