Pseudo-Arclength Equilibrium Layer

1. Exact integration point: implement this as the forward solver for a small implicit MLP or equilibrium layer, for example $z=\tanh(Wz+Ux+c)$, whose residual is $F_\theta(z)=z-\tanh(Wz+Ux+c)$. At each continuation step, evaluate the residual $H$, Jacobian-vector products for $J$, and append the continuation vector $b$. The augmented linear solve can also be reused in implicit backward differentiation when the backward Jacobian is nearly singular. 2. Pseudocode: initialize $z=z_0$ and $\mu=0$; repeat: compute $H$, obtain $J$ by autodiff, compute $b=F(z_0)-y^*$, set $A=[J,b]$, form $r=-\partial_tH-kH$, solve $q=\operatorname{lstsq}(A,r)$ using QR or SVD, update $z\leftarrow z+\Delta s q_z$ and $\mu\leftarrow\mu+\Delta s q_\mu$, and stop when $\|H\|<\epsilon$. Reduce $\Delta s$ if the residual increases. 3. Computed quantities: $J$, singular values, the least-norm solve, and $w$ are computed numerically; $w$ is the smallest-left-singular-vector of $J$. The paper supplies the transversality mechanism, while fold locations and conditioning must be estimated for each network. If $|w^Tb|$ is too small, perturb $b$ with a normalized random direction and record the intervention. 4. First cheap experiment: use a two-dimensional equilibrium MLP with the explicit fold residual $F(z)=[z_1^2+z_2-y_1, z_2-y_2]$ and sweep $y_1$ through zero. Compare ordinary damped Newton, direct pseudoinverse Newton, and augmented continuation. Measure $\sigma_{\min}(J)$, $|w^Tb|$, update norm, residual, and branch tracking. The prediction is that direct Newton update norms scale as $1/\sigma_{\min}(J)$ and diverge near the fold, whereas augmented updates remain bounded whenever $|w^Tb|>0$. Deliberately rotate $b$ so $w^Tb\to0$; the augmented method should then show the predicted failure boundary through a sharp condition-number increase.