# Least-Fixed-Set Propagation for Recurrent Networks

- ID: 1962
- Canonical URL: https://synthcore.org/idea/1962/least-fixed-set-propagation-for-recurrent-networks
- API JSON: https://synthcore.org/api/idea/1962.json
- API Markdown: https://synthcore.org/api/idea/1962.md
- Verification status: unverified
- Source: [arXiv:2608.07794](https://arxiv.org/abs/2608.07794)
- Category: geometry
- Solves: stability, accuracy, sample-efficiency
- ML areas: training, geometry, graph-nn
- Math tags: convex-analysis, geometry, computation, algebra
- Ratings: usefulness 5/10; difficulty 7/10; novelty 6/10

## Idea description

Represent the hidden state of a recurrent or implicit neural block by a convex reachable set and encode its recursive constraints as containment inequalities rather than unrolling a fixed number of steps. Eliminate the set variables to obtain the smallest representable invariant set, which can be used as a tighter robustness certificate, a training regularizer, or a principled initialization for equilibrium solvers.

## Mathematical statement

The paper studies systems of recursive containment inequalities of the form $X_i \supseteq \Phi_i(X_1,\ldots,X_n;P_1,\ldots,P_m)$, where each $X_i$ is an unknown convex subset of a finite-dimensional vector space, $P_j$ are parameter sets, and each $\Phi_i$ is constructed from finite unions, nonnegative dilations, Minkowski sums, convex combinations, convex hulls, and positive geometric joins. The order is set inclusion: smaller sets are more precise. The key result is that every parameter assignment has a unique smallest solution, and a finite Gaussian-elimination-type substitution procedure removes unknowns while preserving that smallest solution. The basic operations used by the implementation are $A+B=\{a+b:a\in A,b\in B\}$, $\alpha A=\{\alpha a:a\in A\}$ for $\alpha\geq 0$, $\operatorname{conv}(A,B)=\{\lambda a+(1-\lambda)b:0\leq\lambda\leq1\}$, and the strict positive join $A\mathring{\star}B=\{\lambda a+(1-\lambda)b:a\in A,b\in B,0<\lambda<1\}$. The extracted example uses substitution in $\varphi=P_2\hat{\oplus}X_2\mathring{\star}(\frac12X_1+\frac12P_4)$ with $\varphi_{X_1}=P_1\hat{\oplus}P_1\mathring{\star}(\frac12X_2+\frac12P_3)$, illustrating recursive replacement of $X_1$ by its defining expression. For a neural fixed-point map $h=\sigma(Ah+Bu+c)$ and input set $U$, instantiate $X\supseteq\sigma(AX+BU+c)$. For a piecewise-linear activation, propagate each activation region as a finite union of polytopes; for a conservative convex implementation, replace $\sigma$ by a convex outer relaxation and keep $\Phi(X)=AX+BU+c$ followed by that relaxation. The mathematical guarantee is inclusion preservation: the eliminated set must contain every reachable hidden state, while being the smallest solution within the selected closed representation family.

## Key formulas

- $$X_i \supseteq \Phi_i(X_1,\ldots,X_n;P_1,\ldots,P_m),$$
- $$A+B=\{a+b\mid a\in A,\ b\in B\},\qquad \alpha A=\{\alpha a\mid a\in A\},\quad \alpha\geq0,$$
- $$A\mathring{\star}B=\{\lambda a+(1-\lambda)b\mid a\in A,\ b\in B,\ 0<\lambda<1\},$$
- $$X\supseteq\sigma(AX+BU+c),\qquad \widehat X_{t+1}=\operatorname{OuterHull}\!\left(\sigma(A\widehat X_t+B U+c)\right),\quad \widehat X=\operatorname{Eliminate}(X).$$

## Implementation notes

(1) Integration point: add a set-valued analysis pass around a recurrent layer, DEQ layer, or message-passing block. The tensor-level network remains unchanged during the first experiment; the new component receives weight matrices $A,B$, bias $c$, an input region $U$, and activation bounds, and returns a polytope or union-of-polytopes approximation to the invariant hidden-state set. Use half-space polytopes $X=\{x\mid Hx\leq q\}$ for the MVP, with optional splitting into activation regions for ReLU. (2) Pseudocode: initialize symbolic variables $X_1,\ldots,X_T$ for recurrent states or block-local states; construct constraints $X_t\supseteq A X_{t-1}+B U+c$ and the activation outer-relaxation constraint; repeatedly choose a variable $X_i$, substitute its defining expression into every right-hand side, distribute over finite unions, and simplify Minkowski sums and dilations; after elimination, evaluate the resulting parameter-only expression as a polytope. For a cyclic equilibrium, retain one variable $X$ and solve the containment expression by widening: start with $X^{(0)}=\emptyset$, compute $X^{(r+1)}=\operatorname{OuterHull}(\Phi(X^{(r)}))$, and accelerate with symbolic substitutions whenever expressions repeat. (3) Compute exactly from the mathematical construction: union, affine image, nonnegative scaling, Minkowski sum, and convex hull. Estimate empirically: activation-region splits, polytope vertex reduction, and termination tolerances. Enforce inclusion after every reduction by solving LP containment checks or adding a safety margin to half-space offsets. (4) First experiment: use a 2-layer ReLU RNN or small DEQ on MNIST sequential pixels, with hidden width 16-32. Compare fixed-point reachable sets from this method against interval bound propagation, zonotopes, and 50-step numerical unrolling on adversarial input boxes. Measure certified output margins, set volume, wall-clock cost, and failure rate of the equilibrium solver. Then train with an auxiliary penalty proportional to the output reachable-set volume or certified-logit loss. Success means tighter certified margins at equal analysis time, fewer unstable equilibrium iterations, or improved robust accuracy at equal clean accuracy; a negative result is either exponential polytope growth or no reduction in bound volume over zonotopes.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.