# Positive Reflected Bellman Layer

- ID: 2155
- Canonical URL: https://synthcore.org/idea/2155/positive-reflected-bellman-layer
- API JSON: https://synthcore.org/api/idea/2155.json
- API Markdown: https://synthcore.org/api/idea/2155.md
- Verification status: unverified
- Source: [arXiv:2608.11936](https://arxiv.org/abs/2608.11936)
- Category: architecture
- Solves: stability, accuracy, generalization
- ML areas: world-model, graph-nn, training-dynamics
- Math tags: stochastic-processes, probability, dynamical-systems
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Replace an unconstrained spatial aggregation in a neural PDE surrogate or controlled-dynamics model with a fixed-branch expectation layer. Each output is a maximum over controls of a nonnegative weighted average of next-state values, with reflected overshoots attenuated by Robin factors. Increasing any input value therefore cannot decrease the output, giving a hard monotonicity and positivity property instead of relying on a penalty.

## Mathematical statement

The paper defines the HJB Hamiltonian H(t,x,p,M,u) as a supremum over controls a of diffusion, drift, reaction, and source terms. For a Rademacher vector xi in {-1,+1}^m, there are P = 2^m equally probable branches. For control a, the weak-Euler proposal is y(a,xi) = x + mu(t,x,a) Delta_t + sigma(t,x,a) sqrt(Delta_t) xi. If y exits the domain, it is reflected using an oblique boundary direction gamma; D(a,xi) is twice the overshoot and ybar(a,xi) is the reflected point. The Robin coefficient kappa enters through exp(-kappa D) and D exp(-kappa D/2). A practical positive update is T[V](x) = max_a { (1/P) sum_xi exp(-r_a Delta_t) [ exp(-kappa_a,xi D_a,xi) V(ybar_a,xi) + D_a,xi exp(-kappa_a,xi D_a,xi / 2) g_a,xi ] + Delta_t f_a }. Here V is the next-time value, r and kappa are nonnegative coefficients, f is the interior source, and g is the boundary source. All coefficients multiplying V are nonnegative, so V <= W implies T[V] <= T[W]; nonnegative V, f, and g imply T[V] >= 0.

## Key formulas

- H(t,x,p,M,u) = sup over a in A of { -0.5 Tr(sigma sigma^T M) - <mu,p> + r u - f }.
- xi in {-1,+1}^m, P = 2^m, and y(a,xi) = x + mu(t,x,a) Delta_t + sigma(t,x,a) sqrt(Delta_t) xi.
- T[V](x) = max over a of { (1/P) sum over xi of exp(-r_a Delta_t) [ exp(-kappa_a,xi D_a,xi) V(ybar_a,xi) + D_a,xi exp(-kappa_a,xi D_a,xi / 2) g_a,xi ] + Delta_t f_a }.
- V <= W implies T[V] <= T[W]; V >= 0, f >= 0, and g >= 0 imply T[V] >= 0.

## Implementation notes

1. Integration point: use the layer as the spatial propagation block in a neural operator, value-function network, or latent controlled-dynamics model. The input is a field V at time n+1 on grid or point-cloud locations x_i; the output is the propagated field at time n. Do not append an unconstrained signed spatial mixer if the certificate is important. 2. Pseudocode: for every x_i, control a_j, and Rademacher branch xi_q, compute y = x_i + mu_ij Delta_t + sigma_ij sqrt(Delta_t) xi_q. If y is outside the domain, compute an oblique projection along gamma, set D to twice the overshoot, and calculate reflected location ybar. Interpolate V at ybar. Accumulate z_ij = mean_q of exp(-r_ij Delta_t) times [exp(-kappa_ijq D)V(ybar) + D exp(-kappa_ijq D/2)g_ijq], then add Delta_t f_ij. Return max_j z_ij. Use temperature-controlled log-sum-exp over controls only during warmup if hard maxima cause optimization difficulty. 3. Predict mu, sigma, r, kappa, g, and f with neural heads; enforce r and kappa nonnegative with softplus and keep branch probabilities exactly 1/P. Use bilinear interpolation on a grid or a positive kernel interpolator on point clouds. Reflection geometry is computed deterministically; learned coefficients are empirical. 4. First experiment: train on a two-dimensional reflected controlled diffusion with an analytic or high-resolution reference on a 32 by 32 grid. Compare four-branch and sixteen-branch layers against an MLP neural operator and an unconstrained learned convolution with matched parameter count and FLOPs. Use one-step and twenty-step rollout losses with nonnegative terminal and boundary sources. Measure rollout error, minimum prediction, monotonicity violations after positive input perturbations, and gradient norms. Success is zero or near-zero positivity violations and at least two times lower long-horizon error at equal FLOPs than the unconstrained baseline.

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.