Positive Boundary-Diffusion Neural Layer

Integrate the construction as a sequence-mixing or recurrent block between two linear projections in a Transformer or SSM layer. First map an input tensor \(X\in\mathbb R^{B\times T\times H}\) to nonnegative channel states \(Q=\operatorname{softplus}(XW_{in})\). Interpret the sequence axis, or a small learned grid of \(N\) latent nodes, as the spatial coordinate. For each diffusion substep, apply the tridiagonal operator: for interior nodes compute \(D_i=d(q_{i-1}-2q_i+q_{i+1})/h^2\); at the left endpoint add \(\kappa q_0/h\), and at the right endpoint add \(\gamma q_{N-1}/h\), with \(\kappa=-\operatorname{softplus}(a)\) and \(\gamma=-\operatorname{softplus}(b)\). Use either explicit Euler \(Q\leftarrow Q+\Delta tD(Q)\) with \(\Delta t\le h^2/(2d+\epsilon)\), or an implicit tridiagonal solve for unconditional numerical stability. Add a gated input injection \(Q\leftarrow Q+\Delta t\operatorname{softplus}(XW_{src})\), then project back with \(W_{out}\) and use a residual connection. Pseudocode: initialize \(Q\); repeat \(K\) times; compute the three-point stencil; add endpoint damping; update \(Q\); and apply softplus if strict positivity is required. The paper-derived quantities are the diffusion stencil, the positive-cone constraint, and the negative-gain stability rule. Estimate \(d\), \(\Delta t\), and the source/output projections by gradient descent, while parameterizing gains through softplus so their signs cannot change. Begin with a six-layer character-level language model or MNIST sequence classifier, comparing against a standard residual MLP and an unconstrained learned tridiagonal layer at matched parameters and FLOPs. Measure activation norms, gradient norms, loss after 1,000 recurrent steps, validation accuracy, and sensitivity to increased depth. Success means no exploding trajectories for depths where the baseline diverges, lower variance across random seeds, and equal-or-better loss at matched compute.