# Dual-Rail Ratio Arithmetic Layer

- ID: 196
- Canonical URL: https://synthcore.org/idea/196/dual-rail-ratio-arithmetic-layer
- API JSON: https://synthcore.org/api/idea/196.json
- API Markdown: https://synthcore.org/api/idea/196.md
- Verification status: mechanism_failed
- Source: [arXiv:2608.26305](https://arxiv.org/abs/2608.26305)
- Category: architecture
- Solves: stability, accuracy, scalability
- ML areas: mlp, training, initialization
- Math tags: algebra, linear-algebra, numerical-analysis
- Ratings: usefulness 6/10; difficulty 5/10; novelty 7/10

## Idea description

Replace scalar neural activations by pairs of nonnegative channels whose ratio represents the signed or unsigned activation. Implement multiplication and addition through pair algebra, and renormalize each pair because the representation is invariant under multiplying both rails by the same positive scalar. This creates an explicitly bounded, cancellation-aware arithmetic layer for deep multiplicative MLPs, rational networks, and neural fields.

## Mathematical statement

For a nonnegative value a in the extended interval from zero to infinity, the paper represents it by a computational pair A=(a_0,a_1) in the nonnegative quadrant, with encoded value val(A)=a_1/a_0; a_0=0 represents positive infinity. The representation has gauge freedom: val(cA)=val(A) for every c>0. Define the tensor product A tensor B=(a_0 b_0,a_1 b_1); then val(A tensor B)=val(A)val(B). Define the direct sum A direct-sum B=(a_0 b_0,a_1 b_0+a_0 b_1); then val(A direct-sum B)=val(A)+val(B). A signed scalar x is represented by two nonnegative ratio pairs X_p and X_n, with x=val(X_p)-val(X_n). Signed multiplication uses four nonnegative products and two additions: the positive output is X_p tensor Y_p direct-sum X_n tensor Y_n, and the negative output is X_p tensor Y_n direct-sum X_n tensor Y_p. For canonical inputs, at most one sign rail is nonzero, so the output remains canonical. In a neural layer, the transferable property is exact algebraic compositionality before floating-point error: every feedforward composition computes the arithmetic expression represented by the graph.

## Key formulas

- $$\operatorname{val}(A)=\frac{a_1}{a_0},\qquad A=(a_0,a_1)\in\mathbb{R}_{\ge 0}^2,\qquad \operatorname{val}(cA)=\operatorname{val}(A)\quad(c>0).$$
- $$A\otimes B=(a_0b_0,a_1b_1),\qquad A\oplus B=(a_0b_0,a_1b_0+a_0b_1).$$
- $$XY=\bigl(X_p\otimes Y_p\oplus X_n\otimes Y_n,\;X_p\otimes Y_n\oplus X_n\otimes Y_p\bigr),\qquad x=\operatorname{val}(X_p)-\operatorname{val}(X_n).$$
- $$P_{ij}=\sum_{k=1}^{p}M_{ik}N_{kj}.$$

## Implementation notes

Integrate this as an optional replacement for the arithmetic block in a small multiplicative MLP, neural field, or multiplicative residual network. Represent each scalar activation by one unsigned pair (a_0,a_1), or by two signed pairs X_p=(x_{p0},x_{p1}) and X_n=(x_{n0},x_{n1}). Initialize an ordinary scalar x as A=(1,max(x,0)) for unsigned inputs, or as X_p=(1,max(x,0)) and X_n=(1,max(-x,0)) for signed inputs. For a pair multiplication, compute tensor(A,B)=(A0*B0,A1*B1). For a pair addition, compute direct_sum(A,B)=(A0*B0,A1*B0+A0*B1). For signed multiplication, compute pp=tensor(Xp,Yp), nn=tensor(Xn,Yn), pn=tensor(Xp,Yn), and np=tensor(Xn,Yp); return positive=direct_sum(pp,nn) and negative=direct_sum(pn,np). After every operation, apply gauge normalization independently to each pair: s=max(abs(A0),abs(A1),epsilon), then replace A by (A0/s,A1/s). Decode only at the final output, or at a deliberately inserted nonlinearity, using A1/(A0+epsilon) for an unsigned pair and the difference of decoded positive and negative rails for signed values. Begin with networks containing only multiplication, addition, and linear readout so the arithmetic representation is isolated. The exact pair formulas are computed directly; denominator distributions, overflow rates, and rail imbalance are measured empirically. First test a 4- to 8-layer multiplicative MLP on degree-20 polynomial regression or bilinear matrix regression against a standard float multiplicative MLP and an unnormalized pair baseline at matched parameter count. Success means lower relative error at large depth, fewer NaNs or infs, and stable optimization at learning rates where the ordinary multiplicative baseline diverges. Report the unavoidable channel and FLOP overhead.

## Verification

- Status: mechanism_failed
- Mechanism evidence: yes
- Mechanism confirmed: no
- Verdict: Built the dual-rail tensor, direct-sum, signed multiplication, gauge normalization, exact-algebra checks, and matched deep-product benchmark. The algebraic identities were verified to roughly 1e-16 relative error, and normalized rails stayed finite at depths where scalar float32 products overflowed; however, the ratio denominator underflowed, producing infinite decoded values, and normalization did not reliably improve relative error. Thus the bounded-rail signal exists, but the promised usable numerical stability was not demonstrated.

### Mechanism check

- Verdict: Built the dual-rail tensor, direct-sum, signed multiplication, gauge normalization, exact-algebra checks, and matched deep-product benchmark. The algebraic identities were verified to roughly 1e-16 relative error, and normalized rails stayed finite at depths where scalar float32 products overflowed; however, the ratio denominator underflowed, producing infinite decoded values, and normalization did not reliably improve relative error. Thus the bounded-rail signal exists, but the promised usable numerical stability was not demonstrated.
- Confidence: 9/10
- Limitations: No trained multiplicative MLP or optimization experiment was run, and no FLOP- or parameter-matched neural comparison was performed. The test used sequential positive products only; signed cancellation, matrix operations, activation nonlinearities, and gradient stability remain untested. The implementation renormalizes after multiplication, so intermediate products can still overflow in general, and ratio decoding needs a log-domain or guarded representation to avoid denominator underflow.

## Artifacts

- [dual_rail_experiment.py](https://synthcore.org/code/1200/dual_rail_experiment.py)
- [mini_benchmark.py](https://synthcore.org/code/1200/mini_benchmark.py)
- [report.md](https://synthcore.org/code/1200/report.md)
- [Download all files as ZIP](https://synthcore.org/download/1200)

## Disclaimer

AI-generated research hypothesis, automatically tested. Not peer-reviewed.