# Shared Symbolic Mechanism Bottleneck MVP

## Implementation

`experiment.py` implements the proposed differentiable symbolic bank: standardized raw inputs, safe reciprocal/log features, sigmoid feature gates, soft operator mixtures over seven operators, sigmoid readout gates, sparse readout regularization, feature-gate regularization, and temperature annealing. It compares this model with a small two-hidden-layer shared MLP on a three-output synthetic kinetics problem.

## Stage-1 quantitative mechanism checks

The checks were run with fixed seed 1450.

1. **Operator softmax temperature law.** For an operator-logit gap `Delta=2`, the predicted entropy is
   `H = log(1+exp(-Delta/tau)) + (Delta/tau)/(1+exp(Delta/tau))`.
   Observed versus predicted entropy:

   | tau | observed | predicted |
   |---:|---:|---:|
   | 1.00 | 0.3653338551 | 0.3653338551 |
   | 0.50 | 0.0900947678 | 0.0900947678 |
   | 0.25 | 0.0030182074 | 0.0030182074 |
   | 0.10 | 4.3284e-08 | 4.3284e-08 |
   | 0.05 | 1.6993e-16 | 1.7418e-16 |

   This confirms the expected rapid concentration of the differentiable operator choice as temperature decreases.

2. **Shared latent readout prediction.** If every output is a coefficient times the same latent `z`, least-squares shared-factor reconstruction has zero residual in exact arithmetic. For 1,000 synthetic samples and three outputs, the measured residual was `3.13e-32`.

3. **Noise scaling prediction.** For additive zero-mean noise, expected irreducible MSE is `sigma^2`. Measured versus predicted MSE:

   | sigma | measured | sigma^2 |
   |---:|---:|---:|
   | 0.00 | 0.000000 | 0.000000 |
   | 0.01 | 0.00010046 | 0.00010000 |
   | 0.03 | 0.00090141 | 0.00090000 |
   | 0.10 | 0.01002553 | 0.01000000 |

All three elementary mathematical predictions were numerically confirmed. The second and third are mechanism sanity checks rather than evidence that optimization discovers the factor.

## Mini-experiment

Training used 1,000 samples from `x in [0.15,5]`, 1% noise, 500 Adam epochs, and fixed initialization. Extrapolation used noiseless samples from `[5,10]`.

| model | in-range MSE | extrapolation MSE |
|---|---:|---:|
| symbolic bottleneck | 0.0800653 | 0.0449622 |
| shared MLP | 0.000284852 | 0.00317361 |

The symbolic model retained 5 active readout units after the simple activity threshold. It did not recover a single interpretable common denominator; its selected operator indices were `[0,2,5,6,0,1,1,0]`.

## Verdict

The math sanity checks passed and the symbolic model showed a useful extrapolation signal, but it did not achieve equal in-range error and did not recover the shared denominator. In particular, the stated output head is additive in shared units, while the kinetics target requires output-specific factors `x_j` multiplied by the common reciprocal denominator; this mismatch likely makes the target unnecessarily difficult for the proposed bank. Therefore this MVP does **not** establish the claimed overall win.

Run with:

```bash
/home/maxwelhelp/main/bin/python3 experiment.py
```

Results are also saved in `results.json`.