# Tikhonov-Minimum-Norm Hypergradients MVP

## What was built
`tikhonov_experiment.py` implements conjugate gradients for `(H + eps I)v = b` and a controlled singular quadratic bilevel problem. It compares the Tikhonov solve with the exact Moore--Penrose solution and runs a small outer optimization with fixed damping versus a decreasing damping continuation schedule.

Run:

```bash
/home/maxwelhelp/main/bin/python3 tikhonov_experiment.py
```

The run writes `results.json`.

## Math checks and quantitative predictions

The Hessian has eigenvalues `[0, 0.1, 1, 10]`. For the stable-range test, `B` has zero component in the nullspace, so the prediction is
`(H + eps I)^(-1) B -> H^+ B`, with error tending to zero. The relative error over `eps=1e-1,...,1e-8` was:

`0.4988, 0.09069, 0.009877, 0.0009966, 9.975e-5, 9.976e-6, 9.976e-7, 9.976e-8`.

This is consistent with linear asymptotic error in epsilon.

For the eigenvalue `lambda=0.1` component, the exact prediction is relative bias `eps/(0.1+eps)`. At `eps=1e-8`, observed bias was `9.9999988e-8` versus predicted `9.9999990e-8`.

For a deliberate violation of the range condition, `B` was given nullspace component `0.8`. The nullspace component of the regularized derivative is exactly `0.8/eps`; the measured log-log slope of solution norm versus epsilon was `-0.99999993` (predicted `-1`). The quantity `eps * null_component` stayed approximately `0.8` throughout the sweep.

Thus all three mechanism predictions manifested numerically.

## Mini-experiment

The inner objective is a singular quadratic and the outer objective is squared distance to a target. The exact minimum-norm hypergradient at the initial point was `203.2592`. A fixed `eps=0.1` outer run ended with loss `0.11922`; the continuation run (epsilon decaying from `0.1` to a floor of `1e-7`) ended with loss `0.12423`. Therefore continuation did **not** beat fixed damping in this tiny, well-conditioned-on-the-range problem.

## Verdict

The proposed mathematical mechanism is real: Tikhonov derivatives converge to the pseudoinverse derivative under the stable-range condition, with the predicted first-order damping bias, while a nullspace mixed derivative produces the predicted `1/eps` instability. The MVP does not establish a practical optimization win; in the small outer test, fixed damping was slightly better.

## Limitations

No MNIST/MLP experiment, nonsmooth generalized Hessian experiment, noisy hypergradient variance study, wall-clock benchmark, or large-width scaling study was performed. The quadratic test uses exact matrix products for the reported outer comparison, although the Tikhonov linear systems are also solved by the included CG implementation in the core sweep.