# Directional Hölder Step Controller

## Implementation

`directional_controller_experiment.py` implements the alpha=1 controller for a differentiable parameter vector. It probes the maintained rate, computes

- `r = F(theta - eta*g) - F(theta) + eta*||g||^2`
- `Lhat = 2*max(r, 0)/(eta*||g||)^2`
- `eta_star = 1/Lhat`
- `eta_candidate = clip(rho*eta_star)`

and applies sufficient-decrease backtracking. The experiment also compares it with cosine-decay SGD on a fixed full-batch nonlinear binary classification problem.

## Toy mechanism verification

For a quadratic `F(theta)=theta^T H theta/2`, alpha=1 predicts:

1. `Lhat` exactly equals directional curvature `g^T H g / ||g||^2`, independently of probe step.
2. With safety factor rho=0.9, accepted `eta*lambda` equals 0.9 for `H=lambda I`, hence `eta` scales as `1/lambda`.
3. The sufficient-decrease test accepts exactly when `eta*lambda <= 2(1-c)`; with `c=0.1`, the boundary is 1.8.

Observed results are in `results.json`:

- Curvature relative error across lambda scales 0.1 to 10: `4.1e-13` down to `3.4e-16`.
- Inverse scaling: observed `eta*lambda` is 0.9000000000 for every tested lambda (0.1, 0.3, 1, 3, 10).
- Boundary tests at `eta*lambda={1.7,1.9,2.1}` exactly match predicted accept/reject outcomes for lambda 1 and 3.

Thus the proposed mechanism manifests in the controlled setting.

## Mini-experiment

Fixed seed 1296, 512 examples, 2x16 tanh MLP, 100 full-batch updates on CUDA:

| metric | cosine SGD | directional controller |
|---|---:|---:|
| final loss | 0.6545 | 0.1407 |
| loss at step 10 | 0.6695 | 0.6487 |
| loss at step 50 | 0.6557 | 0.2783 |
| accuracy | 72.1% | 96.3% |
| accepted fraction | 100% | 100% |
| mean learning rate | 0.0758 | 1.6610 |
| wall time (s) | 0.325 | 0.312 |

This is a promising signal, but not a matched-FLOP benchmark: the controller evaluates additional trial/candidate losses, while GPU timing at this tiny scale is noisy. No AdamW, minibatch-noise, alpha<1, or larger vision benchmark was tested.

## Reproduce

```bash
/home/maxwelhelp/main/bin/python3 directional_controller_experiment.py
```