# Singular-Value-Robust Projector-Splitting LoRA MVP

Run with:

```bash
/home/maxwelhelp/main/bin/python3 projector_splitting_mvp.py
```

The complete machine-readable output is in `results.json`.

## Implementation

`projector_splitting_mvp.py` implements the paper's three-substep QR projector-splitting retraction and the common-base explicit midpoint method. It also includes a small matrix-fitting comparison against independently updated low-rank factors. The comparison baseline is plain factor gradient descent (not Adam), so it is only a directional toy comparison.

## Stage-1 quantitative predictions

The toy verification uses an exactly rank-2 rotating matrix curve with a known full matrix derivative. Three predictions were tested:

1. **QR orthogonality/rank preservation.** Thin QR should keep U and V orthonormal and maintain a rank-r representation without inverse-S operations. For sigma-min values `1e-2`, `1e-6`, and `1e-10`, U orthogonality errors were `8.9e-16`, `2.9e-16`, and `4.7e-16`; V errors were `8.1e-16`, `6.7e-16`, and `5.1e-16`.

2. **Second-order convergence.** The common-base explicit midpoint construction predicts global error proportional to `h^2`. With h values `0.2, 0.1, 0.05, 0.025`, observed orders from the final two refinements were `1.99897`, `1.99897`, and `1.99897` for sigma-min `1e-2`, `1e-6`, and `1e-10`, respectively. This agrees with the predicted order 2.

3. **Robustness as a singular value collapses.** The error constant should remain essentially independent of the smallest retained singular value. At h=`0.025`, errors were `1.00198e-4`, `1.00193e-4`, and `1.00193e-4` for sigma-min `1e-2`, `1e-6`, and `1e-10`; the maximum/minimum ratio was about `1.00005`. Factor norms also remained finite and similar (`5.756`, `5.744`, `5.696`).

The `math_checks` reconstruction residuals around `0.667` are expected: a rank-r projector-splitting step approximates the generally full-rank Euler matrix `Y+hZ`; it is not supposed to reproduce that full-rank increment exactly. The meaningful structural checks are orthogonality, finite factors, and convergence to the rank-r exact curve.

## Toy optimization comparison

On a 20x16 matrix-fitting task with rank 4 and initial core singular values ending at `1e-6`, after 120 steps at step size `0.08`:

- Factor gradient-descent baseline final loss: `0.226712`
- Projector-splitting midpoint final loss: `0.106228`
- NaNs: neither method
- CPU time: baseline `0.0172 s`, idea `0.1048 s`

Thus the idea showed a promising lower-loss signal in this tiny setup, but QR and the two-stage midpoint evaluation were approximately 6.1x slower in this unoptimized NumPy implementation. This is not evidence of a general training win over production Adam or GPU kernels.

## Verdict

The requested mechanism manifested clearly: second-order convergence and stable error constants persisted while sigma-min decreased from `1e-2` to `1e-10`, with no inverse singular-value operation. The toy optimization result favors projector splitting at matched steps, but the baseline and problem are too small to establish a practical LoRA advantage; the strongest supported claim is singular-value robustness.