Task-Tangent Capture Pruning MVP

Mathematical predictions and checks

The experiment uses a random tangent operator T and calibration output directions U, with columns g_i = T^*u_i.

1. Relative-capture bound: removing the lowest task-energy coordinates until the cumulative discarded energy reaches epsilon^2 should produce r <= epsilon, where r = sqrt(sum_i ||(I-P)g_i||^2 / sum_i ||g_i||^2). Observed (epsilon, r) were (0.05, 0.04953), (0.10, 0.09938), (0.20, 0.19810), (0.30, 0.29999), and (0.50, 0.49954). All satisfy the bound.

2. Squared-energy scaling: the empirical squared ratio should equal the discarded tangent-energy fraction. Across random masks, the maximum absolute error in this identity was 1.11e-16. For the epsilon sweep, observed squared ratios were 0.002454, 0.009876, 0.039245, 0.089997, 0.249538, matching the squares of the observed ratios exactly.

3. Task-aware ranking: at equal sparsity, ranking coordinates by task tangent energy should discard less task energy than unrelated magnitude ranking. Ratios (tangent vs magnitude) were: 25%: 0.2303 vs 0.4866; 50%: 0.4128 vs 0.7363; 80%: 0.6882 vs 0.8798.

Mini experiment

A fixed-seed small tanh regression model compares tangent-energy pruning and magnitude pruning, followed by 30 masked SGD steps. At 50% pruning, initial MSE was 0.4957 tangent vs 0.4680 magnitude, and final MSE was 0.3942 vs 0.4182. At 80% pruning, initial MSE was 0.5123 vs 0.4624, and final MSE was 0.4111 vs 0.4624. Thus tangent pruning had a larger immediate shock in this single run but better equal-step recovery/final loss.

Reproduction

/home/maxwelhelp/main/bin/python3 experiment.py

This writes results.json and prints the same measurements. CUDA is used when available, with CPU fallback.

Limitations

This is a toy linear-algebra verification and a tiny regression, not CIFAR/ResNet-scale evidence. It does not measure genuinely sparse-kernel speed or memory, does not compare gradient pruning, and uses one seed and one calibration batch; held-out task-direction generalization and refresh-after-finetuning were not tested.