SynthCore

Weighted Resolvent-Equivariant Attention / REPORT.md

Unverified

Raw ⬇ ZIP

Weighted Resolvent-Equivariant Attention MVP

Implementation

experiment.py implements a weighted reversal operator G = Delta^2 R, numerical commutator/power/resolvent checks, and a small single-head attention classifier using the normalized penalty ||GP-PG||_F^2/(||G||_F^2||P||_F^2+eps). It uses CUDA when available and falls back to CPU on runtime failure.

Stage-1 math verification

All checks use fixed seed 2327.

  1. Linear perturbation scaling. For P(eps)=P0+eps E with [G,P0]=0, the prediction is ||[G,P(eps)]||_F = eps||[G,E]||_F. The predicted slope was 1.9490096903 and the fitted slope was 1.9490096903. Maximum relative deviation over eps from 0 to 0.3 was 1.9e-16. Observed norms at eps [0.001, 0.01, 0.1, 0.3] were [0.00194901, 0.01949010, 0.19490097, 0.58470291].

  2. Multi-step compatibility. For commuting P0, every tested power and its resolvent should commute with G. The maximum norm of [G,P0^t] for t=0..8 was 6.34e-16, and the resolvent commutator at gamma=0.37 was 5.04e-16: numerical zero.

  3. Damping behavior. The identity [G,R(gamma)] = (1-gamma)R(gamma)[G,P]R(gamma) predicts zero at gamma=1 and linear behavior in (1-gamma) near gamma=1. The identity relative error stayed below 1.2e-13 across the sweep, and the gamma=1 norm was exactly 0 numerically. The near-gamma=1 fitted slope versus (1-gamma) was 0.2129. The full resolvent norm was non-monotonic at gamma=0 to 0.1 because the selected non-contractive toy matrix is close to a resolvent singularity; it then decreased strongly toward zero. This is a limitation of that sweep, not a failure of the algebraic identity.

Attention pilot

Two fixed seeds, 180 optimization steps, identical model and data for lambda=0 and lambda=1:

  • Baseline lambda=0: mean accuracy 0.6994, mean validation loss 0.6377, mean commutator RMS 3.3943.
  • Weighted-commutator lambda=1: mean accuracy 0.7351, mean validation loss 0.5181, mean commutator RMS 0.3932.

Thus the penalty reduced the measured commutator by about 8.6x and improved mean accuracy by 3.6 percentage points / validation loss by 0.1196 in this tiny pilot. The requested 10x equivariance target was not reached, so the empirical effect is promising but not a full success against that threshold.

Reproduction

/home/maxwelhelp/main/bin/python3 experiment.py

Outputs are saved to results.json.

Limitations

This is not a 4-layer, 8-head transformer or a second graph benchmark. Only two seeds and one synthetic attention task were tested; FLOP matching, learned kappa, reversal augmentation, sample-complexity curves, and explicit function-level equivariance were not evaluated. The toy resolvent matrix was not constrained to be a well-conditioned Markov transition matrix, causing a non-monotonic low-gamma curve, although the exact resolvent identity and gamma-to-one prediction were verified.