# Эксперимент: Recursive variation-norm regularization (#1303)

{
  "worked": false,
  "confidence": 9,
  "verdict": "Built a finite-width recursive variation-norm MLP with learnable normalized SiLU scales and smooth absolute values, plus a fixed-seed regression benchmark against AdamW/L2. The core implementation checks passed with zero numerical error, and scale changes altered the normalized SiLU shape. However, the variation penalty produced much lower V but substantially worse validation MSE: best variation MSE was 0.4726 versus 0.2584 for L2, indicating underfitting rather than a useful regularization win in this setup.",
  "metrics": {
    "baseline": "AdamW/L2, best validation MSE 0.2583797574 at weight_decay=0.01; train MSE 0.2745188177; parameter squared L2 358.7202; V 183.5451.",
    "idea": "Adam plus recursive variation penalty, best validation MSE 0.4725931883 at lambda=0.001; train MSE 0.4767749608; parameter squared L2 112.3655; V 6.3401. Formula max error=0; recursive-cost max error=0; normalized-SiLU shape difference between s=0.25 and s=4 was 0.7544."
  },
  "how_to_run": "/home/maxwelhelp/main/bin/python3 run_experiment.py",
  "files": [
    "run_experiment.py",
    "results.json"
  ],
  "limitations": "Only one fixed seed, one-dimensional sinusoidal regression task, width 32, depth 2, 700 steps, and SiLU were tested. MNIST/Fashion-MNIST, ReLU/GELU comparisons, depths 4/8, Fourier-frequency analysis, gradient variance, matched-validation-loss or matched-V comparisons, and broader lambda/seed sweeps were not tested."
}