{
 "artifacts": [
  {
   "name": "REPORT.md",
   "url": "https://synthcore.org/code/1093/REPORT.md"
  },
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1093/bench_report.json"
  },
  {
   "name": "experiment.py",
   "url": "https://synthcore.org/code/1093/experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1093/report.md"
  },
  {
   "name": "report_bench_2026-09-02T005553.md",
   "url": "https://synthcore.org/code/1093/report_bench_2026-09-02T005553.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1093/results.json"
  },
  {
   "name": "stage2_bench.py",
   "url": "https://synthcore.org/code/1093/stage2_bench.py"
  }
 ],
 "category": "architecture",
 "description": "Prune parameter directions according to how much task-relevant Jacobian energy they carry, rather than by weight magnitude or individual gradient magnitude. Keep a mask whose discarded tangent component is at most an empirical fraction epsilon of the full tangent vector for calibration task directions, thereby preserving the local output dynamics seen by the task.",
 "download_zip": "https://synthcore.org/download/1093",
 "formulas_latex": [
  "$$\\|(I-P)T^{*}u\\|_{H}\\leq\\epsilon\\|T^{*}u\\|_{H}$$",
  "$$\\widehat r(P)^2=\\frac{\\sum_{i=1}^{m}\\|(I-P)T^{*}u_i\\|_2^2}{\\sum_{i=1}^{m}\\|T^{*}u_i\\|_2^2}\\leq\\epsilon^2$$",
  "$$G=TT^{*},\\qquad T_P=TP,\\qquad G_P=T_PT_P^{*}=TPT^{*}$$",
  "$$\\dot z(t)=-Gz(t)\\quad\\longrightarrow\\quad\\dot z_P(t)=-G_Pz_P(t)$$"
 ],
 "id": 2772,
 "implementation": "Integrate this into structured or unstructured weight pruning after a short calibration phase, using the parameter Jacobian of network outputs on a small representative dataset. (1) Exact integration point: choose a candidate binary parameter mask \\(P\\) for one layer or the whole model; take the Jacobian from output logits or regression outputs with respect to candidate weights, and make pruning decisions before the main fine-tuning run. Do not form the full Jacobian: use reverse-mode vector-Jacobian products to compute \\(g_i=T^*u_i\\) for each calibration direction. (2) Pseudocode: collect calibration examples \\((x_i,y_i)\\); compute outputs \\(f_\\theta(x_i)\\); set \\(u_i=f_\\theta(x_i)-y_i\\), or use several random output-space directions for classification; compute \\(g_i=T^*u_i\\); initialize a mask with magnitude pruning or all weights retained; for each candidate block or weight set \\(S\\), score removal by \\(q(S)=\\sum_i\\|g_i[S]\\|_2^2/\\sum_i\\|g_i\\|_2^2\\); greedily remove the lowest-score candidates while \\(q(S)\\leq\\epsilon^2\\), setting \\(P_{jj}=0\\) for removed coordinates and one otherwise; then fine-tune the masked network. (3) The mathematical quantity is the discarded-to-total tangent norm ratio; task directions, mask granularity, and greedy approximation are engineering choices. Estimate the ratio on held-out calibration batches and optionally refresh it after each fine-tuning epoch. For a dynamics diagnostic, estimate kernel-vector products and monitor \\(\\|(G-G_P)v\\|/\\|Gv\\|\\) for random output vectors \\(v\\). (4) First experiment: use a ResNet-20 on CIFAR-10 or a 6-layer Transformer on a small language-modeling corpus, comparing magnitude pruning, gradient pruning, and tangent-capture pruning at 50% and 80% sparsity. Test \\(\\epsilon\\in\\{0.05,0.1,0.2\\}\\). Measure immediate validation loss increase, fine-tuning loss descent at equal optimizer steps, final accuracy or perplexity, and wall-clock and memory reduction with a genuinely sparse kernel. Success means a smaller immediate loss shock and faster recovery than magnitude pruning at equal sparsity, while the held-out empirical ratio stays below the selected epsilon.",
 "math_summary": "The paper's relative task-capture condition is \\(\\|(I-P)T^{*}u\\|_{H}\\leq\\epsilon\\|T^{*}u\\|_{H}\\). Here \\(H\\) is the parameter Hilbert space with its Euclidean inner product, \\(T:H\\to Y\\) is the network parameter Jacobian mapping parameter perturbations to output perturbations, \\(T^{*}:Y\\to H\\) is its adjoint, \\(u\\in Y\\) is a task-space output direction, and \\(P:H\\to H\\) is an orthogonal pruning projection that retains the unpruned parameter coordinates. The left side is the norm of the parameter update direction discarded by pruning, while the right side is the total parameter tangent direction induced by \\(u\\). The condition is imposed for task-relevant \\(u\\), such as residuals, label directions, or a low-dimensional basis of the cyclic task space generated by the output-space operator \\(G=TT^{*}\\) and task subspace \\(M\\). For a finite calibration set with output-direction matrix \\(U=[u_1,\\ldots,u_m]\\), use the empirical squared ratio \\(\\widehat r(P)^2=\\frac{\\sum_i\\|(I-P)T^{*}u_i\\|_2^2}{\\sum_i\\|T^{*}u_i\\|_2^2}\\leq\\epsilon^2\\). The ML adaptation uses \\(T_P=TP\\) and compares the full local kernel \\(G=TT^{*}\\) with the pruned kernel \\(G_P=T_PT_P^{*}=TPT^{*}\\); low tangent loss is expected to preserve short-time linearized training dynamics.",
 "math_tags": [
  "functional-analysis",
  "linear-algebra",
  "dynamical-systems"
 ],
 "ml_areas": [
  "pruning",
  "training-dynamics",
  "fine-tuning"
 ],
 "paper": {
  "arxiv_id": "2608.27748",
  "arxiv_url": "https://arxiv.org/abs/2608.27748",
  "summary_what_math_gives_to_ml": "The paper provides a concrete criterion for judging whether parameter-direction pruning preserves task-relevant network behavior: the discarded part of the parameter Jacobian must capture only a small fraction of every task-relevant tangent vector. This is more informative than weight magnitude because it measures how parameter removal changes output-space dynamics, and it naturally connects pruning to fixed-kernel NTK evolution. A practical transfer is a calibration-time Jacobian-aware pruning method that selects a mask subject to an empirical tangent-capture constraint, then tests whether the pruned model preserves loss trajectories and fine-tuning stability at substantially lower parameter and inference cost. The available extraction does not expose the complete finite-time semigroup bound, so the implementable proposal below uses the explicit tangent inequality as a pruning certificate rather than claiming an unavailable numerical constant.",
  "title": "The role of parameter Jacobians in the stability of network outputs",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 5,
  "novelty": 6,
  "usefulness": 7
 },
 "solves": [
  "memory",
  "stability",
  "accuracy"
 ],
 "title": "Task-Tangent Capture Pruning",
 "url": "https://synthcore.org/idea/2772/task-tangent-capture-pruning",
 "verification": {
  "benchmark": {
   "track": "tabular",
   "model": "mlp_tiny",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 7.261052072048187,
    "idea_mean": 7.309336423873901,
    "delta_mean": 0.04828435182571411,
    "per_seed_diffs": [
     -0.33841562271118164,
     0.12665271759033203,
     0.06899356842041016,
     0.013986587524414062,
     0.08899927139282227,
     0.069580078125,
     0.06563091278076172,
     0.2908473014831543
    ],
    "idea_wins": 1,
    "n_pairs": 8,
    "p_value": 0.4521,
    "mde": 0.1477390247386159,
    "mde_rel_pct": 2.034677940230525,
    "verdict": "no measurable effect",
    "system_worked": false
   },
   "baseline": {
    "best_cfg": {
     "lr": 0.01
    },
    "sweep": [
     {
      "cfg": {
       "lr": 0.001
      },
      "mean": 95.01333999633789
     },
     {
      "cfg": {
       "lr": 0.003
      },
      "mean": 9.572970628738403
     },
     {
      "cfg": {
       "lr": 0.01
      },
      "mean": 7.331724643707275
     }
    ],
    "full": {
     "mean": 7.261052072048187,
     "std": 0.9775059897186311,
     "per_seed": [
      6.919011116027832,
      5.700474262237549,
      7.776523113250732,
      8.930890083312988,
      7.3590850830078125,
      6.166253566741943,
      8.154062271118164,
      7.372964382171631
     ],
     "n": 8
    }
   },
   "idea": {
    "mean": 7.309336423873901,
    "std": 0.9807488529523701,
    "per_seed": [
     6.58059549331665,
     5.827126979827881,
     7.845516681671143,
     8.944876670837402,
     7.448084354400635,
     6.235833644866943,
     8.219693183898926,
     7.372964382171631
    ],
    "n": 8
   },
   "mechanism_signature": {
    "calibration_examples": 96,
    "confirmed": true,
    "idea_config": {
     "lr": 0.01
    },
    "idea_sweep": [
     {
      "cfg": {
       "lr": 0.001
      },
      "result": {
       "mean": 31.27217388153076,
       "n": 8,
       "per_seed": [
        36.63642120361328,
        29.88167953491211,
        25.0460147857666,
        29.383060455322266,
        19.669511795043945,
        34.03684616088867,
        29.361820220947266,
        46.16203689575195
       ],
       "std": 7.421801352093722
      }
     },
     {
      "cfg": {
       "lr": 0.003
      },
      "result": {
       "mean": 8.934857428073883,
       "n": 8,
       "per_seed": [
        8.492694854736328,
        8.243812561035156,
        10.493083000183105,
        8.787611961364746,
        10.205215454101562,
        6.738030910491943,
        10.40708065032959,
        8.111330032348633
       ],
       "std": 1.2454062470166958
      }
     },
     {
      "cfg": {
       "lr": 0.01
      },
      "result": {
       "mean": 7.309336423873901,
       "n": 8,
       "per_seed": [
        6.58059549331665,
        5.827126979827881,
        7.845516681671143,
        8.944876670837402,
        7.448084354400635,
        6.235833644866943,
        8.219693183898926,
        7.372964382171631
       ],
       "std": 0.9807488529523701
      }
     }
    ],
    "mechanism_signature": {
     "confirmed": true,
     "observed_max_ratio": 0.015607444569468498,
     "observed_mean_ratio": 0.002711589972022921,
     "predicted_ratio_bound": 1,
     "prediction": "task-tangent mask discards a small fraction of calibration tangent energy"
    },
    "sparsity": 0.5,
    "track_choice": "tabular is structurally matched because this intervention is pruning/training-dynamics rather than convolution, attention, or control."
   },
   "custom_track": null
  },
  "mechanism_confirmed": true,
  "peer_reviewed": false,
  "practical_verdict": "no_effect",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 8,
   "verdict": "Built a reproducible Task-Tangent Capture Pruning MVP with epsilon-controlled tangent-energy masks and a small masked-regression comparison against magnitude pruning. The mechanism manifested: discarded tangent ratios stayed below every epsilon target, the squared-energy identity matched to 1.1e-16, and tangent ranking discarded less task energy at all tested sparsities. In the toy regression, tangent pruning achieved lower final loss after 30 steps at both 50% and 80% pruning, although its immediate loss shock was larger in this single run.",
   "metrics": {
    "baseline": "Magnitude-pruning task-energy ratios: 0.4866, 0.7363, 0.8798 at 25%, 50%, and 80% sparsity; final MSE 0.4182 and 0.4624 at 50% and 80%.",
    "idea": "Tangent-pruning task-energy ratios: 0.2303, 0.4128, 0.6882 at 25%, 50%, and 80% sparsity; final MSE 0.3942 and 0.4111 at 50% and 80%. Epsilon sweep ratios were 0.04953, 0.09938, 0.19810, 0.29999, and 0.49954 for targets 0.05, 0.10, 0.20, 0.30, and 0.50."
   },
   "how_to_run": "python3 experiment.py",
   "files": [
    "experiment.py",
    "results.json",
    "REPORT.md"
   ],
   "limitations": "This was a toy random-operator verification plus one tiny tanh regression and one seed. It did not test ResNet/CIFAR, gradient pruning, held-out calibration generalization, mask refresh, genuinely sparse kernels, wall-clock speed, or memory reduction."
  },
  "status": "mech_ok_no_baseline",
  "status_label": "Mechanism confirmed, baseline not beaten",
  "updated_at": "2026-09-02T12:30:10",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": true,
    "tested": true
   },
   "practical_benchmark": {
    "beats_baseline": false,
    "tested": true,
    "verdict": "no_effect"
   },
   "toy_mechanism_gate": {
    "confirmed": true,
    "tested": true
   }
  }
 }
}