{
 "artifacts": [
  {
   "name": "bench_report.json",
   "url": "https://synthcore.org/code/1223/bench_report.json"
  },
  {
   "name": "experiment.py",
   "url": "https://synthcore.org/code/1223/experiment.py"
  },
  {
   "name": "report.md",
   "url": "https://synthcore.org/code/1223/report.md"
  },
  {
   "name": "report_bench_2026-09-03T124900.md",
   "url": "https://synthcore.org/code/1223/report_bench_2026-09-03T124900.md"
  },
  {
   "name": "results.json",
   "url": "https://synthcore.org/code/1223/results.json"
  },
  {
   "name": "stage2_bench.py",
   "url": "https://synthcore.org/code/1223/stage2_bench.py"
  }
 ],
 "category": "regularization",
 "description": "Train a low-dimensional projection of embeddings against centered pairwise geometry instead of only using raw-distance preservation or a JL-style guarantee. The loss removes the population or minibatch distance baseline before comparing distances, forcing the bottleneck to retain the fluctuations that carry ranking and task information.",
 "download_zip": "https://synthcore.org/download/1223",
 "formulas_latex": [
  "$$D=\\|x-y\\|^{2},\\qquad \\mathbb{E}[D]=2d\\sigma^{2},\\qquad D_m=\\frac{d}{m}\\|P(x-y)\\|^{2}.$$",
  "$$\\widetilde D_{ij}=D_{ij}-b,\\qquad \\widetilde D^{(m)}_{ij}=D^{(m)}_{ij}-b_m.$$",
  "$$\\operatorname{Var}\\!\\left(\\mathbb{E}[f(D)\\mid S]\\right)\\leq \\frac{m}{d}\\operatorname{Var}(f(D)).$$",
  "$$\\rho_K=\\frac{2}{\\pi}\\sqrt{\\frac{m}{d}}\\,(1+o(1)).$$"
 ],
 "id": 3043,
 "implementation": "Integrate this at an embedding projection bottleneck: an encoder produces h_i in R^d, then a learned matrix W produces z_i=Wh_i in R^m, with m much smaller than d. During training, sample a minibatch of B embeddings and construct K random or semi-hard pairs. Compute raw distances D_ij=||h_i-h_j||^2 and projected distances Dm_ij=||z_i-z_j||^2; rescale Dm by d/m if W is initialized as an approximately isotropic projection. Estimate baselines b and bm as exponential moving averages of the batch means, with stop-gradient on both baselines. Add L_center=(1/K) sum_ij (standardize(D_ij-b)-standardize(Dm_ij-bm))^2, or use a ranking variant based on pairwise signs. A compact update is: h=encoder(x); z=W h; D=pdist2(h); Dm=pdist2(z)*d/m; b=EMA(mean(D)); bm=EMA(mean(Dm)); loss=task_loss+lambda*MSE((D-b)/std(D),(Dm-bm)/std(Dm)); update encoder and W. The paper supplies the m/d variance ceiling and sqrt(m/d) ranking prediction; batch baselines, normalization, and lambda are empirical. First test on CIFAR-100 or a sentence-embedding dataset with d=768 and m in {64,128,256}. Compare a plain learned projection, a fixed JL random projection, and this loss on Recall@1, Kendall correlation of pairwise rankings, downstream accuracy, and embedding memory at equal m. Success is higher Recall@1 or ranking correlation at the same m, with less than 10% training overhead and no degradation in the original task loss.",
 "math_summary": "For two independent isotropic Gaussian points x,y distributed as N(0,\\sigma^{2}I_d), let D=\\|x-y\\|^{2} and let the projected representation be z=\\sqrt{d/m}\\,Px, where P has orthonormal rows, so the rescaled projected distance is D_m=(d/m)\\|P(x-y)\\|^{2}. The paper distinguishes the baseline E[D]=2d\\sigma^{2} from the centered fluctuation D-E[D]. The optimal squared-error decoder of any distance feature f(D) from a sketch S is E[f(D)\\mid S]. For a rank-m sketch, the squared singular values of this conditional-expectation operator imply that at most an m/d fraction of the variance of any feature of one squared distance is retained. The paper also gives expected Kendall ranking correlation \\rho_K=(2/\\pi)\\sqrt{m/d}(1+o(1)) when m,d\\to\\infty and m/d\\to0. We exploit these facts by explicitly matching centered distances: for pair (i,j), \\widetilde D_{ij}=D_{ij}-b and \\widetilde D^{(m)}_{ij}=D^{(m)}_{ij}-b_m, where b and b_m are stop-gradient batch means or running estimates.",
 "math_tags": [
  "probability",
  "statistics",
  "linear-algebra",
  "information-theory"
 ],
 "ml_areas": [
  "embedding",
  "loss",
  "regularization",
  "inference-speedup"
 ],
 "paper": {
  "arxiv_id": "2609.02155",
  "arxiv_url": "https://arxiv.org/abs/2609.02155",
  "summary_what_math_gives_to_ml": "The paper identifies a concrete failure mode of Johnson–Lindenstrauss guarantees: preserving large pairwise distances does not imply preserving the small centered fluctuations that determine rankings, nearest neighbors, or covariance shape. Its Gaussian calculations provide quantitative signal-retention laws: a rank-m sketch retains at most an m/d fraction of the variance of any feature of one squared distance, while ranking correlation scales only as sqrt(m/d). These formulas can become practical dimension-budget rules and training diagnostics for projection bottlenecks, compressed embeddings, and low-dimensional retrieval representations. The most useful transfer is a centered-distance-aware projection objective and a dimension selector based on ranking quality rather than the JL event alone.",
  "title": "Exact Limits of Random Projections for Preserving Geometry: Distance Recovery, Nearest-Neighbor Rankings, and Covariance Shape in Gaussian Models",
  "year": "2026"
 },
 "ratings": {
  "difficulty": 4,
  "novelty": 6,
  "usefulness": 7
 },
 "solves": [
  "accuracy",
  "generalization",
  "memory"
 ],
 "title": "Centered-Geometry Projection Loss",
 "url": "https://synthcore.org/idea/3043/centered-geometry-projection-loss",
 "verification": {
  "benchmark": {
   "track": "tabular",
   "model": "mlp",
   "n_seeds": 8,
   "metric_direction": "lower is better",
   "comparison": {
    "baseline_mean": 7.6550092101,
    "idea_mean": 7.6313580275,
    "delta_mean": -0.0236511827,
    "per_seed_diffs": [
     0.0098047256,
     -0.3486237526,
     0.1188426018,
     -0.0010251999,
     0.0296325684,
     -0.0009374619,
     0.0052537918,
     -0.0021567345
    ],
    "idea_wins": 4,
    "n_pairs": 8,
    "p_value": 0.9445,
    "mde": 0.1149067143,
    "mde_rel_pct": 1.50106565698,
    "verdict": "no measurable effect",
    "system_worked": false
   },
   "baseline": {
    "best_cfg": {
     "lr": 0.01
    },
    "sweep": [
     {
      "cfg": {
       "lr": 0.001
      },
      "mean": 17.0065875053
     },
     {
      "cfg": {
       "lr": 0.003
      },
      "mean": 11.7050991058
     },
     {
      "cfg": {
       "lr": 0.01
      },
      "mean": 7.4261586666
     }
    ],
    "full": {
     "mean": 7.6550092101,
     "std": 0.7549298959,
     "per_seed": [
      7.1253771782,
      6.5090332031,
      7.52886343,
      8.5413608551,
      7.8146495819,
      6.9143929482,
      8.8882131577,
      7.9181833267
     ],
     "n": 8
    }
   },
   "idea": {
    "mean": 7.6313580275,
    "std": 0.8269721814,
    "per_seed": [
     7.1351819038,
     6.1604094505,
     7.6477060318,
     8.5403356552,
     7.8442821503,
     6.9134554863,
     8.8934669495,
     7.9160265923
    ],
    "n": 8
   },
   "mechanism_signature": {
    "confirmed": false,
    "observed_centered_distance_correlation": 0.9999756813,
    "observed_centered_variance_ratio": 98.0545272827,
    "predicted_centered_distance_correlation": "no universal value for learned encoder",
    "predicted_variance_ceiling": 1
   },
   "custom_track": null
  },
  "mechanism_confirmed": false,
  "peer_reviewed": false,
  "practical_verdict": "no_effect",
  "stage1_mechanism_check": {
   "worked": true,
   "confidence": 8,
   "verdict": "Built a self-contained centered-geometry projection-loss experiment with Gaussian ranking verification and a matched learned bottleneck comparison. The Gaussian Kendall correlation followed the claimed (2/pi)sqrt(m/d) scaling, and centered training improved Kendall ranking from 0.613 to 0.679 and normalized centered distance MSE from 0.351 to 0.237. However, nearest-neighbor recall decreased from 0.238 to 0.179 and raw distance preservation became much worse, so the benefit is specifically for centered geometry/ranking rather than universal embedding quality.",
   "metrics": {
    "baseline": "Kendall tau 0.6129; centered normalized MSE 0.3514; raw normalized MSE 0.7889; Recall@1 0.2375",
    "gaussian_check": "Observed Kendall tau for m={2,4,8,16,32}: {0.097,0.148,0.223,0.320,0.495}; predictions: {0.113,0.159,0.225,0.318,0.450}",
    "idea": "Kendall tau 0.6795; centered normalized MSE 0.2373; raw normalized MSE 1041.7; Recall@1 0.1792"
   },
   "how_to_run": "python3 experiment.py",
   "files": [
    "experiment.py",
    "results.json"
   ],
   "limitations": "This is a small synthetic fixed-embedding experiment, not CIFAR-100 or a real sentence-embedding benchmark. It compares current-batch centering rather than a separately tuned EMA baseline, uses one bottleneck size, and does not measure wall-clock overhead or equal-FLOP training."
  },
  "status": "failed_benchmark",
  "status_label": "Failed on benchmark",
  "updated_at": "2026-09-03T12:49:00",
  "verdict_source": "deterministic test code (paired-seed permutation statistics)",
  "verification_axes": {
   "benchmark_mechanism": {
    "confirmed": false,
    "tested": true
   },
   "practical_benchmark": {
    "beats_baseline": false,
    "tested": true,
    "verdict": "no_effect"
   },
   "toy_mechanism_gate": {
    "confirmed": true,
    "tested": true
   }
  }
 }
}