First-Hit Interacting Optimizer

1. Exact integration point: implement this as a population optimizer around a small neural network, such as a two-layer MLP on MNIST or a small CNN on CIFAR-10. Maintain N parameter particles for N in {4, 8, 16, 32, 64, 128}. Each particle receives its own minibatch gradient. Define a deterministic interaction uᵢdet = α Σⱼ≠ᵢ ψ(θⱼ − θᵢ), with ψ(v) = v/(||v|| + ε). Compare this unnormalized interaction with the normalized version α(N−1)⁻¹ Σⱼ≠ᵢ ψ(θⱼ − θᵢ). For stochastic forcing, draw antisymmetric pairwise Gaussian or Rademacher vectors ξᵢⱼ = −ξⱼᵢ and add uᵢnoise = σ Σⱼ≠ᵢ ξᵢⱼ. 2. Pseudocode: at each step compute every particle gradient gᵢ; compute pairwise displacements; form vᵢ = −gᵢ + uᵢdet + uᵢnoise; update θᵢ ← θᵢ + ηvᵢ + √(2Dη)zᵢ with zᵢ sampled from a standard Gaussian; record the first step at which any particle crosses ℓ*. 3. Computed versus estimated: gradients, interactions, and first-hit times are computed directly. Estimate scaling exponents by regression of log mean(T_N) against log N and test the logarithmic law by regression against log log N. Use matched minibatch compute and common random numbers. 4. First cheap experiment: train a two-layer MLP on MNIST with a fixed loss threshold and compare independent particles, normalized deterministic coupling, unnormalized deterministic coupling, and stochastic pairwise kicks. The falsifiable prediction is that bounded or normalized coupling follows approximately T_N proportional to 1/ln N, coherent unnormalized coupling approaches slope −1 on a log-log plot, and stochastic pairwise forcing approaches T_N proportional to 1/(N ln N). Also measure a practical instability boundary: increasing α or σ should eventually cause exploding particle norms, increasing loss variance, or failure to reach the threshold.