Optimizer ideas

Optimizer update-rule variants and parameter-specific step sizes, benchmarked against a learning-rate-tuned baseline.

2 ideas found

Category: Optimizer Solves: Speedup Math: Stochastic processes Clear all

✗ Failed on benchmark 2026

Martingale Response Control Variate

Use the trajectory martingale decomposition to separate predictable training updates from genuinely unpredictable residual updates, then scale the residual according to its estimated response to future loss. The method targets stochastic or event-driven optimization with history-dependent samples and predicts that response-weighted residual energy, rather than total gradient variance, controls update noise and instability.

Useful7/10

Difficulty7/10

Novelty7/10

Optimizer Stability Speedup Accuracy

Paper: The Memory Hidden in Response Fluctuations: Trajectory-Level Fluctuation-Response Theory and Inequalities for Non-Markovian Jump Dynamics arXiv:2608.20328

✗ Failed on benchmark 2026

Dirac-Coupled Energy-Shaping Optimizer

Construct optimizer variables as interconnected Hamiltonian subsystems: parameters store potential energy, momentum stores kinetic energy, and a skew coupling transfers energy between them without net creation. Positive-semidefinite resistance removes energy and provides an explicit damping knob, separating conservative exploration from dissipative convergence.

Useful7/10

Difficulty6/10

Novelty6/10

Optimizer Stability Speedup Optimizer

Paper: Port-Hamiltonian modelling of coupled rigid/flexible multibody systems arXiv:2608.05143