KO: Kinetics-inspired Neural Optimizer with PDE Simulation Approaches

Excerpt

arXiv:2505.14777v2 Announce Type: replace Abstract: The design of effective optimization algorithms for neural networks remains a fundamental challenge, and most existing methods rely on heuristic extensions of gradient-based updates. We introduce KO (Kinetics-inspired Optimizer), a plug-and-play optimization module grounded in kinetic theory and partial differential equations. KO models parameter dynamics as a particle system, augmenting standard gradient updates with stochastic interactions in