We discuss a new swarm-based gradient descent (SBGD) method for non-convex optimization. The swarm consists of agents, each is identified with position $x$ and mass $m$. There are three key aspects to the SBGD dynamics: (i) persistent transition of mass from agents at high to lower ground; (ii) a random marching direction, aligned with the steepest gradient descent; and (iii) a time stepping protocol which decreases with $m$.

The interplay between positions and masses leads to dynamic distinction between `heavier leaders’ near local minima, and `lighter explorers’ which explore for improved position with large(r) time steps. Convergence analysis and numerical simulations demonstrate the effectiveness of SBGD method as a global optimizer.