Understanding Deep Learning Optimizers: Momentum, AdaGrad, RMSProp & Adam
Gain intuition behind acceleration training techniques in neural networksIntroductionDeep learning made a gigantic step in the world of artificial intelligence. At the current moment, neural networks outperform other types of algorithms on non-tabular data: images, videos, audio, etc. Deep learning models usually have a strong complexity and come up with millions or even billions of trainable parameters. That is why it is essential in the modern era to use acceleration techniques to reduce training time.One of the most…