Meet AdaMod: New Deep Learning Optimizer with Long Term Memory

Meet AdaMod, an improvement over the Adam optimizer by adding in a long term memory aspect. We'll review the three improvements that AdaMod provides (vs Adam) and take AdaMod for a quick spin on a live server so you can see AdaMod in action. AdaMod Paper: https://arxiv.org/abs/1910.12249v1 (Ding, Luo, Ren, Sun)