Deep Learning (AI) Optimizer - Meet DiffGrad

Introduction and example use of DiffGrad, a new deep learning optimizer recently introduced by Dubey, et al. I explain what it does, why it can outperform Adam in some cases, and some example code (FastAI v1) on how to use. I've also added version 1 and an integrated version combining Rectified Adam and DiffGrad. Code is all at my github repo, lessw2020.