Accelerating Deep Q-learning with the Mean-expansion Layer

The AI Seminar is a weekly meeting at the University of Alberta where researchers interested in artificial intelligence (AI) can share their research. Presenters include both local speakers from the University of Alberta and visitors from other institutions. Topics can be related in any way to artificial intelligence, from foundational theoretical work to innovative applications of AI techniques to new fields and problems. In this seminar from the Alberta Machine Intelligence Institute and the Department of Computing Science, Prabhat Nagarajan, PhD Candidate at the University of Alberta, presents the mean-expansion layer for accelerated deep Q-learning. The mean-expansion layer is a simple way to represent a vector of action-values with a lower norm vector by sharing credit across actions within a state. It can be implemented as a simple parameter-free layer to the output of a standard discrete action Q-network. Empirically, the mean expansion layer accelerates learning and improves performance in DQN and IQN in aggregate across 57 Atari games. Bio: Prabhat Nagarajan is a PhD student at the University of Alberta working with Marlos C. Machado. In the past he received undergraduate and master's degrees from the University of Texas at Austin in Computer Science and has completed internships at Sony AI and Microsoft Research. His research is at the intersection of deep learning and reinforcement learning (RL). Specifically his recent research has aimed to understand and develop algorithms in value-based deep RL.