Speaker: Prof. Stephen Gould, ANU.
Tutorial website: http://eccv2020.deepdeclarativenetworks.com
TL;DR: Introduces key concepts for the understanding of deep declarative networks and differentiable optimization layers.
Abstract
This short introduction to deep declarative networks and differentiable optimization will cover three key ideas that will lay the foundation for the remainder of the tutorial. The first is the idea of deep learning models are data flow graphs with signals propagating in the forward direction and gradients propagating in the backward direction. The graphs define a function from input to output that can be thought of as a composition of functions. As such, the chain rule of differentiation allows for straightforward computation of the gradient of the error with respect to the model’s parameters. The second key idea, building on this, is that the implementation of the forward and backward functions can be decoupled allowing for efficient algorithms to be used when appropriate rather than always relying on automatic differentiation. The last key idea is that of implicit differentiation, which enables calculation of the gradient of the solution to an optimization problem with respect to its inputs. Putting these ideas together we arrive at deep declarative networks or differentiable optimization layers. The remainder of the tutorial dives deeper into technical aspects of these ideas and discusses applications in computer vision.
