Why do neural nets learn and generalize?

Slides: https://bayesgroup.github.io/bmml_sem/2019/Golikov_Why%20Do%20Neural%20Nets%20Learn%20and%20Generalize.pdf Eugene Golikov, Neural Systems and Deep Learning lab., MIPT As was noted in [Belkin et al., 2019], neural nets are usually used in the so-called "interpolating regime". In this regime our architecture is large enough to have an ability to fit the training data perfectly, as opposed to "classical regime", where our model is constrained to balance between learning and generalization. Two questions arise immediately: 1) Why does (stochastic) gradient descent - a local optimization method - find a configuration that fits the data perfectly? 2) Why does (stochastic) gradient descent choose a configuration that generalize well, across all configurations that fit the training data? Although the first question is close to being fully answered, the second one remains mostly opened. In our talk we will review some of the recent results concerning both of them.