An important topic in deep learning is that of transfer learning. It not only has practical and theoretical implications, but also presents new research avenues. On the practical side it concerns how the representation learned by a network trained on a large data set, say of cats and dogs, can be used to recognize ants and wasps, whose data sets may be much smaller. On the theoretical side transfer learning is a manifestation of the generalization principle of machine learning. As for research avenues, it points to the need for learning more fine-grained distributions, such as the cause-effect relationship between variables, particularly when transfer learning has to be fast.
