L09.4: Interpreting what Convnets Learn

In this video I turn to some techniques to analyze convolutional network layers. A problem with many machine learning methods in general, and most neural network models in particular, is their interpretability. They are often described as black boxes, it is not easy to interpret what a tensor matrix in a layer is doing to transform its inputs to make predictions with. But convolutional layers for vision processing tasks are fundamentally different and the opposite of black boxes. Because the filters learned by convolutional layers are fundamentally visual patterns extracted from learning data, we can extract and visualize convolutional layers to see what they are doing. In this video I briefly discuss 3 different convnet layer visualization techniques. Ifirst look at visualizing the output activation of a convolution filter given some sample input image. Then I look at a slightly different technique that uses optimization to build an image that will cause a filter to be maximally activated. In the third example I look at the CAM (class activation maps) technique that can determine the importance of areas of an image in making a classification decision. Resources: Textbook: Chollet (2022). "Deep Learning with Python (2ed)". Manning. https://www.amazon.com/dp/1617296864/?bestFormat=true&k=deep%20learning%20with%20python&ref_=nb_sb_ss_w_scx-ent-pd-bk-d_de_k0_1_15 CSci 560 Class Repository: https://github.com/csci560-nndl/nndl Contains video slides and iPython notebooks for this course. 00:00 Introduction 01:13 Interpretability of machine learning methods 03:50 Visualizing intermediate activations of convolutional filters 11:50 Visualizing convnet filters by creating an image that maximizes output activation 20:44 Visualizing heatmaps of class activation maps (CAM) 27:12 Summary