L09.1-2: Computer Vision Tasks and an Image Segmentation Example

In this video I continue our discussion of convolutional neural networks by looking at other types of image processing other that image classification that can be performed. There are three essential types of computer vision tasks that you should be familiar with: 1. Image Classification 2. Image Segmentation 3. Object Detection My first example convnet was with a binary image classification task (dogs vs. cats). In this video I look at a typical image segmentation task. There are two flavors of image segmentation, semantic segmentation and instance segmentation. Instance segmentation involves not only classifying image pixels but also detecting individual objects, so it has some overlap with object detection. In this example we look at the simpler semantic segmentation task. For segmentation, the task is to learn a segmentation map. This can really be thought of as a class label for every individual pixel of an image. So in a segmentation task, instead of identifying the class of the whole picture, you want to label the class of every pixel. For example in this video I segment each pixel as either background, foreground (a pet) or contour. The architecture needed for a segmentation tasks usually involves a series of 2D convolutions to compress the input representation into a suitable high level space. But then the convolutions are essentially reduced to end up with an output the same size as the inputs, since we need a class prediction for every pixel of the input image. Resources: Textbook: Chollet (2022). "Deep Learning with Python (2ed)". Manning. https://www.amazon.com/dp/1617296864/?bestFormat=true&k=deep%20learning%20with%20python&ref_=nb_sb_ss_w_scx-ent-pd-bk-d_de_k0_1_15 CSci 560 Class Repository: https://github.com/csci560-nndl/nndl Contains video slides and iPython notebooks for this course. 00:00 Introduction 00:22 The three main computer vision tasks 03:48 Semantic vs. instance image segmentation 05:10 Image segmentation example dataset 11:40 Image segmentation convolutional model architecture 16:12 Image segmentation model results 19:30 Summary