L05.4: Improving Model Generalization

Once your neural network model is training and shows evidence that it can generalize. And once you have it overfitting, so that you know that you have enough representational power to model the latent manifold of the data you are training with, it is time to turn your attention to maximizing the generalization of your model, the able to make correct and accurate predictions on data that it has not seen before. In this video I discuss several topics about improving the generalization power of your model. Some of the best ways to improve your model generalization is to improve your dataset, by getting more data, minimizing labeling errors, and cleaning the data. You can also sometimes perform feature engineering to create new features from the existing ones you have that will make it easier for a machine learning algorithm to fit a good model. If you have a dataset and have cleaned it, and still need to improve your models generalization, you can regularize your model. In this video I discuss - Using early stopping - Reducing the networks size to reduce representational power of the model - Using L1/L2 weight regularization in your model's layers - Using dropout regularization between your model's layers Resources: Textbook: Chollet (2022). "Deep Learning with Python (2ed)". Manning. https://www.amazon.com/dp/1617296864/?bestFormat=true&k=deep%20learning%20with%20python&ref_=nb_sb_ss_w_scx-ent-pd-bk-d_de_k0_1_15 CSci 560 Class Repository: https://github.com/csci560-nndl/nndl Contains video slides and iPython notebooks for this course. 00:00 Introduction 01:03 Dataset curation: get more and better data 03:24 Feature engineering: create new and better features 04:40 Early stopping 05:38 Reducing network representational power 10:15 Adding weight regularization 14:10 Adding dropout regularization 19:12 Summary