34: Overfit and underfit | TensorFlow | Tutorial

The video discusses in TensorFlow: Using HIGGS dataset, create models to overfit data followed by strategies to reduce overfitting. 00:00:00 - Overview 00:01:25 - Import libraries 00:04:40 - Set path for TensorBoard logs 00:05:48 - Download data: tf.keras.utils.get_file(): HIGGS.csv.gz 00:07:31 - Read CSV data directly from GZIP: tf.data.experimental.CsvDataset() 00:09:48 - Create a function to get data into (feature vector, label) format: tf.stack() 00:11:42 - Data: .batch().map().unbatch() 00:12:15 - Data: look at few records: plt.hist() 00:15:03 - Declare variables 00:16:30 - Data: create train and validation datasets: .take(), .cache(), .skip() 00:19:00 - Learning rate schedule: tf.keras.optimizers.schedules.InverseTimeDecay() 00:23:48 - Callbacks: tfdocs.modeling.EpochDots(), tf.keras.callbacks.EarlyStopping(), tf.keras.callbacks.TensorBoard() 00:25:38 - Function: compile and fit: .compile(), .fit() 00:30:30 - Tiny model: 1 hidden layer, 16 units: tf.keras.Sequential() 00:33:21 - Tiny model: plot loss 00:36:19 - Small model: 2 hidden layers, 16 units 00:37:24 - Medium model: 3 hidden layers, 64 units 00:38:20 - Large model: 4 hiddent layers, 512 units 00:40:40 - Plot the training and validation loss for all models 00:43:20 - TensorBoard: view accuracy and loss 00:47:27 - Strategies to prevent overfitting: get baseline from Tiny model 00:49:20 - Create dictionary to save history 00:50:05 - L2: Add L2 regularization to the Large model: layers.Dense(512, activation='elu', kernel_regularizer=regularizers.l2(0.001)) 00:53:15 - L2: plot losses 00:55:40 - Dropout: Add drop out layers to Large model: layers.Dropout(0.5) 00:58:52 - Dropout: plot losses 01:00:41 - Combined: L2 + Dropout 01:03:08 - Combined: plot losses 01:09:54 - Ending notes # ---------------- # TensorFlow Guide # ---------------- https://www.tensorflow.org/tutorials/keras/overfit_and_underfit