torch.nn.TransformerEncoderLayer - Part 3 - Transformer Layer Normalization

This video shows how the Transformer Encoder Layer Normalization works. This is the layer immediately after the Attention Layer and the Positional Encoding Layer. 0:00 Transformer Layer Normalization Equation 2:10 Expected Value 3:10 Variance torch version - 1.10.0