Self & Cross Attention | Generative AI | Transformers | Solved Example

In this video, you will understand the various topics related to Self and Cross Attention. These concepts are discussed in context of working of Transformers which comprises the underlying architecture of various Generative Algorithms. I have taken various numerical examples to explain the concepts in depth. If you are facing any issues do let me know in the comment section below, I am here to help ❤️ If you found this video useful then please consider subscribing to my channel 🙏 Chapters in the video: 0:00 Introduction 0:31 Problem Statement 01:15 Translation of sequences 01:46 Tranformer Architecture 05:12 Word 2 Vector Conversion 06:36 Start and End Sequences 07:35 Positional Encoding 08:05 Query, Key & Value Matrices 09:16 Compute Self Attention 10:15 Inside Self Attention 12:05 Usage of Softmax Function 13:40 Weighted Sum with Values 14:40 Cross Attention 15:17 Conclusion Background Music Credits (in order of use) Outro Music Credit: Spirit by Sappheiros: "Spirit by Sappheiros" is under a Creative Commons ( cc-by ) license Music promoted by BreakingCopyright: https://bit.ly/sappheiros-spirit