IQmates

2.13K subscribers

279 videos

Knowledge Shared... Mates from all over the world coming together, sharing what gives people true power and freedom - knowledge. From the arts to the science...

Latest Videos

Course Introduction

More on the GPT Architecture

Datasets

Introduction to Transformer Architecture Part 2

Introduction to Transformer Architecture Part 1

Stages of building an LLM

Introduction to LLMs

Course Structure

Working with Text Data

Word Embeddings

Tokenising Splitting Texts

TokenIDs from a Vocabulary

Adding Special Tokens

Byte Pair Encoding

Data Sampling with Sliding Window

Creating token embeddings

Adding positional embeddings

Summary of data preparation

Introduction

Challenges of Long Sequences

RNNs and Bahdanau Attention Mechanism

Simple Self Attention Mechanism

Simple Self Attention for all Inputs

Self Attention with Trainable Weights

Scaled Dot Product Self Attention on all Inputs

Causal or Masked Attention

Adding dropout to causal attention

Causal Attention for all inputs

Multi head Attention

Multi head attention with weight splitting