Curious About What's Inside an LLM? A Python Walkthrough for Beginners

What’s actually hiding inside an LLM? Using Python, we're diving deep into an OpenWeights LLM to reveal its inner workings. This is your chance to go beyond the hype and see the raw data—the weights and biases—that define how these models think. I'll guide you step-by-step through a practical, hands-on demonstration. By the end, you'll have a new appreciation for these LLMs Timestamps & Chapters 00:00 – Intro: My Curiosity About LLMs 00:32 - Open Weights LLM Models 00:44 - Not meant for Experts 01:11 – BERT in Everyday Use 01:54 - DistilBERT 02:22 – Loading DistilBERT in Colab 04:00 - Live Colab Demo 05:56 – What Do I Even Look For? 06:54 - Do the number of parameters equal 66M? 08:50 – Peeking at Layers & Transformer Architecture 09:48 – Looking at the Numbers (Weights & Biases) 11:04 - Matplotlib Histogram of an LLM Layer values 11:43 - Are the weights mostly infinitesimal? 12:36 - How different are 2 of the Transformer layer values? 13:28 - So what did we learn? 15:07 – Outro: A Brief Peek, More to Come Links: DistilBERT on HuggingFace: https://huggingface.co/distilbert/distilbert-base-uncased-finetuned-sst-2-english Arxiv: https://arxiv.org/abs/1910.01108 Medium Article by Victor Sanh - https://medium.com/huggingface/distilbert-8cf3380435b5 #BERTLLM #HowdoLLMsWork #smallLLMs