How Phi-4 Cracked Small Multimodality

Paper 📜 https://arxiv.org/abs/2503.01743 Links + Notes 📝 https://www.oxen.ai/blog/how-phi-4-cracked-small-multimodality Join Arxiv Dives 🤿 https://oxen.ai/community Discord 🗿 https://discord.com/invite/s3tBEn7Ptg -- Use Oxen AI 🐂 https://oxen.ai/ Oxen AI makes versioning your datasets as easy as versioning your code! Even is millions of unstructured images, the tool quickly handles any type of data so you can build cutting-edge AI. -- Chapters 0:00 Intro 2:25 What You Could Use Phi-4 For 3:26 How Phi-4 Works Under the Hood 5:30 Model Architecture: Mixture of LoRAs 9:21 Q: Are the Weights for the Base Model Changed? 10:01 Testing Phi-4 13:56 Vision Training 14:41 The Four Stages of Training 17:56 Audio Training 19:26 Reasoning Training 21:56 Data and Training Details 24:59 The Visual Training Data