LUMOS: Transformers for User Behavior Prediction | Paper Walkthrough

A walkthrough of LUMOS (Large User MOdel Series) — a single transformer that replaces task-specific user-behavior models. Trained on 1.7T user- activity tokens from 250M users; +3.15% DAU in production A/B test. 📄 Paper: https://arxiv.org/abs/2512.08957 ✍️ Authors: Dhruv Nigam, Naman Agarwal, Krishna Murthy, Susmit Saha In this video I break down our paper "LUMOS: Large User MOdels for User Behavior Prediction." We treat user-activity streams the way LLMs treat language, then add a cross-attention pathway over future known events (holidays, sales, app updates) so the model can answer questions like "how will the next sale affect engagement for this cohort?" What you'll learn - Why feature-engineered, task-specific models break down on B2C scale - Multi-modal tokenization: activities + event context + static demos - The cross-attention mechanism over future known events - How LUMOS lifts ROC-AUC by 0.025 across 5 production tasks and cuts MAPE by 4.6% on regression - Online A/B test results: +3.15% Daily Active Users #MachineLearning #DeepLearning #Transformers #UserBehavior #FoundationModels #MLOps #ResearchPaper #LUMOS #RecSys #AI