Solving Large MDPs Exactly: A Tensor-Based MDP Solver

MDPs power everything from LLMs to self-driving cars. But when applied to real-world systems with multiple components, the state space grows exponentially and standard solvers break down. In this video, I explain our latest research where we developed a tensor-based MDP solver that exploits Kronecker structure along with tensor algebra to solve these problems exactly. No approximation. For that same system, memory drops to 8.4 MB and computation is ~10,000× faster. Paper: https://arxiv.org/abs/2604.23068 Code (MATLAB & Python): https://github.com/jodha-sudhir/tensor-mdp-solver