TEST TIME Optimized AI REASONING (MIT)

Optimized Test-Time Training by @mit : Shaping AI’s Future in Reasoning. This brilliant video introduces a novel approach to improving reasoning capabilities in large language models (LLMs) through Test-Time Training (TTT) with a Leave-One-Out (LOO) strategy, specifically applied to the Abstraction and Reasoning Corpus (ARC). ARC tasks require abstract pattern recognition and rule inference, often with only a few input-output examples. TTT addresses this by dynamically fine-tuning lightweight Low-Rank Adapters (LoRA) at inference time. The method deconstructs the main task into independent subtasks, using LOO to exclude one test input-output pair while fine-tuning on the remaining pairs and augmented data. This fine-tuning adapts the model to the specific logic of each task, enabling the LLM to better generalize abstract transformations while avoiding information leakage from the excluded pair. The augmentation process enriches the limited examples with transformations like flips, rotations, and rule-based variations, ensuring robust task-specific adaptation. This dynamic TTT process contrasts with static pre-training or in-context learning by actively updating model parameters during inference. Unlike in-context learning, which leverages examples directly as input without parameter updates, TTT uses the auxiliary dataset to fine-tune LoRA adapters for each subtask independently. This enables the model to handle ARC’s unique challenges, such as generalizing from minimal data and adapting to task-specific reasoning rules. Achieving a state-of-the-art accuracy of 53% on ARC validation, the approach demonstrates significant performance improvements over baseline methods and offers a scalable framework for abstract reasoning tasks, especially in few-shot scenarios. All rights w/ authors: The Surprising Effectiveness of Test-Time Training for Abstract Reasoning https://arxiv.org/pdf/2411.07279v1 00:00 Optimization of Test Time Training 01:08 ARC Intelligence test for AI 02:37 3 Insights into TTT 05:17 Test Time Dataset Creation 08:05 This is not ICL 09:47 Pre-train - Finetune - LoRA Adapter 13:00 ARC Dataset Characteristics 15:54 9000% Human AI 17:47 Leave One Out training 20:17 Cheating? 22:37 Limitations on TTT* 25:21 AI Agents and Security 26:30 Combine w Reward Policy MCTS #reasoning #ai #massachusettsinstituteoftechnology #training #aieducation #robot