The SFT Trainer || Hugging Face

The "SFT Trainer" documentation outlines a Supervised Fine-Tuning (SFT) Trainer within the TRL (Transformer Reinforcement Learning) library, designed for training language models efficiently. It details the process of adapting a language model to a specific dataset through preprocessing, tokenization, and loss computation, using a token-level cross-entropy loss with a one-token shift. The trainer supports various dataset formats, including standard language modeling and conversational data, and offers customization options such as model initialization, example packing, and specific loss computation for assistant messages or completions. Furthermore, the documentation explains how to implement instruction tuning, tool calling, and Vision-Language Model (VLM) training, highlighting the flexibility and capabilities of the SFT Trainer for diverse fine-tuning tasks. It also describes the DataCollatorForLanguageModeling and DataCollatorForVisionLanguageModeling classes, which manage data preparation for text-only and multimodal models, respectively.