Benchmark embedding models #4 - Generate question-answer pairs from text chunks

In this video, you'll learn how to generate question-answer (QA) pairs from text chunks. This video is part 4 of the "Benchmark Embedding Models" series. I'll walk you through the entire process of using Large Language Models (LLMs) to create a high-quality QA dataset from a document we chunked in the previous video. First, I'll show you how to use a proprietary model, Google's Gemini, to generate questions. We'll build a detailed system prompt that instructs the model on its role, the required JSON output format, and provides clear examples to ensure relevant, high-quality results. To guarantee the Gemini API returns valid, structured data, we'll define a Pydantic object. I'll then show you the Python code to load our text chunks from a JSON file, loop through each one, and call the Gemini API to generate questions. We'll see how to link each generated question back to its original chunk_id and handle potential API rate limits. Next, I'll demonstrate how to achieve the same result using a powerful open-source model, Qwen3-30B-A3B-Instruct-2507, running locally. We'll find the quantized GGUF model on Hugging Face and serve it using llama.cpp's llama-server. I'll show you the exact terminal command to get the server running. Here is the link to the GitHub repository: https://github.com/ImadSaddik/Benchmark_Embedding_Models Don't forget to like, subscribe, and leave a comment if you have any questions or feedback! ⭐ Contents ⭐ (00:00) Introduction (02:02) The GitHub repository containing the source code (02:25) The first notebook that uses Gemini (09:30) Switching to the open-source model (Qwen) notebook (09:59) Finding the Qwen GGUF model on Hugging Face (10:53) Starting the local llama-server with llama.cpp (11:57) Adapting the code to use the llama.cpp server endpoint (15:32) Conclusion