Back to Browse

FAISS Vector Library with LangChain and OpenAI (Semantic Search)

43.9K views
Feb 27, 2024
19:59

🧠 Don’t miss out! Get FREE access to my Skool community — packed with resources, tools, and support to help you with Data, Machine Learning, and AI Automations! 📈 https://www.skool.com/data-and-ai-automations-4579 In this video, we take a look at the Facebook AI Similarity Search (FAISS) vector library. Through a few examples, we will grab a document, chunk it, set up embeddings, and search through it. Code: https://ryanandmattdatascience.com/faiss-langchain/ 🚀 Hire me for Data Work: https://ryanandmattdatascience.com/data-freelancing/ 👨‍💻 Mentorships: https://ryanandmattdatascience.com/mentorship/ 📧 Email: [email protected] 🌐 Website & Blog: https://ryanandmattdatascience.com/ 🖥️ Discord: https://discord.com/invite/F7dxbvHUhg 📚 *Practice SQL & Python Interview Questions: https://stratascratch.com/?via=ryan 📖 *SQL and Python Courses: https://datacamp.pxf.io/XYD7Qg 🍿 WATCH NEXT OpenAI/Langchain Playlist: https://www.youtube.com/watch?v=5qP6u-WGSPk&list=PLcQVY5V2UY4Kat6vxC7ESzIIzHWdwlnak&ab_channel=RyanNolanData Vector Embeddings: https://www.youtube.com/watch?v=xdr5ByCpnd4&ab_channel=RyanNolanData Langchain Chains: https://www.youtube.com/watch?v=gQnJEjiaHFw&ab_channel=RyanNolanData Streamlit Langchain: https://www.youtube.com/watch?v=qXcMGBj4i3A&feature=youtu.be In this video, I walk you through using FAISS (Facebook AI Similarity Search) to build a vector database for document search and retrieval. We start by breaking down what FAISS is and how it stores vector embeddings, then move into practical Python implementation using LangChain and OpenAI embeddings. I demonstrate the complete workflow: loading a Metallica Wikipedia article, chunking it with a recursive text splitter, converting chunks into embeddings, and storing them in a FAISS vector library. You'll see three key use cases in action—similarity search queries, retrievers with QA chains, and saving/loading FAISS indexes for reuse. We test the system with real queries like "Who replaced Cliff Burton?" and "What Metallica album do fans hate the most?" to show how semantic search works in practice. By the end of this tutorial, you'll understand how to implement vector databases for document search, perform similarity searches with scoring, use retrievers for question-answering systems, and persist your FAISS indexes to avoid rebuilding them. Whether you're building a chatbot, search engine, or RAG application, this foundational knowledge will help you leverage vector databases effectively. All code is demonstrated in Google Colab with step-by-step explanations. TIMESTAMPS 00:00 Introduction to FAISS 01:00 Setting Up Google Colab & Imports 03:02 Loading Documents & Text Splitting 05:17 Creating Embeddings & Vector Library 08:48 Similarity Search Query Example 12:22 Understanding Similarity Scores 14:40 Using FAISS as a Retriever 17:32 Testing Query Results 18:17 Saving & Loading FAISS Index OTHER SOCIALS: Ryan’s LinkedIn: https://www.linkedin.com/in/ryan-p-nolan/ Matt’s LinkedIn: https://www.linkedin.com/in/matt-payne-ceo/ Twitter/X: https://x.com/RyanMattDS Who is Ryan Ryan is a Data Scientist at a fintech company, where he focuses on fraud prevention in underwriting and risk. Before that, he worked as a Data Analyst at a tax software company. He holds a degree in Electrical Engineering from UCF. Who is Matt Matt is the founder of Width.ai, an AI and Machine Learning agency. Before starting his own company, he was a Machine Learning Engineer at Capital One. *This is an affiliate program. We receive a small portion of the final sale at no extra cost to you.

Download

1 formats

Video Formats

360pmp447.1 MB

Right-click 'Download' and select 'Save Link As' if the file opens in a new tab.

FAISS Vector Library with LangChain and OpenAI (Semantic Search) | NatokHD