Postgres pgvector Extension - Vector Database with PostgreSQL / Langchain Integration

☕️ 𝗕𝘂𝘆 𝗺𝗲 𝗮 𝗰𝗼𝗳𝗳𝗲𝗲: To support the channel and encourage new videos, please consider buying me a coffee here: https://ko-fi.com/bugbytes ⭐Top resource to learn Python - https://datacamp.pxf.io/kOjKkV ⭐ Blog Post: https://bugbytes.io/posts/vector-databases-pgvector-and-langchain/ In this video, we'll look at the pgvector extension for PostgreSQL, that allows you to turn your Postgres database into a vector data-store! pgvector adds the vector data-type and distance computation operators (L2, inner product, and cosine distance) to allow you to query for "similar" items in the vector-space. We'll see how to set pgvector up in a Docker container, and will see how to integrate it with Langchain via the PGVector object. We'll look at how to take a piece of text, split it into chunks, create embeddings from those chunks using OpenAI, and then store the embeddings in the Postgres vector database. We'll also see how to query the database for vectors/documents that are similar to a text prompt/query. 📌 𝗖𝗵𝗮𝗽𝘁𝗲𝗿𝘀: 00:00 Intro 00:41 Introduction to pgvector for PostgreSQL 03:23 Splitting text file into chunks with Langchain RecursiveCharacterTextSplitter 06:10 Using OpenAI to get embeddings for each chunk with OpenAIEmbeddings object 10:54 Setting up pgvector and PostgreSQL in a Docker container 16:38 Using the Langchain PGVector object to connect to PostgreSQL 21:47 Finding similar vectors to a query in pgvector 25:29 Querying pgvector with SQL to get cosine distances 𝗦𝗼𝗰𝗶𝗮𝗹 𝗠𝗲𝗱𝗶𝗮: 📖 Blog: https://bugbytes.io/posts/vector-databases-pgvector-and-langchain/ 👾 Github: https://github.com/bugbytes-io/ 🐦 Twitter: https://twitter.com/bugbytesio 📚 𝗙𝘂𝗿𝘁𝗵𝗲𝗿 𝗿𝗲𝗮𝗱𝗶𝗻𝗴 𝗮𝗻𝗱 𝗶𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻: Blog Post: https://bugbytes.io/posts/vector-databases-pgvector-and-langchain/ pgvector: https://github.com/pgvector/pgvector pgvector DockerHub image: https://hub.docker.com/r/ankane/pgvector State of the Union text: https://github.com/hwchase17/chroma-langchain/blob/master/state_of_the_union.txt OpenAI Embeddings: https://platform.openai.com/docs/guides/embeddings/what-are-embeddings Langchain Vectorstores: https://python.langchain.com/docs/modules/data_connection/vectorstores/ #python #langchain #datascience #postgresql