Please give us a star 🤗 ❤️: https://github.com/cocoindex-io/cocoindex
Documentation 📖: https://cocoindex.io/docs/
Quick Start Guide ⚡: https://cocoindex.io/docs/getting_started/quickstart
Cocoindex 🥥: https://cocoindex.io/
---------------
Code in this quick start demo: https://github.com/cocoindex-io/cocoindex-quickstart (Only ~50 lines of python code, you can start now 🚀)
----------------
Data indexing is the process of transforming raw data into a format optimized for retrieval. It’s essential to back RAG and semantic search.
CocoIndex is an ETL framework to index data for AI, such as RAG; with realtime incremental updates and support custom logic like lego. It is the world’s first open-source engine that supports both custom transformation logic and incremental updates specialized for data indexing.
Key features:
🔥 Data flow programming
🔥 Support custom logic - you can plugin your own choice of chunking, embedding, vector stores; plugin your own logic like lego. We have three examples in the repo for now. In the long run, we also want to support dedupe, reconcile etc.
🔥 Incremental updates. We provide state management out-of-box to minimize re-computation. Right now, it checks if a file from a data source is updated. In future, it will be at smaller granularity, e.g., at chunk level.
🔥 Python SDK (RUST core 🦀 with Python binding🐍)
Please leave a comment if you have any questions! You can also find us in our discord group https://discord.com/invite/zpA9S2DR7s :)
