Deep dive: Using Reranking to improve search experiences with Chroma Cloud

Reranking is a critical step in making search more useful to users. There’s tons of user and application specific context that is useful to users that is lost if you simply return the results from search alone. Reranking considers signals and data from your application to place results that are more likely to satisfy the user’s search needs. Links - Learning to rank: https://en.wikipedia.org/wiki/Learning_to_rank - XGBoost: https://xgboost.readthedocs.io/en/release_3.2.0/ - LambdaMark: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/MSR-TR-2010-82.pdf - LightGBM: https://github.com/lightgbm-org/LightGBM - Efficient Document Ranking with Learnable Late Interactions: https://arxiv.org/abs/2406.17968 - Passage Re-ranking with BERT: https://arxiv.org/abs/1901.04085 - A Thorough Comparison of Cross-Encoders and LLMs for Reranking SPLADE: https://arxiv.org/abs/2403.10407 - Context-1 Research Report: https://www.trychroma.com/research/context-1 Chapters 0:00 - Intro 2:37 - Example application signals and metrics 6:49 - Aggregating metrics 8:04 - Example algorithmic reranker in code — Chroma builds open-source search infrastructure for AI Fast, serverless, and scalable infrastructure supporting vector, full-text, regex, and metadata search. Built on object storage and trusted by millions of developers. Open-source Apache 2.0. https://trychroma.com/