Vector Search in Modern Databases

Abstract

In this talk, we’ll explore the emergent landscape of vector search in databases, a paradigm shift in information retrieval. Vector search, traditionally the domain of specialized systems, is now being integrated into mainstream databases and search engines like Lucene, Elasticsearch, Solr, PostgreSQL, MySQL, MongoDB, and Manticore. This integration marks a significant evolution in handling complex data structures and search queries.

1. Introduction to Vectors and Embeddings in Databases • Definition and significance of vectors and embeddings. • The historical context of vector search and its integration into databases.
2. Computing Embeddings: Where and How • Strategies for embedding computation: In-database processing vs. external tools. • Current capabilities of databases like MySQL (referring to PlanetScale’s initiative), PostgreSQL, etc., in embedding computation.
3. Indexing for Enhanced Vector Search • The role of indexing in optimizing vector search. • Different indexing strategies and their impact on performance and accuracy.
4. Hybrid Search Approaches • Combining vector search with traditional search methods.
5. Measuring Performance and Quality • Beyond speed: Assessing the effectiveness of vector search. • Metrics for evaluating the quality of search results.

Conclusion

The session will conclude with insights into future trends and the potential impact of vector search technologies on data retrieval, AI applications, and beyond.