Vector Database

What is a vector database?

A vector database stores embeddings and answers “which stored vectors are closest to this one?” in milliseconds across millions of items, using approximate nearest-neighbor (ANN) indexes such as HNSW or IVF. Examples: pgvector (Postgres extension), Pinecone, Qdrant, Weaviate, Milvus, and vector features inside Elasticsearch, MongoDB, and Redis.

Do you actually need one?

This is the question vendors won’t ask for you:

• Under ~100k vectors — you don’t. NumPy brute force or SQLite/pgvector on one box is exact, simple, and fast enough.
• You already run Postgres — pgvector handles millions of vectors and keeps vectors next to your relational data (joins, filters, transactions). This is the default answer for most teams.
• Dedicated vector DB — justified at tens of millions of vectors, heavy filtered search, multi-tenant isolation, or when vector search is the product.

Cost reality

ANN indexes live mostly in RAM: 10M × 1,536-dim float32 vectors ≈ 60 GB before index overhead. That’s what you’re paying managed vendors for. Dimension reduction (Matryoshka embeddings), scalar/product quantization, and disk-based indexes cut this 4–30× with small recall loss — ask vendors about them before scaling up instances.

What people get wrong

• Choosing a vector DB before validating retrieval quality. Bad chunking with a fancy DB still retrieves garbage; see RAG.
• Ignoring metadata filtering performance. “Vectors near X where tenant_id = Y” is where many ANN indexes fall over; benchmark filtered queries specifically.
• Treating recall as 100%. ANN is approximate — measure recall against brute force on your own data, don’t trust defaults.

Choosing a Vector Database

Popular options span a spectrum of tradeoffs: managed services like Pinecone minimize operational overhead at the cost of vendor lock-in and per-query pricing, while self-hosted options like Qdrant and Milvus offer more control and can be more cost-effective at very large scale, at the cost of needing to operate the infrastructure yourself. Some general-purpose databases (Postgres with the pgvector extension, for example) have also added vector search capability, which can be attractive for teams that want to avoid introducing a completely separate database system for a workload that’s otherwise a natural fit alongside existing relational data.

Beyond Simple Similarity Search

Modern vector databases increasingly support hybrid search (combining vector similarity with keyword-based BM25 scoring), metadata filtering (restricting a search to a subset of vectors matching structured criteria), and multi-tenancy features like namespaces — capabilities that have become table stakes for production retrieval-augmented generation systems, where pure unfiltered vector similarity search is rarely sufficient on its own to produce consistently high-quality retrieval results at scale.