vector db

A vector database stores embeddings — the dense numerical vectors an encoder produces from text, images, or audio — and answers one question fast: of millions or billions of vectors, which ones sit closest to a query vector. Computing exact distance to every stored vector is linear in the collection size, and at the scale modern applications need — a hundred million embeddings is routine — that brute-force scan is too slow to serve a request in real time. Vector databases exist to trade a small, controllable amount of recall for a large, reliable amount of speed, turning a linear scan into a graph walk or a cluster lookup that finishes in single-digit milliseconds.

The index is what makes the search approximate, not the database itself. HNSW, introduced by Malkov and Yashunin in 2016, builds a multi-layer graph where the top layer has few nodes and long-range links and each lower layer is denser; a query descends from the sparse top layer toward the dense bottom one, taking roughly logarithmic hops instead of comparing against every stored vector. IVF (inverted file) instead clusters the vector space during a training pass and, at query time, searches only the handful of clusters nearest the query — a coarser but cheaper strategy. Product quantization (Jégou and colleagues, 2011) compresses each vector into a short code made of sub-vector centroids, trading some distance-computation accuracy for an order-of-magnitude memory reduction; FAISS's popular IVF-PQ pairs the two, clustering first and compressing what's inside each cluster.

Recall and latency are a dial, not a fixed property of the database. Every ANN index exposes parameters — HNSW's ef_search, IVF's number of clusters probed — that trade search thoroughness for speed; turning the dial up recovers accuracy at the cost of scanning more of the structure, and turning it down returns results faster but risks missing the true nearest neighbor. A brute-force flat index is the one exception: it has no such dial because it checks every vector and returns the exact answer, which is why it remains the right choice for small collections where exactness matters more than scale.

Distance metric has to match how the embedding model was trained. Cosine similarity measures the angle between two vectors regardless of their length, which is why it is the default for text embeddings whose magnitude is not meaningful. Euclidean (L2) distance measures straight-line distance and suits embeddings where overall magnitude carries information, such as some image features. Dot product (inner product) measures both angle and magnitude at once and is the right metric when the embedding model was trained with a dot-product objective — normalized vectors make dot product and cosine equivalent, which is why many systems just normalize and use one code path for both.

The ecosystem splits into libraries, dedicated databases, and extensions to existing databases. FAISS, released by Meta AI in 2017, is a library rather than a server — it runs embedded in a process and underlies much of the rest of the ecosystem. Milvus and Qdrant (the latter written in Rust) and Weaviate are purpose-built servers with replication, filtering, and multi-tenancy; Pinecone is the fully managed, serverless version of the same idea. Chroma is deliberately lightweight and embeddable, popular for local prototyping. pgvector, first released in 2021, adds a vector column type and ANN indexes directly to PostgreSQL — the right choice whenever the rest of the application data already lives there and a second database would be pure operational overhead.

Hybrid search exists because embeddings are bad at exact terms. A dense vector captures meaning but tends to blur rare identifiers — part numbers, product SKUs, exact names — that a sparse keyword method like BM25 matches precisely. Production retrieval systems increasingly run both searches and merge the ranked lists, commonly with reciprocal rank fusion, so a query benefits from semantic matching and exact-term matching at once rather than betting everything on the embedding.

Updating an ANN index is harder than building one, and that asymmetry shapes production design. Most graph and cluster indexes are built around a largely static snapshot; inserting a new vector into HNSW is cheap, but removing one usually means marking it as a tombstone rather than restructuring the graph, and an index that accumulates enough tombstones eventually needs a full rebuild. Filtering by metadata at query time — restrict the search to one tenant's documents, or to items in stock — has to be woven into the index traversal itself, or the ANN search returns a candidate set that then gets filtered down to far fewer results than requested.

Read vector databases as approximate-search infrastructure, not exact databases that happen to hold vectors: the entire field is a set of answers to the same question — how much recall can be spent to buy how much speed — and the right engine is whichever one spends that budget the way a given workload needs, whether that means a dedicated server, an embedded library, or one more column type on a database that was already there.