Under the hood · interactive
How vector search works
Every RAG system, recommendation feed, and "find similar" button rests on one problem: given a point, find its nearest neighbors among millions — fast. Here's the trick that makes it fast, and what it quietly gives up to get there.
An embedding turns a piece of text into a point in space, positioned so that similar meanings sit close together. "Search" then becomes geometry: find the stored points nearest your query. The naive way is to measure the distance to every point and keep the closest — correct, but it costs one comparison per item, every query. At a billion vectors that's hopeless. The fix, used by essentially every vector database (HNSW is the default in Pinecone, Qdrant, Weaviate, pgvector, FAISS), is to precompute a navigable graph and just walk it toward the query.
Greedy search on a proximity graph
Click anywhere to drop a query point. The search starts at a fixed entry node and, at each step, hops to whichever neighbor sits closest to your query — until no neighbor is closer. Watch how few nodes it touches.
Drag the "links / node" slider down to 2 and search a few times: with a sparse graph, greedy search gets stuck in local minima and misses the true nearest more often. That's the tradeoff — approximate, not exact.
Why brute force loses
Exact nearest-neighbor search has no shortcut in high dimensions — to be sure you found the closest point, you have to look at all of them. That's fine for a thousand vectors and ruinous for a hundred million, on every single query. Vector search buys its speed by giving up the guarantee: it returns the approximate nearest neighbors, almost always the right ones, for a tiny fraction of the work. The stat above makes the trade concrete — a handful of hops instead of a full scan.
The small-world insight
Why can so few hops reach the right region? Because the graph is built so that most
links are to near points, but a few reach far across the space — the same "six degrees
of separation" structure as social networks. Near links let you home in precisely; the
occasional long link means you're never more than a few hops from anywhere. Greedy
descent — always step to the neighbor closest to the target — turns that structure into
a fast route. HNSW ("Hierarchical Navigable Small World") stacks several such graphs in
layers, sparse at the top for big jumps, dense at the bottom for the final approach, so
search cost grows like log(n) instead of n.
Where it breaks — and why that's usually fine
Greedy search can get trapped in a local minimum: a node with no neighbor closer to
the query, even though a closer point exists somewhere it can't see. Turn the graph
sparse and you'll watch it happen above. Real systems fight this by keeping more links,
searching from several entry points, and holding a small candidate list instead of a
single current-best (the ef knob you'll meet in any vector DB). You dial
recall up toward exact and pay in latency, or down for speed. For finding the
passages to feed a language model, "almost always the right neighbors" is a bargain
nobody notices — which is exactly why this obscure graph trick sits under half the AI
products you use.
One honest caveat: the toy above is 2D and single-layer, so you can see it. Real embeddings live in hundreds or thousands of dimensions (where "nearest" is far less intuitive), and HNSW uses the full layered hierarchy. The mechanism you're clicking on — a navigable graph plus greedy hops toward the query — is the real one; the dimensions and the layers are the scale-up.