ANN Vector Indexes (HNSW) — Recall Tuning.

🧠The myth

"My vector DB returns the top 5 nearest chunks. If retrieval is bad, my embeddings or my chunking must be bad."

🎯Actually…

Past a few hundred thousand vectors, your DB is not doing nearest-neighbour search at all. It's doing APPROXIMATE nearest-neighbour (ANN). It returns 5 pretty-close vectors — and silently drops the true best match some fraction of the time. That fraction is a config value you probably never set.

pgvector, Qdrant, Weaviate, Pinecone, Milvus: all HNSW (or IVF) by default.

📉Quick proof — run this on your own data

Take 200 real queries. Get ground truth with an exact scan:

SET LOCAL enable_indexscan = off;   -- force brute force
SELECT id FROM chunks ORDER BY emb <=> :q LIMIT 5;

Then the normal indexed query. Overlap of the two id sets ÷ 5 = your recall@5.

Typical result on 1M vectors with default settings: 0.85–0.93. So ~1 query in 10 is missing its best chunk before the LLM ever sees the prompt. No error, no log line. Just a slightly worse answer.

🕸️Why: HNSW is a skip-list of neighbourhoods

Vectors form a multi-layer graph. Search enters at a sparse top layer, greedily walks toward the query, drops a layer, repeats. Greedy walks get stuck in local minima — so you keep a candidate list of size ef_search and explore from all of them.

Bigger ef_search = more of the graph visited = higher recall, more latency. Linear-ish cost, diminishing returns.

🎚️The three knobs

• ef_search (query time) — THE dial. Cheap to change, no rebuild. Default is often 40; try 100–200.

SET LOCAL hnsw.ef_search = 120;   -- pgvector, per-transaction

• m (build time) — edges per node. 16 default; 32–48 for high-dim or 10M+. Costs RAM.
• ef_construction (build time) — graph quality. 64 → 200 gives better recall at every ef_search, but slower index builds.

Rule of thumb: ef_search must be ≥ your LIMIT. If you fetch top-50 for a reranker with ef_search=40, recall collapses — and that's a very common setup.

⚠️Three things practitioners get wrong

1. Benchmarking recall on 10k rows. ANN ≈ exact at small scale; the gap only opens at 100k+. Your staging DB lies.
2. Filtered search. WHERE tenant_id = 7 AND ... + ANN is the danger zone. Post-filtering walks the graph, then throws matches away — a selective filter can return 2 rows out of 5, or zero. Use your DB's native filtered-HNSW, or partition per tenant.
3. Measuring latency after a rebuild but recall never. Index rebuild + a new embedding model = a different graph. Recall moves silently.

🔢Shape of the tradeoff (1M vecs, 768-dim, one core):

ef_search=40   → p95 3ms   recall@10 0.88
ef_search=120  → p95 7ms   recall@10 0.97
ef_search=400  → p95 22ms  recall@10 0.995

7ms is nothing next to a 900ms LLM call. Most RAG systems are tuned for a latency budget they aren't actually spending.

✅Takeaway

Recall is a product decision, not a database default. Pick a target (0.95+ for RAG), measure it, set ef_search to hit it.

🏋️Your exercise

Measure recall@10 on your own index with 50 real queries using the brute-force comparison above. Then raise ef_search until recall crosses 0.97 and record what it cost you in p95 ms. If the answer is "under 15ms" — why are you still running the default?