Vector Embeddings & Semantic Search.

🔹What it is

An embedding is a fixed-length list of numbers a model generates from a piece of text (or an image), placing similar meanings near each other in vector space. Semantic search ranks results by the distance between these vectors — usually cosine similarity — instead of matching literal keywords.

🔹Why it matters

Embeddings are the engine behind RAG, recommendations, deduplication, and "find related items" features. Unlike keyword search, a query for "auto coverage" can surface a doc that says "car insurance" — it matches meaning, not spelling. If you build anything with LLMs, this is the retrieval layer underneath it.

🔹Example

import numpy as np

# pretend these came from an embeddings API (e.g. text-embedding model)
def cos(a, b):
    a, b = np.array(a), np.array(b)
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))

query   = embed("how do I reset my password")
docs = {
  "Account recovery steps": embed("steps to recover a locked account"),
  "Pizza toppings menu":    embed("our menu of pizza toppings"),
}
ranked = sorted(docs, key=lambda d: cos(query, docs[d]), reverse=True)
# -> "Account recovery steps" ranks first, even with zero shared keywords

In production you store doc vectors in a vector DB (pgvector, Pinecone, Qdrant) and let it do the nearest-neighbor search for you.

🔹Common mistakes

• Mixing models: vectors from two different embedding models live in different spaces and are NOT comparable. Re-embed everything when you switch models.
• Treating dot product as cosine similarity without normalizing the vectors first — only equal when vectors are unit length.
• Chunking too coarsely: embedding a whole page averages many topics together, so the one relevant sentence gets diluted and retrieval misses it. Chunk by paragraph/section.

🔹Your exercise

Take 5 short sentences (3 about cooking, 2 about programming). Embed them with any model, compute the cosine similarity of every pair, and confirm the cooking sentences cluster together and away from the programming ones. Then add one ambiguous sentence like "I need to debug this recipe" — which cluster does it land in, and why?