Build a Semantic Search Engine

Keyword search has a dumb failure mode: search “how do I reset my password” and a document titled “account recovery steps” never shows up, because they share no words. Embeddings (chapter 19) fix exactly this — they turn text into vectors where meaning lives close together, so “reset password” and “account recovery” land near each other even with nothing in common on the surface. This capstone turns that idea into a working semantic search engine you can type into, closing out Part III with the encoder doing real work.

No training, no shipped weights — just MiniLM running inference, the way you ran it in the last chapter.

▶ Open the companion notebook in Colab

Run against: PyTorch 2.2.2, transformers 4.57.6 and sentence-transformers 5.7.0 on CPU (macOS). Gradio 6.17.3 for the component signatures; the UI was not driven in a browser.

Embed the corpus once

A search engine has two halves: a collection of documents you’ve turned into vectors ahead of time, and a query you turn into a vector on the fly. Let’s start with a tiny corpus — in a real app this would be thousands of help articles — and embed it all at once with the sentence-transformers model from chapter 19.

from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

docs = [
    "How to reset your account password",
    "Updating your billing and payment method",
    "Steps to recover a locked account",
    "Exporting your data as a CSV file",
    "Turning on two-factor authentication",
]

doc_embeddings = model.encode(docs, convert_to_tensor=True)
print(doc_embeddings.shape)   # torch.Size([5, 384])

Each document is now a point in 384-dimensional space — [5, 384] is “five documents, 384 numbers each.” You embed the corpus once and reuse it for every search; only the query changes.

Search by meaning

Searching is three steps: embed the query, compare it to every document with cosine similarity (the angle-based closeness from chapter 19), and return the nearest few.

def search(query):
    if not query.strip():
        return {}
    q = model.encode(query, convert_to_tensor=True)
    scores = util.cos_sim(q, doc_embeddings)[0]    # one score per document
    top = scores.topk(3)
    return {docs[i]: float(scores[i]) for i in top.indices}

util.cos_sim does the comparison you wrote by hand in the last chapter, just vectorized over the whole corpus. topk(3) grabs the three highest scores. Try it: search("I forgot my login") ranks “reset your account password” (0.61) and “recover a locked account” (0.47) right at the top — neither shares a meaningful word with the query. That’s the whole point of searching by meaning.

Put a search box on it

Same Gradio move as the other capstones: describe the input and output, get a UI. A text box in, a ranked list out.

import gradio as gr

gr.Interface(
    fn=search,
    inputs=gr.Textbox(label="Search", placeholder="Describe what you're looking for…"),
    outputs=gr.Label(num_top_classes=3, label="Best matches"),
    title="Semantic search",
    description="Finds results by meaning, not keywords.",
).launch(share=True)

Run it and you’ve got a search box that understands intent. share=True gives you the public link to send someone. Swap docs for your own list — notes, recipes, product descriptions — and you’ve built search for that, with no change to the logic.

Gotchas

• Embed the corpus once, not per query. Re-encoding all documents on every search is the classic beginner mistake — it’s slow and wasteful. Encode them once at startup; only the query is embedded live.
• Query and documents need the same model. Cosine similarity only means something if both vectors come from the same embedding model. Mixing models gives garbage scores.
• Cosine similarity isn’t truth. A high score means “nearby according to MiniLM,” not “the correct answer.” For a few documents that’s plenty; at scale you’d add re-ranking.
• This is the retrieval half of RAG. Swap the gr.Label for “paste the top matches into a prompt and send them to a chat model” and you’ve built retrieval-augmented generation — which is exactly where Part IV is heading.
• Big corpora need a vector database. Comparing against five documents in a loop is fine; comparing against five million is what tools like FAISS or a vector DB are for. Same idea, built to scale.

What’s next

You’ve now used the encoder side of the transformer for everything it’s good at: classification, embeddings, and now search by meaning. The model understands text and hands you back vectors and labels.

But it never writes anything. Part IV fixes that — the decoder half of the family, the generative models that actually produce words and feel like the “AI” everyone talks about. It’s also the most demanding stretch of the book: bigger downloads, GPUs, more ways to trip. So it opens with one tooling stop — how to fit a model that “shouldn’t” fit onto the GPU you have.

Next: Running Big Models: Memory, dtype, Placement, and 4-bit.