Partner With Us

← All lessons / Building blocks / Lesson 04 of 12

Interactive lesson

Embeddings and search: finding meaning, not just matching words.

How can a computer know that “people giving their time” is about volunteering, even though the word never appears? Embeddings make this possible. They are used in semantic search and RAG.

You’ll learn to
  • Compare exact-word matching with similarity search.
  • Judge whether a retrieved passage is relevant to the question.

Meaning as a location

An embedding turns a piece of text into a list of numbers, representing a point in a “meaning space”. An embedding model is trained so that texts with similar meanings land close together, even when they use completely different words.

Once text is a point, searching becomes geometry: turn the question into a point too, and find the nearest neighbours. This is called semantic search, and it is the engine inside most “chat with your documents” tools.

About this lab: real embeddings have hundreds or thousands of numbers (called dimensions) and are learned by a model. Ours have just five hand-labelled dimensions: volunteering, money, research, health and events, so you can see exactly what is going on. In real embeddings, individual dimensions don't have human-readable names like these: meaning is spread across all of them. The similarity calculations are real.

Interactive

Explore the meaning map

Each dot is a short piece of text from a charity's website. Choose a search question: it appears as a navy star, placed among the texts it is closest to, and the three most similar texts are linked to it. The map positions are arranged for illustration; they are not calculated from the five numbers. Compare the similarity scores rather than distances on the drawing.

Closest matches

This lab uses cosine similarity to compare the directions of its illustrative vectors. A score closer to 1 means more aligned vectors; it does not prove a passage answers the question.

    Interactive

    Keyword search versus meaning search

    Run the same question through both kinds of search. Our simple keyword matcher counts exact shared words; meaning search compares hand-set vectors. Real keyword systems can also match word endings (so “donate” finds “donation”) and synonyms. Which wins depends on the question. Try each one and compare the results.

    Keyword search

      Meaning search

        Key ideas to take away

        • Embeddings represent patterns of meaning as numbers. Related texts often score highly, but similarity does not prove factual agreement or that a passage answers your question.
        • Semantic search finds related ideas even when the words differ. For example, “caring for a relative” finds “support for carers”.
        • Keyword search still matters for exact names, codes and acronyms such as “Gift Aid” or “MND”. Many real systems combine both (hybrid search).
        • Embeddings are only as good as the model that made them. General models may not understand specialist medical or legal language well.

        Check your understanding

        3 quick questions. Try as often as you like. Your progress stays on this device; answers clear when you reload.

        Want to go deeper?Equations, code and research for technical learners. Optional.

        Cosine similarity

        Similarity is usually measured by the angle between two vectors, ignoring their length:

        cos(a, b) = (a · b) / (‖a‖ ‖b‖)      ranges from −1 to 1

        The meaning map uses exactly this on its five-dimensional vectors. Real embedding models produce 384 to 3,072 dimensions, so we can't draw them directly; tools such as UMAP or t-SNE squash them into two dimensions for plotting.

        Real embeddings in Python

        from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("all-MiniLM-L6-v2")   # small, free, runs on a laptop
docs  = ["Support for carers looking after a family member",
         "Setting up a monthly direct debit donation"]
query = "help for someone caring for a relative"

scores = util.cos_sim(model.encode(query), model.encode(docs))
print(scores)   # the carers text scores much higher

        Storing embeddings at scale

        A vector database such as Chroma or Qdrant stores embeddings and supports similarity search. FAISS is a similarity-search library rather than a complete database. Approximate indexes can make searches over large collections faster. Keyword search usually uses BM25, a scoring formula that rewards rare words that appear in a document. You'll use a version of it in Lesson 05.

        Further reading

        Mikolov et al. (2013), “Efficient Estimation of Word Representations in Vector Space” (opens in a new tab), the word2vec paper. Reimers & Gurevych (2019), “Sentence-BERT” (opens in a new tab), on sentence embeddings for search.

        Continue exploring

        We've picked resources that explain these ideas from different angles. They are independent of Code With Impact.

        Sources for this lesson

        The example texts, five-dimensional meaning vectors, map and comparison lab are original Code With Impact material. Similarity scores are calculated in your browser.

        How we reference: we link and credit, we never copy other educators' text, diagrams or code. See our full Sources & credits register.