Close Means Related: How Similarity Works
Once every piece of content has a location on the map of meaning, one simple operation unlocks almost everything: measuring how close two things are. Close means related. Far means unrelated. Search, recommendations, and duplicate detection all come down to this.
This lesson explains how closeness is measured, in plain language, and what that means for the results you see.
What You'll Learn
- How AI measures whether two items are similar
- Why the direction something points matters more than its size
- What a similarity score is and how to read one
- How "nearest neighbours" turns similarity into search results
- Why a high score does not always mean a correct answer
Measuring closeness
Think back to the map. To find items similar to a question, you place the question on the map, then look at what sits nearby.
There are a few ways to measure "nearby," and they usually agree. The most common idea is to compare the direction each item points in, rather than the straight-line distance between them. Imagine every item as an arrow starting from the center of the map. Two arrows pointing almost the same way mean two things about almost the same topic.
Why direction? Because a long text and a short text about the same topic should count as similar. The direction captures what something is about, regardless of how much text there is. This direction-based measure is called cosine similarity. You do not need the formula, only the idea: same direction, same meaning.
Similarity scores
Comparing two embeddings produces a similarity score. With the common direction-based measure, higher scores mean more similar.
Scores rank how related things are. They do not say whether an answer is correct.
| Criteria | Pair of sentences | Rough similarity |
|---|---|---|
| Near-identical meaning | "How do I get a refund?" vs "I want my money back" | Very high |
| Same topic, different question | "How do I get a refund?" vs "How long do refunds take?" | High |
| Loosely related | "How do I get a refund?" vs "What payment methods do you accept?" | Medium |
| Unrelated | "How do I get a refund?" vs "The river flooded overnight" | Low |
Pair of sentences
- Near-identical meaning
- "How do I get a refund?" vs "I want my money back"
- Same topic, different question
- "How do I get a refund?" vs "How long do refunds take?"
- Loosely related
- "How do I get a refund?" vs "What payment methods do you accept?"
- Unrelated
- "How do I get a refund?" vs "The river flooded overnight"
Rough similarity
- Near-identical meaning
- Very high
- Same topic, different question
- High
- Loosely related
- Medium
- Unrelated
- Low
Two cautions about scores:
- Scores are relative. The exact numbers depend on the embedding model. A score that counts as "very similar" for one model may be ordinary for another. What matters is the ranking: which items score highest compared with the rest.
- Similar is not the same as correct. A page about "how long refunds take" is very similar to "how do I get a refund," but it does not answer the question. Similarity finds related content. It does not check that the content answers anything.
Nearest neighbours: similarity becomes search
To search with embeddings, a system does three things:
- Embed the queryPlace your question on the map
- CompareMeasure closeness to stored items
- Return the nearestTop results by similarity
The closest items are called the nearest neighbours, and returning them is called nearest-neighbour search. With millions of items, comparing against every single one would be slow, so systems use clever shortcuts to find the nearest items quickly with very high accuracy. Special databases built for this, called vector databases, store embeddings and run these searches fast. Vector Databases: The Foundation of AI Apps covers them from a developer's point of view.
What similarity gets wrong
Because similarity is about meaning in general, it has predictable blind spots. These show up in real products:
- Related but wrong. It returns content on the right topic that does not answer the actual question.
- Missing exact matches. You search for an order number like "A-4471" and the right record does not rank first, because the number barely affects its location on the map.
- Opposites landing close. "Is this product safe for children?" can match a page that says "not safe for children," because both are about child safety.
- Popular wording wins. Content written in common phrasing can rank above more precise content written in unusual terms.
The practical lesson: similarity search is excellent for finding related material, and weak at exact facts and fine distinctions. Good systems often combine it with other checks, which the next lessons cover.
Key Takeaways
- Search by meaning works by measuring how close two embeddings are on the map.
- The common measure compares direction (cosine similarity): same direction, same meaning.
- Similarity scores are useful for ranking, but exact values depend on the model, and similar does not mean correct.
- Nearest-neighbour search returns the closest items, and vector databases make it fast at scale.
- Similarity struggles with exact codes, negation, and fine distinctions, so good systems add other checks.