Meaning as Numbers: What an Embedding Is
Search a photo app for "beach" and it finds your holiday pictures, even though you never tagged them. Ask a support bot about "getting my money back" and it finds the page titled "Refund policy." Your music app suggests a song you have never heard that fits your taste.
All of these rely on the same idea: an embedding. An embedding turns a piece of content, such as a sentence, a paragraph, an image, or a song, into a list of numbers that captures what it means. Once meaning is numbers, a computer can compare, sort, and search by meaning instead of by exact words. This lesson explains the idea without any math.
What You'll Learn
- Why computers need meaning turned into numbers
- What an embedding is, using a map as the picture
- Why real embeddings use hundreds of dimensions
- Where embeddings come from
- What embeddings capture well, and what they leave out
The problem: words are not meaning
Computers are very good at matching exact text. "Refund" matches "refund." But people say the same thing in many ways: "money back," "return my payment," "cancel and get reimbursed." To a simple text match, these have nothing in common.
Meaning is also about more than words. "The bank of the river" and "the bank approved my loan" share a word but not a meaning. A useful search has to tell them apart.
So we need a way to represent what something is about, in a form a computer can compare.
The picture: a map of meaning
Imagine a huge map where every sentence has a location. The map is arranged so that sentences with similar meaning sit close together, and sentences about different things sit far apart.
- "How do I get a refund?" and "I want my money back" sit almost on top of each other.
- "What is your return policy?" sits nearby.
- "What time does the store open?" sits in a different region.
- "The river bank flooded" is far away from all of them.
- Text"I want my money back"
- Embedding modelReads it and works out its meaning
- EmbeddingA list of numbers: its location on the map
An embedding is simply that location, written as numbers. On a flat paper map, a location is two numbers: how far across and how far up. An embedding is the same idea with many more numbers.
Why so many numbers
Two numbers can only describe two things. A flat map could put "money topics" on one side and "time topics" on the other, but meaning has far more going on: topic, tone, formality, whether it is a question, whether it is positive or negative, and countless subtler features.
So real embeddings use hundreds or even thousands of numbers for each piece of content. You cannot picture a map with that many directions, and you do not need to. The key idea stays exactly the same: every item gets a location, and similar meanings end up close together.
Also, no single number means one neat thing like "about money." The meaning is spread across all of them together, much like the knowledge inside a neural network is spread across its weights. How Neural Networks Actually Work explains that idea if you are curious.
Where embeddings come from
Embeddings are produced by an embedding model: a neural network trained on huge amounts of text (or images, or audio) to place similar things close together. During training, it sees many pairs of things that belong together, such as a question and its answer, or a photo and its caption, and learns to put them near each other on the map.
Once trained, the model can place any new text on the map, even sentences it has never seen. That is what makes embeddings useful: you can embed your own documents, products, or support articles, and they land in sensible places.
Embedding models are related to the chatbots you use, but they do a different job:
Chat models write. Embedding models place things on the map.
| Criteria | Chat model | Embedding model |
|---|---|---|
| Input | Your message | A piece of text, image, or audio |
| Output | New text, word by word | One list of numbers |
| Main job | Write and converse | Represent meaning for comparing |
| Speed and cost | Slower, costlier per use | Very fast and cheap per item |
Chat model
- Input
- Your message
- Output
- New text, word by word
- Main job
- Write and converse
- Speed and cost
- Slower, costlier per use
Embedding model
- Input
- A piece of text, image, or audio
- Output
- One list of numbers
- Main job
- Represent meaning for comparing
- Speed and cost
- Very fast and cheap per item
Because embedding models are fast and cheap, a company can embed millions of documents or products once, then search through them instantly.
What embeddings capture, and what they miss
Embeddings are good at capturing:
- Topic: what something is about.
- Paraphrases: different words, same meaning.
- Related ideas: "laptop" near "computer," "vacation" near "holiday."
- Rough intent: a question about a problem sits near pages that solve it.
They are weaker at:
- Exact details: an order number, a product code, or a specific date may not affect the location much.
- Negation: "I love this" and "I do not love this" can land surprisingly close, because they are about the same thing.
- Very long texts: squeezing a whole long document into one location blurs the details, which is why documents are usually split into smaller pieces first.
These weaknesses explain many search mistakes you have probably seen, and later lessons build on them.
Key Takeaways
- An embedding turns content into a list of numbers that represents its meaning.
- Think of it as a location on a map of meaning, where similar meanings sit close together.
- Real embeddings use hundreds or thousands of numbers, and meaning is spread across all of them.
- Embedding models are trained to place related things near each other, and can place new content they have never seen.
- Embeddings capture topic and paraphrase well, but can miss exact details, negation, and fine points in long texts.