Weights: Where a Network's Knowledge Actually Lives

Here is a claim that sounds wrong the first time you hear it. A trained neural network contains no facts. There is no table of information inside it, no list of rules, no stored sentences, no filed-away images. Open one up and you find a very long list of numbers and nothing else.

Those numbers are the weights from the last lesson. Understanding that the weights are the knowledge, and that nothing else is in there, clears up more confusion about AI than almost any other single idea.

What You'll Learn

• How weights connect one layer of neurons to the next
• Why the weights are the only place a network stores what it learned
• What actually changes when a network is trained, and what stays fixed
• Why this explains several things people find strange about AI

Every connection carries its own number

In the last lesson a neuron had a weight for each input it received. Now scale that picture up.

In a network, neurons are arranged in groups. Every neuron in one group typically connects to every neuron in the next group, and each of those connections carries its own separate weight. That multiplies quickly. Connect a group of 100 neurons to another group of 100 and you have created 10,000 connections, each holding its own number.

Real networks are far larger than that. A modest one holds hundreds of thousands of weights. Large modern systems hold billions. When you see a model described with a number like "8B" or "70B", that count refers to these learned numbers, which are also called parameters. If you want the specifics of what those model sizes mean in practice, How LLMs Actually Work covers that ground directly.

The important point here is structural. The network's size is really a count of adjustable dials, and every one of them is a small piece of whatever the model ended up learning.

Before training, the numbers are random

This is the detail that makes the whole thing click.

When a network is first built, its weights are set to small random values. The architecture is fully in place at that moment. All the neurons exist, all the connections exist, all the layers are wired correctly. And the network is completely useless. Feed it a photo and it produces confident nonsense, because the numbers along every path were picked at random.

Nothing about the structure changes after that. Training does not add neurons, invent new connections, or write rules into memory. Training only adjusts those numbers. Every bit of capability a network ever gains comes from moving weights away from their random starting values toward values that happen to produce good answers.

A model that recognizes handwriting and a model that predicts electricity demand can have identical structures. What separates them is entirely which numbers ended up in the dials.

The mixing desk picture

Imagine an enormous audio mixing desk with millions of faders. Sound comes in one side and out the other, and every fader nudges how one part of the signal flows through.

Slide the faders randomly and you get noise. Spend a long time adjusting them until the output sounds right, and you have a desk that produces good sound. But nowhere on that desk is the music stored. There is no recording inside it. There is only a particular arrangement of fader positions that transforms whatever comes in.

A trained neural network is that desk. The "knowledge" is distributed across millions of dial settings, none of which means anything by itself.

Why this explains so much

Once you see knowledge as dial positions rather than stored facts, several puzzling things about AI become straightforward.

Why a model file is just a file. A trained model can be copied, shared, and loaded onto another machine because it is literally a list of numbers saved to disk. Nothing else needs to travel with it.

Why models cannot simply be edited. You cannot open a model and correct a fact the way you would fix a row in a spreadsheet. The thing you want to change is smeared across millions of weights that also do a hundred other jobs. This is why fixing model behavior usually means more training rather than a quick edit.

Why models are hard to inspect. If knowledge lived in readable rules, you could audit it. Instead you get numbers whose meaning only emerges in combination. This is the root of what people call the black box problem, and it is a genuine open research area rather than a temporary inconvenience.

Why two models trained on the same data differ. Training starts from random values and follows a winding path. Run it twice and you land on two different sets of weights that both work reasonably well.

That last row is the trade being made. You give up the ability to read the logic, and in exchange you get a system that handles messy situations nobody wrote instructions for. For problems like recognizing speech or reading handwriting, where nobody could realistically write the rules, that trade is worth it.

Weights are learned, not designed

One last framing that matters for the rest of this course.

Nobody chooses the weights. No engineer sits down and decides that connection number 4,182,993 should hold 0.37. There are far too many, and no human could reason about what any individual one should be.

Instead the numbers are discovered through a search process. The network makes a guess, finds out how wrong it was, and nudges its weights slightly in a direction that would have made it less wrong. Repeat that an enormous number of times and the weights drift into an arrangement that works.

The next three lessons unpack exactly that loop: how a network measures its own wrongness, how it decides which direction to nudge, and why it needs so much data to get there.

Key Takeaways

• Every connection between neurons carries its own weight, so networks hold anywhere from thousands to billions of them.
• The weights are the knowledge. A trained model contains no facts, rules, or stored examples, only numbers.
• Weights start random and the structure never changes; training adjusts the numbers and nothing else.
• This is why models can be copied as files, cannot be edited like a spreadsheet, and are genuinely hard to explain.
• Nobody designs the weights. They are discovered by a repeated guess-and-correct loop, which the next lessons cover.