The Parts of a Computer That Matter for AI

When people talk about AI, they usually talk about models and apps. But every AI answer is, in the end, a physical process: electricity moving through chips, numbers moving between memory and processors. Understanding that physical layer explains a lot, from why AI needs special chips to why some models run on a phone and others need a building full of servers.

You do not need to know electronics for this. You only need a clear picture of three parts of a computer and how data moves between them.

What You'll Learn

• What a processor, memory, and storage each do
• Why the speed of moving data matters as much as the speed of calculating
• How the memory hierarchy works, from slow and large to fast and small
• What actually happens, physically, when an AI model answers a question

Three parts that matter

Every computer, from a phone to a data center server, has the same three basic parts.

The processor does the calculations. It takes numbers, performs operations on them (add, multiply, compare), and produces new numbers. The main general-purpose processor is the CPU. Many devices also have extra processors built for special jobs, such as graphics or AI, which later lessons cover.

Memory (often called RAM) holds the data the processor is working on right now. It is fast, but it is cleared when the power goes off.

Storage (a solid-state drive or hard drive) keeps data permanently: your files, apps, and AI model files. It holds far more than memory, but it is much slower to read from.

Moving data is the hidden cost

A useful way to picture this is a kitchen. Storage is the pantry down the hall: it holds everything, but fetching from it is slow. Memory is the counter next to you: less space, but close at hand. The processor is the chef's hands: very fast, but it can only work on what is already on the counter.

A fast chef with a slow path to the counter still cooks slowly. The same is true for computers. A processor can only calculate as fast as data arrives. For many modern workloads, and AI in particular, moving data is the bottleneck, not the calculation itself. Keep this idea in mind. It comes back in every lesson of this course.

The memory hierarchy

Computers solve the speed problem with layers of memory, from small and very fast to large and slow:

The closer data is to the processor, the faster it can be used, and the less of it fits. A lot of hardware design, including AI chips, is about keeping the right data in the fastest layer at the right time.

What happens when a model answers

A trained AI model is, physically, a very large collection of numbers called weights (also called parameters). These numbers are what the model learned during training. How Neural Networks Actually Work explains what they mean.

When you ask a model a question, roughly this happens:

1. Load. The model's weights are read from storage into memory. This happens once, when the model starts, not for every question.
2. Compute. Your question is turned into numbers. The processor combines those numbers with the model's weights through a huge number of multiplications and additions.
3. Repeat. A language model produces its answer one token at a time. For each new token, it runs through the weights again.
4. Return. The resulting numbers are turned back into text and shown to you.

Step 3 is the one to notice. For every token the model writes, a large share of its weights has to move from memory to the processor again. A long answer means doing this hundreds or thousands of times. That is why both how fast the processor calculates and how fast memory can feed it decide how quickly an AI answers.

Training vs using a model

Two very different hardware jobs appear throughout this course:

• Training creates the model. It processes enormous amounts of data, adjusting the weights over and over. It happens once per model and needs huge numbers of chips working together for a long time.
• Inference is using the finished model to answer questions. Each answer is much smaller work, but it happens constantly, for every user, every time.

Both rely on the same basic operations. The next lesson explains why a particular kind of chip is so good at them.

Key Takeaways

• Computers have a processor (calculates), memory (holds current work), and storage (keeps files).
• A processor can only work as fast as data reaches it. For AI, moving data is often the real bottleneck.
• The memory hierarchy goes from tiny and fast (cache) to large and slow (storage).
• A model is a huge set of weights. Answering means combining your input with those weights through many multiplications.
• Training builds the model once with massive computing. Inference is using it, one answer at a time, constantly.