Training, Alignment, and Inference: How Models Are Built and Run

You now understand tokens, next-token prediction, context windows, parameters, and Mixture of Experts. This final lesson connects them into the full life story of a model: how it is made (training), how it is shaped into an assistant (alignment), and how it answers you (inference). These are two very different activities with very different costs, and confusing them is behind a lot of misunderstandings about AI, including why a "free" chatbot still costs the company real money every time you press enter.

No code here, just a clear picture of where models come from and why running them is not free.

What You'll Learn

• The three stages that turn raw text into a helpful assistant
• The difference between training and inference
• Why serving a model (inference) costs real money every single time
• A simple, honest intuition for the KV cache and why long chats get pricier

Making a model: three stages

Building a modern assistant happens in stages, each one shaping the model further.

1. Pre-training. The model reads an enormous amount of text and does one thing billions of times: predict the next token, check what the real next token was, and nudge its parameters to do better. This is where the model absorbs grammar, facts, reasoning patterns, and world knowledge. It is by far the most expensive and time-consuming stage, running on large clusters of specialized chips for a long time. The result is a "base model" that is knowledgeable but raw: it completes text rather than helpfully answering questions.

2. Fine-tuning (instruction tuning). The base model is then trained on curated examples of instructions paired with good responses. This teaches it to behave like an assistant: to answer a question rather than continue it, to follow formats, to stay on task. It is far cheaper than pre-training because it uses much less, much higher-quality data.

3. Alignment with human feedback. Finally, humans (and increasingly other AI systems) rank the model's responses, and the model is tuned to prefer the kinds of answers people rate as helpful, honest, and safe. The best-known version of this is RLHF, reinforcement learning from human feedback. This stage shapes tone, refusals, and the "personality" you experience, and it is a big reason two models trained on similar data can feel quite different to talk to.

The key insight: the knowledge comes from pre-training; the helpfulness comes from fine-tuning and alignment. A model is not just what it read, but how it was subsequently shaped.

Training vs inference: two different worlds

Once the model is made, using it is a separate activity called inference. The distinction matters:

A subtle but important point: during inference, the model does not learn. Its parameters are frozen. When a chatbot seems to "remember" something you told it, that is the context window at work (from lesson three), not the model updating itself. Your conversation does not change the model's dials. Learning from user interactions, if it happens at all, is a separate, deliberate future training run, not something that occurs live in your chat.

Why inference costs real money

Here is the part that reshapes how you think about "free" AI. Every time you send a prompt, the model runs on expensive hardware (specialized AI chips, usually called GPUs) in a data center, drawing significant power to produce your answer token by token. That cost is real and it recurs on every request. Pre-training was a giant one-time bill; inference is a meter that runs forever.

This explains a lot of what you see in the market:

• Free tiers have usage limits, because each message has a genuine marginal cost.
• Faster or "smarter" models often cost more, because they use more compute per token.
• Providers charge per token (from lesson one) precisely because tokens are the unit of work being metered.
• Shorter prompts and answers are not just tidy; they are literally cheaper to serve.

Understanding this makes you a more realistic AI user. "Why can't it just be unlimited and free?" has a concrete answer: someone is paying for a data center to run for every token you generate.

A simple intuition for the KV cache

One last piece explains why long conversations get slower and pricier, beyond just having more tokens. As the model generates each new token, it has to consider all the previous tokens in the context. Redoing that work from scratch for every new token would be painfully slow. So the system keeps a running scratchpad of the useful intermediate results for the tokens seen so far, and reuses it. This scratchpad is commonly called the KV cache.

The intuition you need is simple: the KV cache lets the model avoid re-processing the whole conversation for each new token, which makes generation practical. But that scratchpad grows with the length of the conversation, and it takes up scarce, expensive memory on the AI chip. So a very long chat is not just more tokens; it is a bigger, memory-hungry scratchpad that costs more to maintain. This is another concrete reason the habits from lesson three (fresh chats, trimming context) save money and keep things fast.

You do not need to manage the KV cache; it is invisible and automatic. It is simply the reason "longer context" is never truly free even when it fits.

Wrapping up the course

Step back and look at the whole picture you have built:

• Text becomes tokens (lesson 1).
• The model predicts the next token repeatedly, sampling with a temperature (lesson 2).
• It can only see what fits in its context window (lesson 3).
• Its knowledge lives in billions of parameters (lesson 4).
• Mixture of Experts lets it be huge yet fast by activating only a slice per token (lesson 5).
• It was built by pre-training, fine-tuning, and alignment, and serving it via inference costs real money every time (this lesson).

That is a genuinely accurate mental model of how modern AI assistants work, without a single line of math. It will make you a sharper, more skeptical, and more effective user of every AI tool you touch. To apply this in day-to-day tools, the AI Essentials course is a natural next step.

Key Takeaways

• Models are built in three stages: pre-training (knowledge), fine-tuning (instruction-following), and alignment/RLHF (helpful, safe behavior).
• Training builds the model once at huge cost; inference is using it, and the model's parameters are frozen during inference.
• A model does not learn from your chat live; apparent memory comes from the context window, not parameter updates.
• Inference costs real money on every request because it runs on power-hungry AI chips, which is why free tiers have limits and providers meter tokens.
• The KV cache is a scratchpad that speeds up generation but grows with conversation length, so long chats cost more in both tokens and memory.