Choosing the Right Model (Sizes and Quantization)

When you open a model catalog you will see dozens of names, each offered in several versions with codes like "Q4" and "Q8." It can feel overwhelming. This lesson turns that wall of options into two simple decisions: how big a model to pick, and how compressed a version to grab. Get these two right and everything runs smoothly.

You already met the first idea, model size, in the laptop lesson. Here we tie it together with the compression codes so you can pick with confidence.

What You'll Learn

• What model families are and a few good beginner choices
• What "quantization" (the Q codes) means in plain language
• Which compression level to pick for everyday use
• A simple recipe for choosing any model in under a minute

Model Families in Plain Terms

Most open models come from a handful of "families," each made by a different organization and updated over time. You do not need to follow the news; you just need a few dependable names to search for. Common, well-regarded families include:

• Llama (from Meta) - a strong all-rounder, popular for general chat and writing
• Gemma (from Google) - capable and comes in small, efficient sizes
• Mistral - known for being efficient and good at its size
• Phi (from Microsoft) - small models that punch above their weight
• Qwen and gpt-oss - other widely used open options

Any of these makes a fine starting point. Within a family you pick a size (the "B" number from earlier), such as an 8B version. Newer versions of a family are generally better than older ones, so if you see a higher version number, it is usually a safe upgrade.

Quantization, Explained Simply

Here is the one genuinely new idea in this lesson. A model is a huge collection of numbers. In its original form, each number is stored very precisely, which makes the file enormous and memory-hungry. Quantization is a compression trick that stores those numbers with less precision, shrinking the file dramatically while keeping almost all of the quality.

A useful analogy: think of a high-resolution photo. You can save it as a smaller file that looks nearly identical to the eye but takes far less space. Quantization does the same for a model. The "Q" codes tell you how much it has been compressed:

The headline for beginners: a Q4 version (often written Q4_K_M) is the recommended default. It roughly quarters the memory needed compared to the full model, runs fast, and the quality drop is small enough that most people never notice it in everyday tasks. This is why both Ollama and LM Studio tend to give you a compressed version by default.

Putting It Together: The One-Minute Recipe

Choosing a model is really just these steps:

And when you are staring at versions and unsure which compression to grab:

A Few Practical Tips

• Start smaller than you think. A fast 7-8B model you actually enjoy using beats a sluggish 14B model you avoid. You can always upgrade.
• Match the task to the size. Quick drafting and simple questions run great on small models. Save the bigger models for when you want more careful reasoning.
• Keep two or three around. A tiny fast model for quick jobs and a larger one for harder tasks is a comfortable setup. Delete any you do not use to reclaim disk space.
• Ignore exotic options for now. You will see other formats and codes. As a beginner, family plus size plus Q4 covers you completely.

Key Takeaways

• Pick from a well-known family (Llama, Gemma, Mistral, Phi, Qwen, gpt-oss); newer versions are generally better.
• Quantization is compression: it shrinks a model's file and memory needs while keeping almost all the quality, like saving a photo at a smaller size.
• The Q codes show compression level. A Q4 version (Q4_K_M) is the recommended everyday default for its balance of size, speed, and quality.
• The recipe: choose a family, choose a size that fits your RAM, choose a Q4 version, then download and test.
• If it runs slowly, step down a size or to a more compressed version. Start smaller and upgrade later.