Cost Functions and Gradient Descent
Now we know linear regression finds the best line. But how does it find it? The answer involves two key concepts: cost functions and gradient descent.
The Cost Function
A cost function measures how wrong your model is. The lower the cost, the better the model.
Why Squared Error?
Visualizing the Cost Function
Think of the cost function as a landscape. We want to find the lowest point.
Gradient Descent: Finding the Minimum
Gradient descent is an optimization algorithm that finds the minimum of the cost function step by step.
The Learning Rate
The learning rate controls how big each step is.
Gradient Descent Intuition
Putting It Together
Key Takeaways
- Cost function measures how wrong the model is (MSE for regression)
- Gradient descent finds parameters that minimize the cost
- The gradient tells us which direction reduces cost
- Learning rate controls step size (too small = slow, too large = unstable)
- Training = iteratively updating parameters until cost is minimized
- This is how neural networks and many ML algorithms learn!
Next, we'll implement linear regression with sklearn and build a real model!
Quiz
Up next · Lesson 13 of 35
Building Your First Linear Regression Model
2 min read
12 of 35 lessons done
Questions & Answers
Public questions about this lesson. The AI Tutor answers right away in your language, the FreeAcademy team checks answers, and every learner can read them.

