The Prediction Game

Starting With Something You Already Know

Forget Python. Forget libraries. Forget math notation. Let’s play a game.

Round 1: The House Price Game

You’re a real estate agent. A client asks: “How much is this house worth?” They give you some info:

Feature	Value
Bedrooms	3
Bathrooms	2
Square Feet	1,500
Age (years)	10
Has Pool	No

What’s your guess?

Your Brain’s Algorithm

Without realizing it, you do this:

Think of similar houses you’ve seen
Remember what they sold for
Adjust based on differences
Make a guess

That’s machine learning. You learned patterns from past data and applied them to new data.

Round 2: Let’s Be More Systematic

What if I told you the average house in your area sells for:

Base price: $200,000
Each bedroom adds about $25,000
Each bathroom adds about $15,000
Each square foot adds about $150

Now you can compute:

Base:                           $200,000
+ 3 bedrooms × $25,000:         + $75,000
+ 2 bathrooms × $15,000:        + $30,000
+ 1,500 sq ft × $150:          + $225,000
                               ----------
Predicted price:                $530,000

You just built your first linear model!

The formula you just used:price = base + (bedrooms × weight1) + (bathrooms × weight2) + (sqft × weight3)Those “weights” (

25k,

15k, $150) are what machine learning learns automatically from data.

Let’s Code It (Still No Libraries!)

# Your first "model" - just a function!
def predict_house_price(bedrooms, bathrooms, sqft):
    base = 200000
    bedroom_value = 25000
    bathroom_value = 15000
    sqft_value = 150
    
    predicted = (
        base + 
        bedrooms * bedroom_value + 
        bathrooms * bathroom_value + 
        sqft * sqft_value
    )
    return predicted

# Test it
house1 = predict_house_price(3, 2, 1500)
print(f"House 1 predicted: ${house1:,}")  # $530,000

house2 = predict_house_price(4, 3, 2200)
print(f"House 2 predicted: ${house2:,}")  # $675,000

The Million Dollar Question

But wait… how did we know those weights?

Why $25,000 per bedroom and not$ 30,000?
Why $150 per sq ft and not$ 200?

We guessed. And our guesses might be wrong. Machine learning answers this: Given a bunch of houses with known prices, can we figure out the best weights automatically?

Real Data, Real Problem

Here’s actual data (simplified):

# Past house sales (our "training data")
houses = [
    # [bedrooms, bathrooms, sqft] -> actual_price
    {"features": [2, 1, 1000], "price": 250000},
    {"features": [3, 2, 1500], "price": 380000},
    {"features": [4, 2, 1800], "price": 450000},
    {"features": [3, 3, 2000], "price": 520000},
    {"features": [5, 4, 3000], "price": 750000},
]

Our goal: Find weights that make our predictions match these actual prices as closely as possible.

Step 1: How Wrong Are We?

If we use our guessed weights, let’s see how we do:

def predict_house_price(features):
    bedrooms, bathrooms, sqft = features
    base = 200000
    return base + bedrooms * 25000 + bathrooms * 15000 + sqft * 150

# Check each house
for house in houses:
    predicted = predict_house_price(house["features"])
    actual = house["price"]
    error = predicted - actual
    print(f"Predicted: ${predicted:,}, Actual: ${actual:,}, Error: ${error:,}")

Output:

Predicted: $430,000, Actual: $250,000, Error: $180,000  (too high!)
Predicted: $530,000, Actual: $380,000, Error: $150,000  (too high!)
Predicted: $595,000, Actual: $450,000, Error: $145,000  (too high!)
Predicted: $620,000, Actual: $520,000, Error: $100,000  (too high!)
Predicted: $825,000, Actual: $750,000, Error: $75,000   (too high!)

We’re consistently too high! Our weights are off.

Step 2: Measure Total “Wrongness”

We need a single number that tells us how wrong we are overall. Simple approach: Sum of all errors

total_error = 0
for house in houses:
    predicted = predict_house_price(house["features"])
    actual = house["price"]
    error = predicted - actual
    total_error += error

print(f"Total error: ${total_error:,}")  # $650,000 too high overall

Problem: What if some errors are positive and some negative? They cancel out! Better approach: Sum of squared errors

total_squared_error = 0
for house in houses:
    predicted = predict_house_price(house["features"])
    actual = house["price"]
    error = predicted - actual
    total_squared_error += error ** 2

print(f"Total squared error: {total_squared_error:,.0f}")

This is called the Loss Function or Cost Function. Lower is better!

Why squared? Think of it like grading a student’s exam:

No negative numbers — errors can’t cancel out (a + $50K overshoot shouldn't "forgive" a -$ 50K undershoot)
Big errors get penalized more — being off by $100K is more than twice as bad as being off by$ 50K. Squaring enforces this: $100K^2 = 10B$ vs $50K^2 = 2.5B$ (a 4x penalty for a 2x error)
Smooth and differentiable — the curve has no sharp corners, so gradient descent can glide smoothly toward the minimum (we’ll need this in Module 2)

There are alternatives — Mean Absolute Error (MAE) treats all errors equally and is more robust to outliers. But MSE is the default starting point because its math is cleaner and it punishes the predictions you’re most embarrassingly wrong about.

Step 3: Try Different Weights

What if we try different values?

def calculate_total_error(base, bed_weight, bath_weight, sqft_weight):
    total_squared_error = 0
    for house in houses:
        bedrooms, bathrooms, sqft = house["features"]
        predicted = base + bedrooms * bed_weight + bathrooms * bath_weight + sqft * sqft_weight
        actual = house["price"]
        error = predicted - actual
        total_squared_error += error ** 2
    return total_squared_error

# Our original guess
error1 = calculate_total_error(200000, 25000, 15000, 150)
print(f"Original weights error: {error1:,.0f}")

# Try lowering everything
error2 = calculate_total_error(100000, 20000, 10000, 100)
print(f"Lower weights error: {error2:,.0f}")

# Try something else
error3 = calculate_total_error(50000, 15000, 25000, 175)
print(f"Alternative weights error: {error3:,.0f}")

The challenge: There are infinite combinations of weights. How do we find the best ones?

The Insight: Systematic Search

What if we:

Start with random weights
Check how wrong we are
Slightly adjust weights
If error goes down, keep the change
Repeat until error stops improving

This is the core idea behind Gradient Descent - which we’ll explore in the next module! Think of it like tuning a radio dial in the dark. Random search is spinning the dial blindly and hoping for a good station. Gradient descent is listening to the static — when it gets quieter, you keep turning that direction.

# A simple (but slow) approach: try lots of combinations
# This is "random search" -- the brute force method.
# It works, but it's like trying every combination on a lock
# instead of listening for the click.
best_error = float('inf')
best_weights = None

import random

for _ in range(10000):  # Try 10,000 random combinations
    base = random.randint(0, 200000)
    bed = random.randint(5000, 50000)
    bath = random.randint(5000, 50000)
    sqft = random.randint(50, 300)
    
    error = calculate_total_error(base, bed, bath, sqft)
    
    if error < best_error:
        best_error = error
        best_weights = (base, bed, bath, sqft)

print(f"Best weights found: {best_weights}")
print(f"Best error: {best_error:,.0f}")
# Note: with 4 weights and infinite possible values,
# random search has astronomically low odds of finding
# the true optimum. We need something smarter.

What You Just Learned

Let’s recap with proper ML terminology:

What You Did	ML Term
Used past house sales	Training Data
Features like bedrooms, sqft	Input Features (X)
The actual price	Target/Label (y)
The weights ( $25k,$ 15k, etc.)	Model Parameters
The prediction formula	Model
How wrong our predictions were	Loss/Error
Sum of squared errors	Loss Function
Trying to minimize error	Training/Optimization

The Mathematical Connection

When you calculated:

price = base + (bedrooms × weight1) + (bathrooms × weight2) + (sqft × weight3)

In math notation, this is:

\hat{y} = w_0 + w_1 x_1 + w_2 x_2 + w_3 x_3

Or in matrix form (from our Linear Algebra course):

\hat{y} = \mathbf{w} \cdot \mathbf{x}

This is a dot product - the same operation you do when calculating weighted grades!

🚀 Mini Projects

Project 1

Build a house price estimator from scratch

Project 2

Create a used car valuation tool

Project 3

Visualize prediction errors and find patterns

Key Takeaways

ML is Pattern Matching

Find patterns in past data, apply to new data

Weights Capture Knowledge

The learned weights encode what matters

Loss Measures Wrongness

Lower loss = better predictions

Training = Minimizing Loss

Find weights that make predictions best match reality

Practice Challenge

Try this on your own:

# New dataset: Car prices
cars = [
    # [age_years, mileage_k, horsepower] -> price
    {"features": [2, 15, 200], "price": 35000},
    {"features": [5, 50, 180], "price": 22000},
    {"features": [1, 8, 250], "price": 45000},
    {"features": [8, 100, 150], "price": 12000},
    {"features": [3, 30, 220], "price": 32000},
]

# Your task:
# 1. Create a predict_car_price function with guessed weights
# 2. Calculate total squared error
# 3. Try different weights and find better ones
# 4. What patterns do you notice? (age and mileage should be negative!)

Solution Hints

Key insight: Unlike houses where more is usually better, for cars:

Older cars are worth less (negative weight for age)
Higher mileage is worth less (negative weight for mileage)
More horsepower is worth more (positive weight)

Try something like:

price = 50000 - (age * 3000) - (mileage * 200) + (horsepower * 100)

Next Up

In the next module, we’ll learn:

How to systematically find the best weights (not just random guessing)
The key insight of gradient descent - following the slope downhill
How this connects to calculus

Continue to Module 2: Learning From Mistakes

Discover gradient descent - the algorithm that powers all modern ML

🔗 Math → ML Connection

What you learned in this module connects to formal ML:

Concept in This Module	Formal ML Term	Where It’s Used
Guessing weights	Model parameters	Every ML model has parameters to learn
Formula: `price = base + weight × feature`	Linear model	Neural network layers, linear regression
Measuring “wrongness”	Loss function	Training any model (MSE, cross-entropy, etc.)
Finding better weights	Optimization	Gradient descent, Adam, SGD
Past data with answers	Training data	Supervised learning

Next module: We’ll replace “random guessing” with a systematic approach called gradient descent - the same algorithm that trains ChatGPT!

🚀 Going Deeper (Optional)

The Mathematics Behind Linear Models

For learners who want the formal treatment:

Matrix Formulation

What we wrote as:

price = base + w1×bedrooms + w2×bathrooms + w3×sqft

Can be written in matrix form as:

\hat{y} = X \mathbf{w}

Where:

$X$ is the feature matrix (each row is a house, each column is a feature)
$\mathbf{w}$ is the weight vector
$\hat{y}$ is the prediction vector

Why Squared Error?

We use squared error (not absolute error) because:

It’s differentiable - we can compute gradients (needed for Module 2)
It penalizes large errors more - a $100K error is worse than two$ 50K errors
It leads to closed-form solutions in linear regression

Closed-Form Solution

For linear regression, there’s actually a formula that gives optimal weights directly:

\mathbf{w}^* = (X^T X)^{-1} X^T y

We’ll derive this in Linear Regression module.

Recommended Resources

3Blue1Brown: Essence of Linear Algebra - Visual intuition
Our Linear Algebra Course - Full treatment

​The Prediction Game

​Starting With Something You Already Know

​Round 1: The House Price Game

​Your Brain’s Algorithm

​Round 2: Let’s Be More Systematic

​Let’s Code It (Still No Libraries!)

​The Million Dollar Question

​Real Data, Real Problem

​Step 1: How Wrong Are We?

​Step 2: Measure Total “Wrongness”

​Step 3: Try Different Weights

​The Insight: Systematic Search

​What You Just Learned

​The Mathematical Connection

​🚀 Mini Projects