Linear Algebra for Machine Learning
Have You Ever Wondered…
- How does Spotify know that if you like Coldplay, you might also like Imagine Dragons?
- How does Instagram apply those fancy filters to your photos in milliseconds?
- How does Netflix predict you’ll rate a movie 4.2 stars before you’ve even watched it?
- How does Google Photos find all pictures of your dog without you tagging them?
Estimated Time: 16-20 hours
Difficulty: Beginner-friendly (we assume you forgot everything)
Prerequisites: Basic Python, willingness to experiment
What You’ll Build: Spotify-style song recommender, Instagram-style filters, Netflix-style rating predictor
Difficulty: Beginner-friendly (we assume you forgot everything)
Prerequisites: Basic Python, willingness to experiment
What You’ll Build: Spotify-style song recommender, Instagram-style filters, Netflix-style rating predictor
📋 Prerequisite Self-Check
📋 Prerequisite Self-Check
Before starting, make sure you can:✅ Python Basics
- Create and manipulate lists:
my_list = [1, 2, 3] - Write simple loops:
for i in range(10) - Define and call functions:
def my_func(x): return x * 2 - Use basic NumPy:
import numpy as np; arr = np.array([1, 2, 3])
- Basic arithmetic (you can use a calculator!)
- Understand coordinates on a graph (x, y)
- Comfortable with the idea that letters can represent numbers
- Previous linear algebra (we start from zero)
- Calculus knowledge
- Matrix manipulation experience
- Any ML/AI background
🧪 Quick Diagnostic: Are You Ready?
🧪 Quick Diagnostic: Are You Ready?
Try these quick checks to gauge your readiness:Python Check (can you read this code?):Math Check (can you solve this?):
If point A is at (2, 3) and point B is at (5, 7), what’s the distance between them?Remediation Paths:
| Gap Identified | Recommended Action |
|---|---|
| Python syntax | Python Crash Course - 4-6 hours |
| NumPy basics | NumPy section of Python course - 1-2 hours |
| Coordinate geometry | We cover it in Module 1! Just proceed. |
| Graph reading | YouTube: “Reading graphs basics” - 30 min |
The “Aha!” Moment: Everything is a List of Numbers
Here’s the secret that unlocks all of machine learning: Anything can be turned into a list of numbers. And once it’s numbers, math can work magic.Your Favorite Song → Numbers
Your Face → Numbers
A Netflix Movie → Numbers
- Compare things (how similar are two songs?)
- Transform things (apply a filter to a photo)
- Find patterns (what do users who liked X also like?)
- Compress things (store a 10MB image in 100KB)
🔗 ML Connection: This “everything is numbers” insight is the foundation of ALL machine learning:
Every module in this course connects directly to these ML applications!
| ML Concept | Linear Algebra Foundation |
|---|---|
| Word Embeddings (GPT, BERT) | Words → vectors of 768+ numbers |
| Neural Network Layers | Matrix multiplication transforms |
| Attention Mechanism | Dot products measure relevance |
| Image Recognition | Pixels → feature vectors → classification |
| Recommendation Systems | Users & items as vectors in shared space |
Who Uses This (Companies & Salaries)
OpenAI
GPT-4 does 100+ trillion matrix operations per prompt. Every AI breakthrough is linear algebra at scale.
Pixar/Disney
Every frame of Toy Story involves millions of matrix transformations for 3D rendering.
Google Search
PageRank uses eigenvalues to rank websites. It’s why Google won the search wars.
| Role | How They Use Linear Algebra | Median Salary |
|---|---|---|
| ML Engineer | Neural network weights, transformations, embeddings | $175K |
| Data Scientist | PCA, clustering, recommendation systems | $150K |
| Graphics Engineer | 3D transformations, shaders, physics | $180K |
| Quantitative Analyst | Portfolio optimization, risk modeling | $250K+ |
| Robotics Engineer | Kinematics, sensor fusion, SLAM | $165K |
Mathematical Notation Quick Reference
Before we dive in, here’s a cheat sheet of the notation you’ll encounter. Don’t memorize it — just come back here when you see something unfamiliar.Vectors
Vectors
| Symbol | Meaning | Example |
|---|---|---|
| or | A vector (bold or arrow) | |
| The -th element of vector | ||
| Length (magnitude) of vector | ||
| Dot product | ||
| Transpose (row ↔ column) |
Matrices
Matrices
| Symbol | Meaning | Example | ||
|---|---|---|---|---|
| , , | Matrices (capital letters) | |||
| or | Element at row , column | |||
| Transpose (flip rows/columns) | ||||
| Inverse of matrix | ||||
| Identity matrix | ||||
| or $ | A | $ | Determinant |
Operations & Summations
Operations & Summations
| Symbol | Meaning | Example |
|---|---|---|
| Sum from to | ||
| Product from to | ||
| Real numbers | means is a real number | |
| -dimensional real space | is a 3D vector |
Special Notation
Special Notation
| Symbol | Meaning | ML Context |
|---|---|---|
| (lambda) | Eigenvalue | How much a direction stretches |
| (sigma) | Singular value | Importance of a pattern in SVD |
| (nabla/del) | Gradient operator | Direction of steepest change |
| (theta) | Model parameters | Weights in neural networks |
| Approximately equal |
Quick Math Examples
Vector Addition — Add component by component: Scalar Multiplication — Multiply each component: Dot Product — Multiply corresponding elements and sum: Matrix × Vector — Each output is a dot product:🚀 Going Deeper: For Advanced Learners
🚀 Going Deeper: For Advanced Learners
Want more mathematical rigor? Each module includes optional “Going Deeper” sections that cover:
These sections are OPTIONAL. You can build all the projects and understand ML applications without them. They’re for learners who:
| Module | Advanced Topic | Why It Matters |
|---|---|---|
| Vectors | Vector spaces, basis, span | Understand why neural network layers work |
| Matrices | Linear transformations, rank | Debug dimensionality issues in ML models |
| Eigenvalues | Spectral theorem, diagonalization | Optimize PCA computation, understand graph neural networks |
| SVD | Matrix approximation theory, Eckart-Young | Why truncated SVD is optimal for compression |
- Have a math/physics background and want the formal treatment
- Plan to pursue ML research or read academic papers
- Are simply curious about the “why” behind the formulas
- Linear Algebra Done Right by Sheldon Axler (rigorous but readable)
- Gilbert Strang’s MIT lectures on YouTube (free, excellent)
- Mathematics for Machine Learning textbook (free PDF at mml-book.github.io)
What You’ll Actually Learn (And Why You’ll Care)
Module 1: Vectors
Module 1: Vectors
Real-World Examples You Already Know:
- GPS Navigation: Your location is a vector
[latitude, longitude]. Distance between two places? Vector math. - Fitness Trackers: Your daily stats
[steps, calories, heart_rate, sleep_hours]— that’s a vector describing your day. - Job Matching: LinkedIn represents you as
[skills, experience, education, location]and finds similar candidates. - Dating Apps: Tinder/Hinge match you based on preference vectors. Similar vectors = potential match.
Module 2: Matrices
Module 2: Matrices
Real-World Examples You Already Know:
- Photo Editing: Every Instagram filter is a matrix multiplication. Brightness, contrast, blur — all matrix operations.
- Video Games: When you rotate your character, move the camera, or zoom in — that’s matrix math happening 60 times per second.
- Spreadsheets: Excel pivot tables, VLOOKUP across sheets — you’re doing matrix operations without knowing it.
- Maps/GPS: Transforming GPS coordinates to screen pixels involves matrix multiplication.
Module 3: Eigenvalues & PCA
Module 3: Eigenvalues & PCA
Real-World Examples You Already Know:
- Surveys: 50 questions reduce to 3-4 “personality types” — that’s PCA finding the key dimensions.
- Stock Market: Hundreds of stocks move together because of 5-10 hidden factors (economy, interest rates, oil prices).
- Customer Segments: Millions of customers cluster into 5-6 types based on purchasing patterns.
- Compression: JPEG images keep 90% quality with 10% file size by keeping only the important eigenvectors.
Module 4: SVD & Recommendations
Module 4: SVD & Recommendations
Real-World Examples You Already Know:
- “Customers who bought X also bought Y”: Amazon uses matrix factorization to find these patterns.
- YouTube Recommendations: “Because you watched X” — they decomposed your viewing history.
- Spell Check: “Did you mean…?” often uses SVD to find similar words.
- Fraud Detection: Normal transactions form patterns; fraud breaks those patterns.
Your Learning Journey
1
Week 1-2: Vectors
Learn to see everything as vectors. Build a song/image similarity search engine.
2
Week 2-3: Matrices
Master transformations. Build Instagram-style photo filters from scratch.
3
Week 3-4: Eigenvalues & PCA
Find hidden patterns. Compress images, reduce dimensions, and understand what your data really contains.
4
Week 4-5: SVD & Recommendations
The crown jewel. Build a Netflix-style recommendation engine that predicts ratings.
Why Most Math Courses Fail (And How This One’s Different)
- Traditional Course
- This Course
- Definition of a vector space
- Axioms of vector addition
- Proof of linear independence
- Abstract theorem
- “Exercise left to the reader”
- Student falls asleep
Prerequisites (Honestly, Not Much)
What You Need:- Basic Python: Variables, lists, loops, functions
- Willingness to experiment: Run code, break things, learn
- Curiosity: Wonder how apps work under the hood
- Previous linear algebra knowledge (we start from scratch)
- Mathematical proofs (we focus on intuition and code)
- Perfect grades in math (many engineers struggle with math — that’s okay!)
Setup (5 Minutes)
🎮 Interactive Visualization Tools
We’ve designed this course to be highly visual. Use these tools alongside the course:3Blue1Brown: Essence of Linear Algebra
The BEST visual introduction to linear algebra. Watch these alongside our modules for deeper geometric intuition.
GeoGebra Vector Playground
Drag vectors, see dot products visually, explore transformations. Use this when working through Module 1-2.
Desmos Matrix Calculator
Enter matrices, see their effects on vectors, visualize eigenvalues. Perfect for Module 3-4.
Immersive Linear Algebra
An entire free textbook with interactive 3D visualizations built in. Great supplementary resource.
🔗 When to Use These Tools:
- Module 1 (Vectors): GeoGebra to visualize addition, dot products
- Module 2 (Matrices): Desmos to see transformations on 2D shapes
- Module 3 (Eigenvalues): 3Blue1Brown video + Desmos visualization
- Module 4 (PCA/SVD): Our built-in interactive widgets
The Projects You’ll Build
By the end of this course, you’ll have a portfolio of real, working projects:Song Recommender
Find similar songs using vector similarity. Input: a song you like. Output: 10 songs you’ll probably love.
Photo Filter App
Apply blur, sharpen, edge detection, and custom effects using matrix operations.
Image Compressor
Compress images to 10% of their size while keeping them recognizable. Understand how JPEG works.
Movie Recommender
Predict user ratings for movies they haven’t seen. The actual technique Netflix uses.
Quick Taste: Vector Similarity in Action
Before we dive deep, let’s see the magic in action. This is what you’ll fully understand by the end of Module 1:By the End of This Course
You will: ✅ See vectors and matrices everywhere (in apps, in data, in neural networks)✅ Build 4 portfolio-worthy ML projects from scratch
✅ Read ML papers and actually understand the notation
✅ Debug ML models because you understand what’s happening inside
✅ Explain to others why linear algebra powers AI Most importantly: You’ll stop being scared of math in ML papers. When you see: You’ll think: “Oh, that’s just transforming a vector with a matrix. Like applying a filter to an image.”
Ready?
Let’s stop talking and start building. The next module introduces vectors by asking a simple question: “How does Spotify know what song to play next?”Next: Vectors — The Language of Similarity
Learn what vectors really are, why everything is a vector, and how to measure similarity between any two things.