Lesson 11 of 16 · Transformations
Rank and Determinant
Understand the determinant as how much a matrix scales area, what a zero or negative determinant means, linear independence, and the rank of a matrix, with interactive pictures and NumPy.
- 9 min read
- Intermediate
- Reviewed
- What the determinant measures, as a picture
- What a zero or negative determinant means
- What linearly dependent and independent vectors are
- What the rank of a matrix tells you
- How to find both in NumPy, and why they matter for data
How much does a matrix stretch area?
In the Transformations lesson, the orange unit square changed shape as you moved the sliders. Sometimes it grew, sometimes it shrank, and sometimes it collapsed into a line. One number captures that.
For a 2 × 2 matrix, it is the ad - bc you met in the inverse formula:
det [ a b ] = ad - bc
[ c d ]Move the sliders and watch the determinant and the orange square together:
What the value tells you
| Determinant | What happens to space |
|---|---|
| bigger than 1 | areas grow |
| between 0 and 1 | areas shrink |
| exactly 0 | space is squashed flat onto a line or a point: information is lost |
| negative | space is also flipped, like a mirror image (the size of the number still gives the area scale) |
import numpy as np
examples = {
"stretch": [[3, 0], [0, 2]],
"shear": [[1, 1], [0, 1]],
"flip": [[-1, 0], [0, 1]],
"squash": [[1, 2], [2, 4]],
}
for name, M in examples.items():
print(f"{name:8} det = {round(np.linalg.det(np.array(M)), 6) + 0.0}")The shear has determinant 1: it slants the square but keeps its area. The squash has determinant 0: it is singular, so it has no inverse. That is why a zero determinant means "no inverse": a transformation that flattens space cannot be undone.
Dependent and independent vectors
Why would a matrix squash space? Because its columns point along the same line.
In [[1, 2], [2, 4]], the second column [2, 4] is twice the first [1, 2]. Both columns lie on one line, so the matrix can only reach that line.
Rank: how many real directions?
import numpy as np
full = np.array([[2, 1],
[1, 2]])
squashed = np.array([[1, 2],
[2, 4]])
zero = np.zeros((2, 2))
print("rank of full: ", np.linalg.matrix_rank(full))
print("rank of squashed:", np.linalg.matrix_rank(squashed))
print("rank of zero: ", np.linalg.matrix_rank(zero))Rank 2 fills the plane, rank 1 squashes it onto a line, and rank 0 crushes everything to the origin.
Rank in a dataset
Rank becomes really useful for data. If one feature is just a combination of others, it adds no information, and the data matrix has a lower rank than its number of columns.
import numpy as np
# Columns: price in rupees, price in thousands of rupees, number of rooms
X = np.array([
[5000000, 5000, 3],
[3200000, 3200, 2],
[7800000, 7800, 4],
[4100000, 4100, 3],
])
print("columns:", X.shape[1])
print("rank: ", np.linalg.matrix_rank(X))Three columns, but rank 2: the second column is the first divided by 1,000. One feature is redundant. Some models, such as plain linear regression, become unstable or fail with redundant features like this.
Common mistakes
Use the determinant and rank to decide which matrices can be undone.
import numpy as np
# 1. Write det_2x2(M) using ad - bc, and print it for A and B below.
# 2. Print the rank of each matrix with np.linalg.matrix_rank.
# 3. Print which of them has an inverse.
A = np.array([[4, 2],
[1, 3]])
B = np.array([[3, 6],
[1, 2]])
# Output must be exactly:
# det A: 10
# det B: 0
# rank A: 2
# rank B: 1
# A has an inverse: True
# B has an inverse: False
A 2 × 2 matrix has determinant 0. What does it do to the plane?
Summary
- The determinant is the factor by which a matrix scales area or volume. For 2 × 2 it is ad - bc.
- 0 means space is squashed and the matrix has no inverse. A negative value means space is also flipped.
- Vectors are linearly dependent when one can be built from the others, and independent when each adds a new direction.
- The rank is the number of independent columns: how many dimensions the output really fills.
- A data matrix with lower rank than its number of columns has redundant features.
Next: projections, or how to find the shadow of one vector on another.
Finished reading? Tick it off to track your progress.