Velamuvi AIAcademy

Lesson 11 of 16 · Transformations

Rank and Determinant

Understand the determinant as how much a matrix scales area, what a zero or negative determinant means, linear independence, and the rank of a matrix, with interactive pictures and NumPy.

  • 9 min read
  • Intermediate
  • Reviewed
In this lesson you will learn
  • What the determinant measures, as a picture
  • What a zero or negative determinant means
  • What linearly dependent and independent vectors are
  • What the rank of a matrix tells you
  • How to find both in NumPy, and why they matter for data

How much does a matrix stretch area?

In the Transformations lesson, the orange unit square changed shape as you moved the sliders. Sometimes it grew, sometimes it shrank, and sometimes it collapsed into a line. One number captures that.

For a 2 × 2 matrix, it is the ad - bc you met in the inverse formula:

det [ a  b ]  =  ad - bc
    [ c  d ]

Move the sliders and watch the determinant and the orange square together:

îĵ
[[2, 1], [1, 2]]î lands on (2, 1), ĵ lands on (1, 2)determinant: 3
The orange area is the unit square after the transformation. Its area always equals the size of the determinant.

What the value tells you

DeterminantWhat happens to space
bigger than 1areas grow
between 0 and 1areas shrink
exactly 0space is squashed flat onto a line or a point: information is lost
negativespace is also flipped, like a mirror image (the size of the number still gives the area scale)
Try it yourselfPython
import numpy as np

examples = {
    "stretch":   [[3, 0], [0, 2]],
    "shear":     [[1, 1], [0, 1]],
    "flip":      [[-1, 0], [0, 1]],
    "squash":    [[1, 2], [2, 4]],
}
for name, M in examples.items():
    print(f"{name:8} det = {round(np.linalg.det(np.array(M)), 6) + 0.0}")

The shear has determinant 1: it slants the square but keeps its area. The squash has determinant 0: it is singular, so it has no inverse. That is why a zero determinant means "no inverse": a transformation that flattens space cannot be undone.

Dependent and independent vectors

Why would a matrix squash space? Because its columns point along the same line.

In [[1, 2], [2, 4]], the second column [2, 4] is twice the first [1, 2]. Both columns lie on one line, so the matrix can only reach that line.

column 1column 2independent
Columns 1 and 2 lie on the same line: they are dependent. The third vector points in a new direction.

Rank: how many real directions?

Try it yourselfPython
import numpy as np

full = np.array([[2, 1],
                 [1, 2]])
squashed = np.array([[1, 2],
                     [2, 4]])
zero = np.zeros((2, 2))

print("rank of full:    ", np.linalg.matrix_rank(full))
print("rank of squashed:", np.linalg.matrix_rank(squashed))
print("rank of zero:    ", np.linalg.matrix_rank(zero))

Rank 2 fills the plane, rank 1 squashes it onto a line, and rank 0 crushes everything to the origin.

Rank in a dataset

Rank becomes really useful for data. If one feature is just a combination of others, it adds no information, and the data matrix has a lower rank than its number of columns.

Try it yourselfPython
import numpy as np

# Columns: price in rupees, price in thousands of rupees, number of rooms
X = np.array([
    [5000000, 5000, 3],
    [3200000, 3200, 2],
    [7800000, 7800, 4],
    [4100000, 4100, 3],
])

print("columns:", X.shape[1])
print("rank:   ", np.linalg.matrix_rank(X))

Three columns, but rank 2: the second column is the first divided by 1,000. One feature is redundant. Some models, such as plain linear regression, become unstable or fail with redundant features like this.

Common mistakes

Practice exercise

Use the determinant and rank to decide which matrices can be undone.

Your solutionPython
import numpy as np

# 1. Write det_2x2(M) using ad - bc, and print it for A and B below.
# 2. Print the rank of each matrix with np.linalg.matrix_rank.
# 3. Print which of them has an inverse.
A = np.array([[4, 2],
              [1, 3]])
B = np.array([[3, 6],
              [1, 2]])

# Output must be exactly:
# det A: 10
# det B: 0
# rank A: 2
# rank B: 1
# A has an inverse: True
# B has an inverse: False
Knowledge check

A 2 × 2 matrix has determinant 0. What does it do to the plane?

Summary

  • The determinant is the factor by which a matrix scales area or volume. For 2 × 2 it is ad - bc.
  • 0 means space is squashed and the matrix has no inverse. A negative value means space is also flipped.
  • Vectors are linearly dependent when one can be built from the others, and independent when each adds a new direction.
  • The rank is the number of independent columns: how many dimensions the output really fills.
  • A data matrix with lower rank than its number of columns has redundant features.

Next: projections, or how to find the shadow of one vector on another.

Finished reading? Tick it off to track your progress.