Lesson 8 of 16 · Matrices
Transpose
Learn the transpose: flipping rows into columns, writing dot products as xᵀy, symmetric matrices, the (AB)ᵀ = BᵀAᵀ rule, and the XᵀX matrix behind regression and covariance.
- 8 min read
- Intermediate
- Reviewed
- What the transpose of a matrix is
- How the transpose changes a matrix's shape
- How a dot product is written with a transpose
- What symmetric matrices are
- Why XᵀX appears all over machine learning
Flip rows into columns
A = [ 1 2 3 ] Aᵀ = [ 1 4 ]
[ 4 5 6 ] [ 2 5 ]
[ 3 6 ]Picture it as flipping the matrix over its main diagonal. The first row [1, 2, 3] becomes the first column.
import numpy as np
A = np.array([[1, 2, 3],
[4, 5, 6]])
print(A.T)
print("Shape of A:", A.shape, " shape of A.T:", A.T.shape)A matrix of shape (m × n) becomes (n × m). Transposing twice brings you back: (Aᵀ)ᵀ = A.
Fixing shapes with a transpose
The transpose is the everyday tool for making shapes fit. Suppose you have 3 examples with 2 features, and you want each feature's dot product with a vector of 3 weights, one per example:
import numpy as np
X = np.array([[1, 2],
[3, 4],
[5, 6]]) # shape (3, 2)
v = np.array([1, 0, 1]) # shape (3,)
# X @ v fails: (3, 2) @ (3,) -> inner sizes 2 and 3 do not match
print(X.T @ v) # (2, 3) @ (3,) -> (2,)The dot product as a transpose
Maths books treat vectors as columns. A dot product is then written as a row times a column:
x · y = xᵀyxᵀ turns the column x into a row, and (1 × n) @ (n × 1) gives a 1 × 1 result: a single number. You will see xᵀy, wᵀx and θᵀx constantly in machine learning papers. They all mean "dot product".
Symmetric matrices
import numpy as np
distances = np.array([
[0, 5, 9],
[5, 0, 4],
[9, 4, 0],
]) # distance from city i to city j
print(np.array_equal(distances, distances.T))A table of distances between cities is symmetric, because the distance from A to B equals the distance from B to A. Symmetric matrices have especially nice properties, which you will meet in the eigenvector lesson.
The transpose of a product
Like taking off socks and shoes: what went on last comes off first.
import numpy as np A = np.array([[1, 2], [3, 4]]) B = np.array([[0, 1], [5, 2]]) print(np.array_equal((A @ B).T, B.T @ A.T)) print(np.array_equal((A @ B).T, A.T @ B.T))
XᵀX: the most famous product
For a data matrix X with n examples and d features, XᵀX is a d × d matrix that compares every feature with every other feature.
import numpy as np
X = np.array([[1, 2],
[2, 1],
[3, 4]]) # 3 examples, 2 features
G = X.T @ X
print(G)
print("Shape:", G.shape, " symmetric:", np.array_equal(G, G.T))Entry (i, j) of XᵀX is the dot product of feature column i with feature column j. It is always square and symmetric.
Common mistakes
Build a film-by-film comparison matrix from user ratings with a transpose.
import numpy as np
# Ratings by 3 users (rows) for 2 films (columns)
R = np.array([[5, 3],
[4, 4],
[1, 5]])
# 1. Print the transpose of R and its shape.
# 2. Compute F = Rᵀ @ R, a film-by-film matrix, and print it.
# 3. Print whether F is symmetric.
# Output must be exactly:
# [[5 4 1]
# [3 4 5]]
# (2, 3)
# [[42 36]
# [36 50]]
# Symmetric: True
A has shape (4 × 7). What is the shape of Aᵀ A?
Summary
- The transpose Aᵀ swaps rows and columns, turning shape (m × n) into (n × m).
- A dot product is written xᵀy, and a linear model as wᵀx + b.
- A symmetric matrix equals its transpose.
- (AB)ᵀ = BᵀAᵀ: reverse the order.
- XᵀX compares every feature with every other, and leads to the covariance matrix.
You have finished the Matrices module. Next: seeing matrices as transformations of space.
Finished reading? Tick it off to track your progress.