Velamuvi AIAcademy

Lesson 8 of 16 · Matrices

Transpose

Learn the transpose: flipping rows into columns, writing dot products as xᵀy, symmetric matrices, the (AB)ᵀ = BᵀAᵀ rule, and the XᵀX matrix behind regression and covariance.

  • 8 min read
  • Intermediate
  • Reviewed
In this lesson you will learn
  • What the transpose of a matrix is
  • How the transpose changes a matrix's shape
  • How a dot product is written with a transpose
  • What symmetric matrices are
  • Why XᵀX appears all over machine learning

Flip rows into columns

A = [ 1  2  3 ]          Aᵀ = [ 1  4 ]
    [ 4  5  6 ]               [ 2  5 ]
                              [ 3  6 ]

Picture it as flipping the matrix over its main diagonal. The first row [1, 2, 3] becomes the first column.

Try it yourselfPython
import numpy as np

A = np.array([[1, 2, 3],
              [4, 5, 6]])

print(A.T)
print("Shape of A:", A.shape, " shape of A.T:", A.T.shape)

A matrix of shape (m × n) becomes (n × m). Transposing twice brings you back: (Aᵀ)ᵀ = A.

Fixing shapes with a transpose

The transpose is the everyday tool for making shapes fit. Suppose you have 3 examples with 2 features, and you want each feature's dot product with a vector of 3 weights, one per example:

Try it yourselfPython
import numpy as np

X = np.array([[1, 2],
              [3, 4],
              [5, 6]])          # shape (3, 2)
v = np.array([1, 0, 1])          # shape (3,)

# X @ v fails: (3, 2) @ (3,) -> inner sizes 2 and 3 do not match
print(X.T @ v)                   # (2, 3) @ (3,) -> (2,)

The dot product as a transpose

Maths books treat vectors as columns. A dot product is then written as a row times a column:

x · y  =  xᵀy

xᵀ turns the column x into a row, and (1 × n) @ (n × 1) gives a 1 × 1 result: a single number. You will see xᵀy, wᵀx and θᵀx constantly in machine learning papers. They all mean "dot product".

Symmetric matrices

Try it yourselfPython
import numpy as np

distances = np.array([
    [0, 5, 9],
    [5, 0, 4],
    [9, 4, 0],
])                                 # distance from city i to city j

print(np.array_equal(distances, distances.T))

A table of distances between cities is symmetric, because the distance from A to B equals the distance from B to A. Symmetric matrices have especially nice properties, which you will meet in the eigenvector lesson.

The transpose of a product

Like taking off socks and shoes: what went on last comes off first.

Try it yourselfPython
import numpy as np

A = np.array([[1, 2], [3, 4]])
B = np.array([[0, 1], [5, 2]])

print(np.array_equal((A @ B).T, B.T @ A.T))
print(np.array_equal((A @ B).T, A.T @ B.T))

XᵀX: the most famous product

For a data matrix X with n examples and d features, XᵀX is a d × d matrix that compares every feature with every other feature.

Try it yourselfPython
import numpy as np

X = np.array([[1, 2],
              [2, 1],
              [3, 4]])             # 3 examples, 2 features

G = X.T @ X
print(G)
print("Shape:", G.shape, " symmetric:", np.array_equal(G, G.T))

Entry (i, j) of XᵀX is the dot product of feature column i with feature column j. It is always square and symmetric.

Common mistakes

Practice exercise

Build a film-by-film comparison matrix from user ratings with a transpose.

Your solutionPython
import numpy as np

# Ratings by 3 users (rows) for 2 films (columns)
R = np.array([[5, 3],
              [4, 4],
              [1, 5]])

# 1. Print the transpose of R and its shape.
# 2. Compute F = Rᵀ @ R, a film-by-film matrix, and print it.
# 3. Print whether F is symmetric.

# Output must be exactly:
# [[5 4 1]
#  [3 4 5]]
# (2, 3)
# [[42 36]
#  [36 50]]
# Symmetric: True
Knowledge check

A has shape (4 × 7). What is the shape of Aᵀ A?

Summary

  • The transpose Aᵀ swaps rows and columns, turning shape (m × n) into (n × m).
  • A dot product is written xᵀy, and a linear model as wᵀx + b.
  • A symmetric matrix equals its transpose.
  • (AB)ᵀ = BᵀAᵀ: reverse the order.
  • XᵀX compares every feature with every other, and leads to the covariance matrix.

You have finished the Matrices module. Next: seeing matrices as transformations of space.

Finished reading? Tick it off to track your progress.