Lesson 16 of 16 · Decompositions
Linear Algebra in NumPy
A practical NumPy guide for linear algebra: creating arrays, reshaping, broadcasting, vectorised code and the np.linalg toolbox, ending with a mini semantic search engine built from scratch.
- 9 min read
- Intermediate
- Reviewed
- How to create and reshape the arrays you need
- How broadcasting combines arrays of different shapes
- Why vectorised code beats Python loops
- The np.linalg functions you will use most
- How to build a small semantic search engine with matrices
Creating arrays
import numpy as np print(np.zeros((2, 3))) # all zeros, shape 2 x 3 print(np.ones(3)) # a vector of ones print(np.eye(2)) # the identity matrix print(np.arange(6)) # 0 to 5 print(np.linspace(0, 1, 5)) # 5 evenly spaced numbers from 0 to 1
Reshaping
import numpy as np a = np.arange(6) print(a.reshape(2, 3)) print(a.reshape(3, -1)) # -1: "work out this size for me" column = a.reshape(-1, 1) # turn a 1-D array into a column vector print(column.shape)
Reshaping is constant in machine learning: flattening a 28 × 28 image into a vector of 784, or arranging a long list of numbers into a batch.
Broadcasting
import numpy as np
X = np.array([[1.0, 200.0],
[2.0, 300.0],
[3.0, 400.0]]) # shape (3, 2)
means = X.mean(axis=0) # shape (2,)
stds = X.std(axis=0)
X_scaled = (X - means) / stds # (3, 2) minus (2,): the means are used for every row
print(np.round(X_scaled, 3) + 0.0)Broadcasting is how you centre and scale every column of a dataset in one line, with no loop. This standardisation is the usual first step before PCA, k-nearest neighbours or training a neural network.
Vectorise: avoid Python loops
Both versions below compute the same predictions. The vectorised one is one line and, on large data, often hundreds of times faster:
import numpy as np
X = np.array([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
w = np.array([0.5, -1.0])
# Loop version
predictions = []
for row in X:
total = 0.0
for j in range(len(w)):
total += row[j] * w[j]
predictions.append(float(total)) # float() keeps the printout simple
print(predictions)
# Vectorised version
print(X @ w)The np.linalg toolbox
| Function | What it does | Lesson |
|---|---|---|
np.linalg.norm(v) | length of a vector | Vector Length |
A @ B | matrix multiplication | Matrix Multiplication |
np.linalg.inv(A) | inverse | Identity and Inverse |
np.linalg.solve(A, b) | solve Ax = b | Linear Systems |
np.linalg.lstsq(A, b) | least squares | Linear Systems |
np.linalg.det(A) | determinant | Rank and Determinant |
np.linalg.matrix_rank(A) | rank | Rank and Determinant |
np.linalg.eigh(A) | eigenvalues and vectors (symmetric) | Eigenvalues |
np.linalg.svd(A) | singular value decomposition | SVD |
Mini project: a semantic search engine
Time to use the whole course at once. Each document below has a small "embedding" vector, as a real embedding model would produce (real ones have hundreds of dimensions). The search engine:
- stores all document vectors in one matrix,
- normalises every row to length 1 (broadcasting),
- scores every document against the query with one matrix-vector product (cosine similarity),
- sorts the scores to rank the results.
import numpy as np
titles = ["How to bake bread", "Training a neural network", "Best pasta recipes",
"Linear algebra for ML", "Making pizza dough"]
# Columns: [cooking, machine learning, maths]
E = np.array([
[0.90, 0.05, 0.05],
[0.05, 0.90, 0.40],
[0.95, 0.00, 0.05],
[0.00, 0.60, 0.80],
[0.85, 0.05, 0.10],
])
E_unit = E / np.linalg.norm(E, axis=1, keepdims=True) # every row now has length 1
def search(query_vector, top=3):
q = query_vector / np.linalg.norm(query_vector)
scores = E_unit @ q # cosine similarity with every document at once
order = np.argsort(scores)[::-1] # indices from best to worst
for i in order[:top]:
print(f" {scores[i]:.3f} {titles[i]}")
print("Query: 'maths behind deep learning'")
search(np.array([0.0, 0.7, 0.7]))
print("Query: 'something to cook tonight'")
search(np.array([1.0, 0.0, 0.0]))That is, in miniature, how vector databases answer millions of questions a day. The only differences in a real system are longer vectors from an embedding model and clever indexes that avoid checking every document.
Common mistakes
Build a "similar products" feature from a similarity matrix.
import numpy as np
# Embeddings of 4 products: [sporty, formal, casual]
names = ["running shoes", "office shirt", "hoodie", "suit"]
E = np.array([
[0.9, 0.0, 0.4],
[0.0, 0.9, 0.2],
[0.3, 0.0, 0.9],
[0.0, 1.0, 0.0],
])
# 1. Normalise every row of E to length 1 (use keepdims=True).
# 2. Compute the 4 x 4 cosine similarity matrix S = E_unit @ E_unit.T, rounded to 2 places.
# 3. For "office shirt" (row 1), print the most similar OTHER product.
# Output must be exactly:
# [[1. 0.09 0.67 0. ]
# [0.09 1. 0.21 0.98]
# [0.67 0.21 1. 0. ]
# [0. 0.98 0. 1. ]]
# Most like office shirt: suit
X has shape (100, 4) and m has shape (4,). What shape is X - m?
Summary
- Create arrays with
zeros,ones,eye,arangeandlinspace, and change shape withreshape. - Broadcasting combines different shapes, such as subtracting column means from a whole dataset.
- Vectorised code replaces Python loops with fast whole-array operations.
np.linalgcovers norms, solving, inverses, determinants, rank, eigenvectors and the SVD.- A semantic search engine is just normalising, one matrix-vector product and a sort.
You have finished Linear Algebra
You can now think of data as vectors and matrices, compute with dot products and matrix multiplication, see matrices as transformations of space, solve linear systems and least-squares problems, and understand the determinant, rank, projections, eigenvectors, the SVD and PCA. These are the foundations of every machine learning model, from linear regression to large language models.
Where next?
- Take the final quiz to check what you remember.
- Continue with Calculus for Machine Learning, to learn how models improve their parameters.
- Follow a career roadmap to see which maths each AI job needs.
Finished reading? Tick it off to track your progress.