Lesson 10 of 16 · Transformations
Linear Systems
Learn to write systems of linear equations as Ax = b, see their solutions as crossing lines, recognise when there is one, none or infinitely many solutions, and solve them with NumPy, including least squares.
- 8 min read
- Intermediate
- Reviewed
- What a system of linear equations is
- How to write one as Ax = b
- What one, no or infinitely many solutions look like
- How to solve systems with np.linalg.solve
- How least squares finds the best answer when there is no exact one
Several equations at once
A café sells tea and coffee. One morning, 3 teas and 2 coffees cost 340. Another morning, 1 tea and 4 coffees cost 380. What does each drink cost?
3t + 2c = 340
1t + 4c = 380Writing it as Ax = b
[ 3 2 ] [ t ] [ 340 ]
[ 1 4 ] × [ c ] = [ 380 ]
A x bEach row of A, dotted with x, gives one equation. In this form, a computer can solve thousands of equations as easily as two.
import numpy as np
A = np.array([[3, 2],
[1, 4]])
b = np.array([340, 380])
x = np.linalg.solve(A, b)
print("tea =", round(x[0], 6), " coffee =", round(x[1], 6))
print("check:", np.round(A @ x, 6))Tea costs 60 and coffee 80. Checking by putting the answer back in is always a good habit.
A picture: crossing lines
Each equation with two unknowns is a straight line. The solution is the point where the lines cross, because only that point lies on both.
There are only three possibilities:
| Lines | Number of solutions | Matrix A |
|---|---|---|
| cross at one point | exactly one | invertible (determinant not 0) |
| parallel, never meet | none | singular |
| the same line | infinitely many | singular |
import numpy as np
# Parallel lines: x + y = 2 and x + y = 5 can never both be true
A = np.array([[1, 1],
[1, 1]])
b = np.array([2, 5])
print("determinant:", round(np.linalg.det(A), 6) + 0.0)
if np.linalg.matrix_rank(A) < A.shape[0]:
print("No single solution: the matrix is singular")
else:
print(np.linalg.solve(A, b))Check the rank (or the determinant) before calling np.linalg.solve. On some setups, including this browser, a singular matrix does not raise an error: you just get nan values back.
Bigger systems
Three unknowns need three equations. The code does not change:
import numpy as np
# A bakery: flour, sugar and butter in grams for three recipes,
# and the totals used today. How many batches of each recipe were made?
A = np.array([[200, 100, 300], # flour per batch of cake, cookies, bread
[150, 200, 20], # sugar
[100, 150, 40]]) # butter
b = np.array([1600, 1290, 980]) # total flour, sugar, butter used
batches = np.linalg.solve(A, b)
print(np.round(batches, 6) + 0.0)When there is no exact answer: least squares
Real data is noisy. You often have more equations than unknowns, and no single answer satisfies them all. For example, fitting a line y = m·x + c through 5 measured points gives 5 equations but only 2 unknowns.
import numpy as np
# Measured points that lie roughly on a line
hours = np.array([1, 2, 3, 4, 5])
scores = np.array([52, 58, 61, 70, 74])
# Each row says: m * hours + c * 1 = score
A = np.column_stack([hours, np.ones(5)])
solution, residuals, rank, singular_values = np.linalg.lstsq(A, scores, rcond=None)
m, c = solution
print(f"best line: score = {m:.1f} × hours + {c:.1f}")
print("prediction for 6 hours:", round(m * 6 + c, 1))You just did linear regression with linear algebra. The best-fit line is the least-squares solution of an overdetermined system.
Common mistakes
Turn a word problem into Ax = b and solve it.
import numpy as np # A cinema sold adult and child tickets. # Monday: 4 adult + 2 child tickets = 1000 # Tuesday: 3 adult + 5 child tickets = 1100 # 1. Write the system as A and b. # 2. Solve it with np.linalg.solve. # 3. Print each price, and check the answer with A @ prices. # Output must be exactly: # Adult: 200.0 # Child: 100.0 # Check: [1000. 1100.]
Two equations in two unknowns describe parallel lines. How many solutions are there?
Summary
- A system of linear equations is written Ax = b: coefficients, unknowns and results.
- Each solution is where all the equations hold at once, such as where lines cross.
- There is exactly one solution, none or infinitely many. One means A is invertible.
- Solve square systems with
np.linalg.solve. - With more equations than unknowns, least squares (
np.linalg.lstsq) gives the best fit. That is linear regression.
Next: the determinant and rank, which measure how much a matrix squashes space.
Finished reading? Tick it off to track your progress.