Lesson 3 of 16 · Vectors
The Dot Product
Understand the dot product two ways, as a sum of products and as a measure of how much two vectors point the same way, and see how it powers neurons, predictions and similarity search.
- 9 min read
- Intermediate
- Reviewed
- How to calculate a dot product
- What the dot product means as a picture
- What a positive, zero or negative dot product tells you
- What orthogonal vectors are
- Why every neuron and every linear model is built on dot products
Multiply, then add
a = [2, 3, 1]
b = [4, 0, 5]
a · b = (2 × 4) + (3 × 0) + (1 × 5) = 8 + 0 + 5 = 13Two vectors in, one scalar out. In NumPy, use np.dot(a, b) or the @ operator:
import numpy as np a = np.array([2, 3, 1]) b = np.array([4, 0, 5]) print(np.dot(a, b)) print(a @ b) print(sum(a * b)) # the same thing, step by step: multiply, then add
An everyday dot product
You have used dot products at a till without knowing it:
import numpy as np
quantities = np.array([2, 1, 3]) # 2 coffees, 1 sandwich, 3 cookies
prices = np.array([150, 220, 40]) # price of each item
total = quantities @ prices
print("Total to pay:", total)Each quantity is multiplied by its price, and everything is added up. That is exactly a dot product, and it is exactly how a linear model makes a prediction.
The dot product as a prediction
A linear model gives each feature a weight that says how much it matters. The prediction is the dot product of the features and the weights, plus a bias:
prediction = w · x + bimport numpy as np
# Features of a house: [size in m², bedrooms, age in years]
x = np.array([120, 3, 15])
# Weights learned by a model: price change per unit of each feature (in thousands)
w = np.array([40, 150, -20])
b = 500 # bias: the base price
prediction = w @ x + b
print("Predicted price (thousands):", prediction)Notice the negative weight for age: older houses get lower predictions. The weights carry the model's "knowledge".
The dot product as a picture
The same number has a geometric meaning: how much two vectors point in the same direction.
| Dot product | Meaning | Angle between the arrows |
|---|---|---|
| large and positive | they point roughly the same way | small (less than 90°) |
| zero | they are at right angles | exactly 90° |
| negative | they point roughly opposite ways | large (more than 90°) |
import numpy as np
a = np.array([4, 1])
for name, v in [("b", [3, 3]), ("c", [-1, 4]), ("d", [-3, -2])]:
print(f"a · {name} = {a @ np.array(v)}")The full formula
The picture comes from a second formula for the same number:
a · b = |a| × |b| × cos(θ)Here |a| and |b| are the lengths of the vectors (the next lesson) and θ (theta) is the angle between them. You do not need trigonometry to use it. Just remember what cos(θ) does: it is 1 when the vectors point the same way, 0 at a right angle and -1 when they point in opposite directions.
Common mistakes
Use dot products to score applicants with a linear model.
import numpy as np
# A model scores job applicants from [years of experience, skills test score, interview score]
weights = np.array([2, 0.5, 1])
bias = -10
applicants = {
"Ada": np.array([5, 80, 7]),
"Ravi": np.array([2, 95, 9]),
"Mei": np.array([8, 60, 6]),
}
# For each applicant, print their score = weights · features + bias.
# Then print the name of the applicant with the highest score.
# Output must be exactly:
# Ada: 47.0
# Ravi: 50.5
# Mei: 42.0
# Best: Ravi
The dot product of two vectors is 0. What does that tell you?
Summary
- The dot product multiplies matching components and adds them: two vectors in, one number out.
- It is a weighted sum, which is how linear models and neurons make predictions:
w · x + b. - Geometrically, it measures how much two vectors point the same way.
- Positive means similar direction, zero means orthogonal, negative means opposite.
- In NumPy, use
a @ bornp.dot(a, b).
Next: measuring the length of a vector and the distance between two of them.
Finished reading? Tick it off to track your progress.