Velamuvi AIAcademy

Lesson 3 of 16 · Vectors

The Dot Product

Understand the dot product two ways, as a sum of products and as a measure of how much two vectors point the same way, and see how it powers neurons, predictions and similarity search.

  • 9 min read
  • Intermediate
  • Reviewed
In this lesson you will learn
  • How to calculate a dot product
  • What the dot product means as a picture
  • What a positive, zero or negative dot product tells you
  • What orthogonal vectors are
  • Why every neuron and every linear model is built on dot products

Multiply, then add

a = [2, 3, 1]
b = [4, 0, 5]

a · b = (2 × 4) + (3 × 0) + (1 × 5) = 8 + 0 + 5 = 13

Two vectors in, one scalar out. In NumPy, use np.dot(a, b) or the @ operator:

Try it yourselfPython
import numpy as np

a = np.array([2, 3, 1])
b = np.array([4, 0, 5])

print(np.dot(a, b))
print(a @ b)
print(sum(a * b))       # the same thing, step by step: multiply, then add

An everyday dot product

You have used dot products at a till without knowing it:

Try it yourselfPython
import numpy as np

quantities = np.array([2, 1, 3])          # 2 coffees, 1 sandwich, 3 cookies
prices = np.array([150, 220, 40])         # price of each item

total = quantities @ prices
print("Total to pay:", total)

Each quantity is multiplied by its price, and everything is added up. That is exactly a dot product, and it is exactly how a linear model makes a prediction.

The dot product as a prediction

A linear model gives each feature a weight that says how much it matters. The prediction is the dot product of the features and the weights, plus a bias:

prediction = w · x + b
Try it yourselfPython
import numpy as np

# Features of a house: [size in m², bedrooms, age in years]
x = np.array([120, 3, 15])

# Weights learned by a model: price change per unit of each feature (in thousands)
w = np.array([40, 150, -20])
b = 500                                  # bias: the base price

prediction = w @ x + b
print("Predicted price (thousands):", prediction)

Notice the negative weight for age: older houses get lower predictions. The weights carry the model's "knowledge".

The dot product as a picture

The same number has a geometric meaning: how much two vectors point in the same direction.

Dot productMeaningAngle between the arrows
large and positivethey point roughly the same waysmall (less than 90°)
zerothey are at right anglesexactly 90°
negativethey point roughly opposite wayslarge (more than 90°)
ab (a·b = 15)c (a·c = 0)d (a·d = -14)
b points fairly close to a, so a·b is positive. c is at a right angle to a, so a·c = 0. d points away from a, so a·d is negative.
Try it yourselfPython
import numpy as np

a = np.array([4, 1])
for name, v in [("b", [3, 3]), ("c", [-1, 4]), ("d", [-3, -2])]:
    print(f"a · {name} = {a @ np.array(v)}")

The full formula

The picture comes from a second formula for the same number:

a · b = |a| × |b| × cos(θ)

Here |a| and |b| are the lengths of the vectors (the next lesson) and θ (theta) is the angle between them. You do not need trigonometry to use it. Just remember what cos(θ) does: it is 1 when the vectors point the same way, 0 at a right angle and -1 when they point in opposite directions.

Common mistakes

Practice exercise

Use dot products to score applicants with a linear model.

Your solutionPython
import numpy as np

# A model scores job applicants from [years of experience, skills test score, interview score]
weights = np.array([2, 0.5, 1])
bias = -10

applicants = {
    "Ada": np.array([5, 80, 7]),
    "Ravi": np.array([2, 95, 9]),
    "Mei": np.array([8, 60, 6]),
}

# For each applicant, print their score = weights · features + bias.
# Then print the name of the applicant with the highest score.

# Output must be exactly:
# Ada: 47.0
# Ravi: 50.5
# Mei: 42.0
# Best: Ravi
Knowledge check

The dot product of two vectors is 0. What does that tell you?

Summary

  • The dot product multiplies matching components and adds them: two vectors in, one number out.
  • It is a weighted sum, which is how linear models and neurons make predictions: w · x + b.
  • Geometrically, it measures how much two vectors point the same way.
  • Positive means similar direction, zero means orthogonal, negative means opposite.
  • In NumPy, use a @ b or np.dot(a, b).

Next: measuring the length of a vector and the distance between two of them.

Finished reading? Tick it off to track your progress.