Lesson 4 of 16 · Vectors
Vector Length and Distance
Learn the length (norm) of a vector, unit vectors and normalising, Euclidean and Manhattan distance, and cosine similarity, the measures behind nearest neighbours and semantic search.
- 9 min read
- Intermediate
- Reviewed
- How to find the length of a vector with Pythagoras
- What a norm is, and the difference between L1 and L2
- What unit vectors are and how to normalise a vector
- How to measure the distance between two data points
- What cosine similarity is and why search engines use it
How long is an arrow?
The vector [3, 4] goes 3 across and 4 up. Its length is the hypotenuse of a right-angled triangle, so Pythagoras gives it:
length = √(3² + 4²) = √(9 + 16) = √25 = 5In any number of dimensions, the rule is the same: square every component, add them up, take the square root.
import numpy as np v = np.array([3, 4]) print(np.sqrt(np.sum(v ** 2))) # the formula, step by step print(np.linalg.norm(v)) # NumPy's ready-made version w = np.array([1, 2, 2]) print(np.linalg.norm(w)) # √(1 + 4 + 4) = 3
The length is also the square root of a vector's dot product with itself: ‖v‖ = √(v · v).
Two kinds of norm
import numpy as np
v = np.array([3, -4])
print("L2:", np.linalg.norm(v)) # 5.0
print("L1:", np.linalg.norm(v, ord=1)) # 3 + 4 = 7.0Unit vectors and normalising
import numpy as np v = np.array([3.0, 4.0]) unit = v / np.linalg.norm(v) print(unit) # [0.6 0.8] print(np.linalg.norm(unit)) # 1.0
Distance between two points
import numpy as np
home = np.array([1, 2])
school = np.array([4, 6])
print("Straight-line distance:", np.linalg.norm(school - home))
print("Walking along the grid:", np.linalg.norm(school - home, ord=1))Distance between feature vectors measures how different two examples are. Two houses with similar size, rooms and age are "close"; very different houses are "far apart". The k-nearest neighbours algorithm predicts by looking at the closest known examples.
Cosine similarity
Sometimes length does not matter, only direction. A long document and a short one about the same topic should count as similar, even though their word-count vectors have very different lengths.
cosine similarity = (a · b) / (‖a‖ × ‖b‖)import numpy as np
def cosine_similarity(a, b):
return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))
short_review = np.array([2, 1, 0]) # counts of: "great", "fast", "broken"
long_review = np.array([8, 4, 0]) # same words, used 4 times as often
angry_review = np.array([0, 1, 5])
print(round(cosine_similarity(short_review, long_review), 3))
print(round(cosine_similarity(short_review, angry_review), 3))The short and long positive reviews score 1.0, a perfect match in direction, even though one vector is four times longer.
Common mistakes
Build the core of a semantic search engine: rank documents by cosine similarity.
import numpy as np
# Tiny "embeddings" of four documents
docs = {
"cooking pasta": np.array([0.9, 0.1, 0.0]),
"baking bread": np.array([0.8, 0.3, 0.1]),
"football results": np.array([0.0, 0.2, 0.9]),
"pasta recipes": np.array([1.0, 0.0, 0.1]),
}
query = np.array([0.95, 0.05, 0.05]) # someone searching for pasta
# 1. Write cosine_similarity(a, b).
# 2. Print each document's similarity to the query, rounded to 3 places.
# 3. Print the best match.
# Output must be exactly:
# cooking pasta: 0.997
# baking bread: 0.952
# football results: 0.063
# pasta recipes: 0.998
# Best match: pasta recipes
What is the length (L2 norm) of the vector [6, 8]?
Summary
- The norm
‖v‖is a vector's length: √(sum of squares) for the L2 norm, sum of absolute values for L1. - A unit vector has length 1. Normalising divides a vector by its length.
- Euclidean distance
‖a - b‖measures how far apart two data points are. - Cosine similarity compares only directions, from 1 (same) to -1 (opposite). It powers semantic search.
You have finished the Vectors module. Next: matrices, which hold many vectors at once.
Finished reading? Tick it off to track your progress.