Velamuvi AIAcademy

Lesson 4 of 16 · Vectors

Vector Length and Distance

Learn the length (norm) of a vector, unit vectors and normalising, Euclidean and Manhattan distance, and cosine similarity, the measures behind nearest neighbours and semantic search.

  • 9 min read
  • Intermediate
  • Reviewed
In this lesson you will learn
  • How to find the length of a vector with Pythagoras
  • What a norm is, and the difference between L1 and L2
  • What unit vectors are and how to normalise a vector
  • How to measure the distance between two data points
  • What cosine similarity is and why search engines use it

How long is an arrow?

The vector [3, 4] goes 3 across and 4 up. Its length is the hypotenuse of a right-angled triangle, so Pythagoras gives it:

length = √(3² + 4²) = √(9 + 16) = √25 = 5
v = [3, 4], length 5
The vector is the long side of a right-angled triangle with sides 3 and 4.

In any number of dimensions, the rule is the same: square every component, add them up, take the square root.

Try it yourselfPython
import numpy as np

v = np.array([3, 4])
print(np.sqrt(np.sum(v ** 2)))      # the formula, step by step
print(np.linalg.norm(v))            # NumPy's ready-made version

w = np.array([1, 2, 2])
print(np.linalg.norm(w))            # √(1 + 4 + 4) = 3

The length is also the square root of a vector's dot product with itself: ‖v‖ = √(v · v).

Two kinds of norm

Try it yourselfPython
import numpy as np

v = np.array([3, -4])
print("L2:", np.linalg.norm(v))             # 5.0
print("L1:", np.linalg.norm(v, ord=1))      # 3 + 4 = 7.0

Unit vectors and normalising

Try it yourselfPython
import numpy as np

v = np.array([3.0, 4.0])
unit = v / np.linalg.norm(v)

print(unit)                         # [0.6 0.8]
print(np.linalg.norm(unit))         # 1.0

Distance between two points

Try it yourselfPython
import numpy as np

home = np.array([1, 2])
school = np.array([4, 6])

print("Straight-line distance:", np.linalg.norm(school - home))
print("Walking along the grid:", np.linalg.norm(school - home, ord=1))

Distance between feature vectors measures how different two examples are. Two houses with similar size, rooms and age are "close"; very different houses are "far apart". The k-nearest neighbours algorithm predicts by looking at the closest known examples.

Cosine similarity

Sometimes length does not matter, only direction. A long document and a short one about the same topic should count as similar, even though their word-count vectors have very different lengths.

cosine similarity = (a · b) / (‖a‖ × ‖b‖)
Try it yourselfPython
import numpy as np


def cosine_similarity(a, b):
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))


short_review = np.array([2, 1, 0])        # counts of: "great", "fast", "broken"
long_review = np.array([8, 4, 0])         # same words, used 4 times as often
angry_review = np.array([0, 1, 5])

print(round(cosine_similarity(short_review, long_review), 3))
print(round(cosine_similarity(short_review, angry_review), 3))

The short and long positive reviews score 1.0, a perfect match in direction, even though one vector is four times longer.

Common mistakes

Practice exercise

Build the core of a semantic search engine: rank documents by cosine similarity.

Your solutionPython
import numpy as np

# Tiny "embeddings" of four documents
docs = {
    "cooking pasta": np.array([0.9, 0.1, 0.0]),
    "baking bread": np.array([0.8, 0.3, 0.1]),
    "football results": np.array([0.0, 0.2, 0.9]),
    "pasta recipes": np.array([1.0, 0.0, 0.1]),
}
query = np.array([0.95, 0.05, 0.05])     # someone searching for pasta

# 1. Write cosine_similarity(a, b).
# 2. Print each document's similarity to the query, rounded to 3 places.
# 3. Print the best match.

# Output must be exactly:
# cooking pasta: 0.997
# baking bread: 0.952
# football results: 0.063
# pasta recipes: 0.998
# Best match: pasta recipes
Knowledge check

What is the length (L2 norm) of the vector [6, 8]?

Summary

  • The norm ‖v‖ is a vector's length: √(sum of squares) for the L2 norm, sum of absolute values for L1.
  • A unit vector has length 1. Normalising divides a vector by its length.
  • Euclidean distance ‖a - b‖ measures how far apart two data points are.
  • Cosine similarity compares only directions, from 1 (same) to -1 (opposite). It powers semantic search.

You have finished the Vectors module. Next: matrices, which hold many vectors at once.

Finished reading? Tick it off to track your progress.