Lesson 2 of 16 · Vectors
Vector Addition and Scaling
Learn to add, subtract and scale vectors, see each operation as a picture, and understand linear combinations and span, the ideas behind averages, embeddings and neural networks.
- 8 min read
- Intermediate
- Reviewed
- How to add and subtract vectors, and what that looks like
- How multiplying by a scalar stretches, shrinks or flips a vector
- What a linear combination is
- What the span of vectors means
- How these operations appear in machine learning
Adding vectors
As a picture, addition means tip to tail: walk along the first arrow, then walk along the second from where you stopped. The sum is the arrow from the start to where you end up.
import numpy as np a = np.array([3, 1]) b = np.array([1, 2]) print(a + b) print(b + a) # the order does not matter
Think of directions in a city: "3 blocks east and 1 north, then 1 east and 2 north" ends in the same place as "4 east and 3 north".
Subtracting vectors
Subtraction answers "how do I get from here to there?". It is how you measure the difference between two data points, such as a prediction and the true answer.
import numpy as np
predicted = np.array([210, 3.1])
actual = np.array([200, 3.0])
difference = predicted - actual
print("Error vector:", difference)Scaling a vector
| Scalar | Effect |
|---|---|
| bigger than 1 | stretches the vector |
| between 0 and 1 | shrinks it |
| 0 | gives the zero vector [0, 0] |
| negative | flips it to the opposite direction, then stretches or shrinks |
The word scalar comes from this: a scalar scales a vector.
Linear combinations
Put scaling and adding together and you get the most important pattern in linear algebra.
import numpy as np v1 = np.array([1, 0]) v2 = np.array([0, 1]) combo = 3 * v1 + 2 * v2 print(combo) # Mixing ingredients: 2 portions of recipe A and 1 portion of recipe B recipe_a = np.array([200, 50, 10]) # grams of flour, sugar, butter recipe_b = np.array([100, 0, 40]) print(2 * recipe_a + 1 * recipe_b)
A neural network layer is built from linear combinations: each neuron multiplies its inputs by weights and adds them up.
Span
- The span of
[1, 0]alone is just the horizontal line: you can only stretch along it. - The span of
[1, 0]and[0, 1]is the whole 2D plane: any[x, y]isx[1, 0] + y[0, 1]. - The span of
[1, 2]and[2, 4]is only a line, because the second is just double the first. It adds no new direction.
Common mistakes
Find the centre of a group of users with a linear combination, as clustering algorithms do.
import numpy as np # Three users described by [hours of sport, hours of music, hours of cooking] watched per week users = [np.array([4, 1, 0]), np.array([6, 0, 2]), np.array([2, 2, 1])] # 1. Work out the centroid (average vector) of the three users with a linear combination. # Add them up, then multiply by 1/3. Round to 2 places with np.round. # 2. Work out how far the first user is from the centroid: users[0] - centroid, rounded to 2 places. # Output must be exactly: # Centroid: [4. 1. 1.] # First user minus centroid: [ 0. 0. -1.]
What is 3 × [2, -1] + [1, 4]?
Summary
- Add vectors component by component. As arrows, place them tip to tail.
- Subtract to get the arrow from one point to another, such as an error.
- Scalar multiplication stretches, shrinks or flips a vector.
- A linear combination is a weighted sum of vectors. Averages and neuron outputs are linear combinations.
- The span is everything a set of vectors can reach.
Next: the dot product, the single most used operation in machine learning.
Finished reading? Tick it off to track your progress.