Velamuvi AIAcademy

Lesson 1 of 16 · Vectors

Scalars and Vectors

Understand scalars and vectors two ways, as arrows and as lists of numbers, plus components, dimensions, feature vectors and embeddings, with pictures and NumPy.

  • 7 min read
  • Intermediate
  • Reviewed
In this lesson you will learn
  • The difference between a scalar and a vector
  • The two ways to picture a vector: an arrow and a list of numbers
  • What components and dimensions are
  • How every data point in machine learning becomes a vector
  • How to create and use vectors with NumPy

Why linear algebra?

Machine learning works with lots of numbers at once: every pixel of an image, every feature of a customer, every number inside a neural network. Linear algebra is the maths of handling whole groups of numbers together, and its two main tools are vectors and matrices. This first module is about vectors.

Scalars: single numbers

Scalars have a size, and nothing else. "25 degrees" does not point anywhere.

Vectors: numbers with a direction, or a list of numbers

The two pictures are the same thing seen two ways:

  • As an arrow: "go 3 to the right and 2 up". Physics and geometry like this view.
  • As a list: [3, 2], two numbers in a fixed order. Data science and programming like this view.
a = [3, 2]b = [-2, 3]c = [1, -3]
Three vectors drawn as arrows from the origin. Each one is just a pair of numbers.

Moving an arrow does not change the vector. [3, 2] drawn from (1, 1) is the same vector as [3, 2] drawn from the origin: same length, same direction. Usually we draw vectors starting from the origin.

Components and dimension

We can draw vectors in 2 and 3 dimensions. Beyond that we cannot draw them, but the maths works exactly the same in 100 or 10,000 dimensions. That is the great power of thinking of a vector as a list.

Notation you will see

Books and papers write vectors in several ways. They all mean the same thing:

NotationMeaning
v (bold) or v⃗ (arrow)the vector called v
[3, 2] or (3, 2)a vector written as a row
a tall column of numbersa column vector, the most common form in maths books
vᵢthe i-th component of v

Data points are vectors

Here is the idea that connects linear algebra to machine learning:

Try it yourselfPython
house = [120, 3, 2, 15]          # size, bedrooms, bathrooms, age

print("Dimension:", len(house))
print("Size (first component):", house[0])
print("Age (last component):", house[-1])

A photo of 28 × 28 pixels becomes a vector with 784 components. A customer becomes a vector of their age, spending and visits. Once everything is a vector, the same maths works for houses, photos and customers alike.

Vectors in NumPy

A Python list can hold numbers, but it cannot do vector maths. NumPy's array can:

Try it yourselfPython
import numpy as np

v = np.array([3, 2])
w = np.array([1, 0, 4])

print(v)
print("Dimension of v:", v.shape[0])
print("Dimension of w:", w.size)
print("Second component of w:", w[1])

Common mistakes

Practice exercise

Describe customers as feature vectors and read their components.

Your solutionPython
import numpy as np

# Three customers as feature vectors: [age, visits per month, average spend]
ada = np.array([34, 12, 450])
ravi = np.array([28, 3, 120])
mei = np.array([45, 8, 300])

# 1. Print the dimension of ada's vector.
# 2. Print the number of visits of each customer (the second component), on one line.
# 3. Print the age of the oldest customer.

# Output must be exactly:
# Dimension: 3
# Visits: 12 3 8
# Oldest: 45
Knowledge check

A dataset describes each flat with 5 numbers. What are the flats in linear algebra terms?

Summary

  • A scalar is one number. A vector is an ordered list of numbers, pictured as an arrow.
  • Each number is a component, and the number of components is the dimension.
  • Every data point becomes a feature vector, and words become embeddings.
  • In NumPy, np.array([...]) makes a vector, and .shape gives its size.

Next: adding vectors and stretching them.

Finished reading? Tick it off to track your progress.