Lesson 5 of 16 · Many Variables
Partial Derivatives
Understand partial derivatives: how a function of several variables changes when you nudge just one input and hold the others still. Notation, rules, slices of a surface, and the partial derivatives of a model's loss.
- 8 min read
- Intermediate
- Reviewed
- What a function of several variables is
- What a partial derivative measures
- How to compute partial derivatives with the usual rules
- How to picture a partial derivative as the slope of a slice
- How partial derivatives of the loss guide each weight of a model
Functions with many inputs
So far, every function had one input. Real models have many. A house-price model might use size, bedrooms and age, and its loss depends on every one of its weights.
A function of two inputs can be pictured as a landscape: x and y say where you stand on the map, and f(x, y) is the height of the ground there.
Change one thing at a time
How does the height change as you move? It depends on the direction. The simplest question is: what if I only move east (change x) and keep y fixed? That rate of change is a partial derivative.
To compute ∂f/∂x, treat every other variable as if it were a fixed number, then differentiate as usual.
Checking with code
Numerically, nudge one input and keep the other still:
def f(x, y):
return x**2 + 3*x*y + y**2
h = 0.00001
x, y = 1, 2
df_dx = (f(x + h, y) - f(x - h, y)) / (2 * h) # only x moves
df_dy = (f(x, y + h) - f(x, y - h)) / (2 * h) # only y moves
print(f"∂f/∂x ≈ {df_dx:.4f} (rule: 2x + 3y = {2*x + 3*y})")
print(f"∂f/∂y ≈ {df_dy:.4f} (rule: 3x + 2y = {3*x + 2*y})")A partial derivative is the slope of a slice
Fix y = 1. Then f(x, 1) = x² + 3x + 1, an ordinary one-variable curve: a slice through the landscape. The partial derivative ∂f/∂x at y = 1 is just the slope of that slice.
Partial derivatives of a loss
Here is the real reason this matters. A straight-line model predicts p = w·x + b, with two parameters: the weight w and the bias b. Its mean squared error loss depends on both:
L(w, b) = mean of (w·x + b - y)² over all data pointsUsing the rules (and the chain rule from a later lesson), its partial derivatives are:
∂L/∂w = mean of 2·(w·x + b - y)·x
∂L/∂b = mean of 2·(w·x + b - y)import numpy as np
x = np.array([1.0, 2.0, 3.0, 4.0])
y = np.array([3.0, 5.0, 7.0, 9.0]) # the true rule is y = 2x + 1
def loss(w, b):
return np.mean((w * x + b - y) ** 2)
w, b = 1.0, 0.0 # a poor first guess
error = w * x + b - y
print("loss:", round(loss(w, b), 3))
print("∂L/∂w:", round(np.mean(2 * error * x), 3))
print("∂L/∂b:", round(np.mean(2 * error), 3))Both partial derivatives are negative. That means increasing w would lower the loss, and so would increasing b. Indeed, the best values are w = 2 and b = 1, both bigger than our guess. Each partial derivative tells one parameter which way to move.
Common mistakes
Compute partial derivatives numerically and compare them with the rules.
def f(x, y):
return x**2 * y + y**3
h = 0.00001
x, y = 2, 1
# Estimate both partial derivatives at (2, 1) with central differences
# and print them rounded to 4 places.
# Output must be exactly:
# df/dx: 4.0
# df/dy: 7.0
What is ∂f/∂y for f(x, y) = 5x² + 2xy?
Summary
- A function of several variables is like a landscape: inputs are the position, output is the height.
- A partial derivative ∂f/∂x is the rate of change when only x moves and everything else is frozen.
- Compute it with the normal rules, treating the other variables as constants.
- It is the slope of a slice through the landscape.
- The partial derivatives of a loss tell each parameter which way to move.
Next: collecting all the partial derivatives into one vector, the gradient.
Finished reading? Tick it off to track your progress.