Velamuvi AIAcademy

Lesson 5 of 16 · Many Variables

Partial Derivatives

Understand partial derivatives: how a function of several variables changes when you nudge just one input and hold the others still. Notation, rules, slices of a surface, and the partial derivatives of a model's loss.

  • 8 min read
  • Intermediate
  • Reviewed
In this lesson you will learn
  • What a function of several variables is
  • What a partial derivative measures
  • How to compute partial derivatives with the usual rules
  • How to picture a partial derivative as the slope of a slice
  • How partial derivatives of the loss guide each weight of a model

Functions with many inputs

So far, every function had one input. Real models have many. A house-price model might use size, bedrooms and age, and its loss depends on every one of its weights.

A function of two inputs can be pictured as a landscape: x and y say where you stand on the map, and f(x, y) is the height of the ground there.

Change one thing at a time

How does the height change as you move? It depends on the direction. The simplest question is: what if I only move east (change x) and keep y fixed? That rate of change is a partial derivative.

To compute ∂f/∂x, treat every other variable as if it were a fixed number, then differentiate as usual.

Checking with code

Numerically, nudge one input and keep the other still:

Try it yourselfPython
def f(x, y):
    return x**2 + 3*x*y + y**2

h = 0.00001
x, y = 1, 2

df_dx = (f(x + h, y) - f(x - h, y)) / (2 * h)     # only x moves
df_dy = (f(x, y + h) - f(x, y - h)) / (2 * h)     # only y moves

print(f"∂f/∂x ≈ {df_dx:.4f}   (rule: 2x + 3y = {2*x + 3*y})")
print(f"∂f/∂y ≈ {df_dy:.4f}   (rule: 3x + 2y = {3*x + 2*y})")

A partial derivative is the slope of a slice

Fix y = 1. Then f(x, 1) = x² + 3x + 1, an ordinary one-variable curve: a slice through the landscape. The partial derivative ∂f/∂x at y = 1 is just the slope of that slice.

-5-4-3-2-10123-20246810
slice f(x, 1) = x² + 3x + 1
at x = 0: f(x) = 1slope of tangent = 3
The slope of the slice equals ∂f/∂x = 2x + 3y with y = 1, so 2x + 3. At x = 0 it is 3.

Partial derivatives of a loss

Here is the real reason this matters. A straight-line model predicts p = w·x + b, with two parameters: the weight w and the bias b. Its mean squared error loss depends on both:

L(w, b) = mean of (w·x + b - y)²   over all data points

Using the rules (and the chain rule from a later lesson), its partial derivatives are:

∂L/∂w = mean of 2·(w·x + b - y)·x
∂L/∂b = mean of 2·(w·x + b - y)
Try it yourselfPython
import numpy as np

x = np.array([1.0, 2.0, 3.0, 4.0])
y = np.array([3.0, 5.0, 7.0, 9.0])      # the true rule is y = 2x + 1

def loss(w, b):
    return np.mean((w * x + b - y) ** 2)

w, b = 1.0, 0.0                          # a poor first guess
error = w * x + b - y
print("loss:", round(loss(w, b), 3))
print("∂L/∂w:", round(np.mean(2 * error * x), 3))
print("∂L/∂b:", round(np.mean(2 * error), 3))

Both partial derivatives are negative. That means increasing w would lower the loss, and so would increasing b. Indeed, the best values are w = 2 and b = 1, both bigger than our guess. Each partial derivative tells one parameter which way to move.

Common mistakes

Practice exercise

Compute partial derivatives numerically and compare them with the rules.

Your solutionPython
def f(x, y):
    return x**2 * y + y**3

h = 0.00001
x, y = 2, 1

# Estimate both partial derivatives at (2, 1) with central differences
# and print them rounded to 4 places.

# Output must be exactly:
# df/dx: 4.0
# df/dy: 7.0
Knowledge check

What is ∂f/∂y for f(x, y) = 5x² + 2xy?

Summary

  • A function of several variables is like a landscape: inputs are the position, output is the height.
  • A partial derivative ∂f/∂x is the rate of change when only x moves and everything else is frozen.
  • Compute it with the normal rules, treating the other variables as constants.
  • It is the slope of a slice through the landscape.
  • The partial derivatives of a loss tell each parameter which way to move.

Next: collecting all the partial derivatives into one vector, the gradient.

Finished reading? Tick it off to track your progress.