Lesson 4 of 16 · Change
Derivatives in Python
Compute derivatives in Python three ways: numerical differentiation with finite differences, symbolic differentiation with SymPy, and a first look at automatic differentiation, plus np.gradient for real data.
- 9 min read
- Intermediate
- Reviewed
- How to pass a function to another function in Python
- How to write a numerical derivative function
- How to choose the step size h, and why too small is a problem
- How to estimate rates of change from data with np.gradient
- How to get exact derivatives with SymPy
Three ways to differentiate
| Method | How it works | Used for |
|---|---|---|
| Numerical | measure the slope with a tiny step h | quick checks, data |
| Symbolic | apply the rules to the formula, like you do on paper | exact formulas |
| Automatic | apply the rules to every operation while the code runs | training neural networks |
This lesson covers the first two in code and previews the third, which gets its own lesson later.
Functions are values
In Python a function can be passed to another function, just like a number. Write the function name without brackets:
def square(x):
return x**2
def apply_twice(f, x):
return f(f(x))
print(apply_twice(square, 3)) # square(square(3)) = square(9) = 81square (no brackets) is the function itself. square(3) calls it. This lets us write one derivative function that works for any function.
Numerical differentiation
import math
def derivative(f, x, h=0.00001):
"""Estimate f'(x) with a central difference."""
return (f(x + h) - f(x - h)) / (2 * h)
def square(x):
return x**2
print(round(derivative(square, 3), 4)) # exact: 2·3 = 6
print(round(derivative(math.exp, 0), 4)) # exact: e⁰ = 1
print(round(derivative(math.log, 2), 4)) # exact: 1/2
print(round(derivative(math.sin, 0), 4)) # exact: cos(0) = 1h=0.00001 is a default argument: you can leave it out, or pass a different h.
How small should h be?
Mathematically, smaller h is better. On a computer it is not, because decimals are stored with about 16 significant digits. With a tiny h, f(x + h) and f(x) are almost identical, and subtracting them loses most of those digits.
import math
exact = math.e # the derivative of eˣ at x = 1 is e¹
for h in [0.1, 0.01, 0.001, 0.00001, 1e-8, 1e-12, 1e-15]:
estimate = (math.exp(1 + h) - math.exp(1)) / h # forward difference
print(f"h = {h:<8} error = {abs(estimate - exact):.1e}")The error falls as h shrinks, then climbs again once rounding errors take over. A step between about 1e-5 and 1e-8 is usually safe.
Derivatives from data
Often you have measurements, not a formula. np.gradient estimates the rate of change between data points:
import numpy as np
time = np.array([0, 1, 2, 3, 4, 5]) # seconds
distance = np.array([0, 2, 8, 18, 32, 50]) # metres (it is 2t²)
speed = np.gradient(distance, time) # rate of change of distance
print("speed:", speed)The true speed is 4t, which gives 0, 4, 8, 12, 16, 20. Inside the data, np.gradient uses central differences and gets it exactly. At the two ends it only has one neighbour, so it uses a one-sided difference and is less accurate (2 instead of 0, 18 instead of 20).
Symbolic differentiation with SymPy
import sympy as sp
x = sp.symbols("x") # declare x as a symbol
f = x**3 - 4*x + 1
f_prime = sp.diff(f, x) # differentiate with respect to x
print("f'(x) =", f_prime)
print("f'(2) =", f_prime.subs(x, 2)) # substitute x = 2
print(sp.diff(sp.log(x), x))
print(sp.diff(x**2 * sp.sin(x), x)) # the product rule, done for youSymPy writes ** for powers and * for multiplication. The first run may take a few seconds while the browser loads the library.
You can also turn a SymPy formula into a fast, normal Python function with sp.lambdify:
import sympy as sp
x = sp.symbols("x")
sigmoid = 1 / (1 + sp.exp(-x))
slope = sp.diff(sigmoid, x)
print(slope)
slope_function = sp.lambdify(x, slope) # formula -> Python function
print(round(slope_function(0), 4)) # 0.25, the steepest pointCommon mistakes
Build a reusable numerical derivative function and test it on three functions.
import math # 1. Write derivative(f, x, h=0.00001) that returns the central difference # (f(x + h) - f(x - h)) / (2 * h). # 2. Write cube(x) that returns x**3. # 3. Print the derivative, rounded to 4 places, of: # cube at x = 2, math.exp at x = 1 and math.sqrt at x = 4 # Output must be exactly: # cube at 2: 12.0 # exp at 1: 2.7183 # sqrt at 4: 0.25
Why does a very tiny step such as h = 1e-15 give a bad derivative estimate?
Summary
- Functions can be passed around like values: write the name without brackets.
- Numerical derivatives use
(f(x + h) - f(x - h)) / (2h). Choose h around 1e-5. - Too small an h causes floating-point error.
np.gradientestimates rates of change from data.- SymPy gives exact derivatives with
sp.diff, values with.subs, and fast functions withsp.lambdify. - Neural networks use automatic differentiation, coming later in the course.
Next: functions of many variables and partial derivatives.
Finished reading? Tick it off to track your progress.