Velamuvi AIAcademy

Lesson 4 of 16 · Change

Derivatives in Python

Compute derivatives in Python three ways: numerical differentiation with finite differences, symbolic differentiation with SymPy, and a first look at automatic differentiation, plus np.gradient for real data.

  • 9 min read
  • Intermediate
  • Reviewed
In this lesson you will learn
  • How to pass a function to another function in Python
  • How to write a numerical derivative function
  • How to choose the step size h, and why too small is a problem
  • How to estimate rates of change from data with np.gradient
  • How to get exact derivatives with SymPy

Three ways to differentiate

MethodHow it worksUsed for
Numericalmeasure the slope with a tiny step hquick checks, data
Symbolicapply the rules to the formula, like you do on paperexact formulas
Automaticapply the rules to every operation while the code runstraining neural networks

This lesson covers the first two in code and previews the third, which gets its own lesson later.

Functions are values

In Python a function can be passed to another function, just like a number. Write the function name without brackets:

Try it yourselfPython
def square(x):
    return x**2

def apply_twice(f, x):
    return f(f(x))

print(apply_twice(square, 3))     # square(square(3)) = square(9) = 81

square (no brackets) is the function itself. square(3) calls it. This lets us write one derivative function that works for any function.

Numerical differentiation

Try it yourselfPython
import math

def derivative(f, x, h=0.00001):
    """Estimate f'(x) with a central difference."""
    return (f(x + h) - f(x - h)) / (2 * h)

def square(x):
    return x**2

print(round(derivative(square, 3), 4))      # exact: 2·3 = 6
print(round(derivative(math.exp, 0), 4))    # exact: e⁰ = 1
print(round(derivative(math.log, 2), 4))    # exact: 1/2
print(round(derivative(math.sin, 0), 4))    # exact: cos(0) = 1

h=0.00001 is a default argument: you can leave it out, or pass a different h.

How small should h be?

Mathematically, smaller h is better. On a computer it is not, because decimals are stored with about 16 significant digits. With a tiny h, f(x + h) and f(x) are almost identical, and subtracting them loses most of those digits.

Try it yourselfPython
import math

exact = math.e      # the derivative of eˣ at x = 1 is e¹

for h in [0.1, 0.01, 0.001, 0.00001, 1e-8, 1e-12, 1e-15]:
    estimate = (math.exp(1 + h) - math.exp(1)) / h       # forward difference
    print(f"h = {h:<8} error = {abs(estimate - exact):.1e}")

The error falls as h shrinks, then climbs again once rounding errors take over. A step between about 1e-5 and 1e-8 is usually safe.

Derivatives from data

Often you have measurements, not a formula. np.gradient estimates the rate of change between data points:

Try it yourselfPython
import numpy as np

time = np.array([0, 1, 2, 3, 4, 5])          # seconds
distance = np.array([0, 2, 8, 18, 32, 50])   # metres (it is 2t²)

speed = np.gradient(distance, time)          # rate of change of distance
print("speed:", speed)

The true speed is 4t, which gives 0, 4, 8, 12, 16, 20. Inside the data, np.gradient uses central differences and gets it exactly. At the two ends it only has one neighbour, so it uses a one-sided difference and is less accurate (2 instead of 0, 18 instead of 20).

Symbolic differentiation with SymPy

Try it yourselfPython
import sympy as sp

x = sp.symbols("x")                  # declare x as a symbol

f = x**3 - 4*x + 1
f_prime = sp.diff(f, x)              # differentiate with respect to x
print("f'(x) =", f_prime)
print("f'(2) =", f_prime.subs(x, 2)) # substitute x = 2

print(sp.diff(sp.log(x), x))
print(sp.diff(x**2 * sp.sin(x), x))  # the product rule, done for you

SymPy writes ** for powers and * for multiplication. The first run may take a few seconds while the browser loads the library.

You can also turn a SymPy formula into a fast, normal Python function with sp.lambdify:

Try it yourselfPython
import sympy as sp

x = sp.symbols("x")
sigmoid = 1 / (1 + sp.exp(-x))
slope = sp.diff(sigmoid, x)
print(slope)

slope_function = sp.lambdify(x, slope)   # formula -> Python function
print(round(slope_function(0), 4))       # 0.25, the steepest point

Common mistakes

Practice exercise

Build a reusable numerical derivative function and test it on three functions.

Your solutionPython
import math

# 1. Write derivative(f, x, h=0.00001) that returns the central difference
#    (f(x + h) - f(x - h)) / (2 * h).
# 2. Write cube(x) that returns x**3.
# 3. Print the derivative, rounded to 4 places, of:
#    cube at x = 2, math.exp at x = 1 and math.sqrt at x = 4

# Output must be exactly:
# cube at 2: 12.0
# exp at 1: 2.7183
# sqrt at 4: 0.25
Knowledge check

Why does a very tiny step such as h = 1e-15 give a bad derivative estimate?

Summary

  • Functions can be passed around like values: write the name without brackets.
  • Numerical derivatives use (f(x + h) - f(x - h)) / (2h). Choose h around 1e-5.
  • Too small an h causes floating-point error.
  • np.gradient estimates rates of change from data.
  • SymPy gives exact derivatives with sp.diff, values with .subs, and fast functions with sp.lambdify.
  • Neural networks use automatic differentiation, coming later in the course.

Next: functions of many variables and partial derivatives.

Finished reading? Tick it off to track your progress.