Lesson 1 of 16 · Change
Limits
Understand limits in plain words: what a function approaches as x gets close to a value or grows without end, holes in graphs, continuity, and why limits are the foundation of derivatives and machine learning.
- 9 min read
- Intermediate
- Reviewed
- What calculus is and why machine learning needs it
- What a limit is, in plain words and in notation
- How to find a limit with a table of values
- Limits at a hole in a graph and limits at infinity
- What a continuous function is
Why calculus?
Calculus is the maths of change. It answers questions like "if I nudge this number a little, how much does that number change?"
That is exactly the question a machine learning model asks millions of times while it learns: "if I nudge this weight a little, does my error go up or down, and by how much?" The answer is a derivative, and derivatives are built on one simple idea: the limit.
Getting closer and closer
Look at f(x) = (x² - 1) / (x - 1). At x = 1 it breaks: the bottom becomes 0, and you cannot divide by zero. But what happens near x = 1?
def f(x):
return (x**2 - 1) / (x - 1)
for x in [0.9, 0.99, 0.999, 0.9999]:
print(f"f({x}) = {f(x):.4f}")
print()
for x in [1.1, 1.01, 1.001, 1.0001]:
print(f"f({x}) = {f(x):.4f}")From both sides, the values get closer and closer to 2. We say "the limit of f(x) as x approaches 1 is 2".
lim f(x) = 2 read: "the limit of f(x), as x approaches 1, is 2"
x → 1The graph shows why. The function is just the line y = x + 1 with a single point missing, a hole, at x = 1:
A famous limit
Here is a limit that matters in physics and signal processing. sin(x)/x is undefined at x = 0, but:
import math
for x in [0.5, 0.1, 0.01, 0.001]:
print(f"x = {x:<6} sin(x)/x = {math.sin(x) / x:.6f}")The values approach 1, even though the function has no value at 0.
Limits at infinity
Sometimes we ask what happens as x grows without end. Take 1/x: as x gets bigger, 1/x gets closer to 0, but never reaches it.
lim 1/x = 0 "as x goes to infinity, 1/x approaches 0"
x → ∞You have met an important example already: the sigmoid function, used in classification models. It squashes any number into the range 0 to 1:
import math
def sigmoid(x):
return 1 / (1 + math.exp(-x))
for x in [0, 2, 5, 10, 20]:
print(f"sigmoid({x:>2}) = {sigmoid(x):.8f} sigmoid({-x:>3}) = {sigmoid(-x):.8f}")As x → ∞, sigmoid(x) → 1, and as x → -∞, sigmoid(x) → 0. Those limits are why a sigmoid output can be read as a probability: it never goes outside 0 and 1.
Continuous functions
Lines, parabolas, eˣ and the sigmoid are continuous everywhere. 1/x breaks at x = 0, and our (x² - 1)/(x - 1) has a hole at x = 1. Most functions used in machine learning are continuous, because training relies on small changes in the input giving small changes in the output.
Why limits lead to derivatives
The next lesson asks: how steep is a curve at a single point? To measure steepness you need two points. The trick is to bring the second point closer and closer to the first and look at the limit of the steepness. That limit is the derivative, the most important idea in this course.
Computers cannot take a true limit, but they can get very close by using a tiny step, such as 0.0001. You will see both approaches.
Common mistakes
Use a table of values to estimate a limit. (The exact answer is ln 2 ≈ 0.6931, and you will meet this expression again: it is the slope of 2ˣ at x = 0.)
# Estimate the limit of (2**h - 1) / h as h approaches 0. # For each h in the list, print the value rounded to 4 decimal places. steps = [0.1, 0.01, 0.001, 0.0001] # Output must be exactly: # h = 0.1: 0.7177 # h = 0.01: 0.6956 # h = 0.001: 0.6934 # h = 0.0001: 0.6932
f(x) = (x² - 4)/(x - 2) is undefined at x = 2. What is its limit as x approaches 2?
Summary
- Calculus is the maths of change, and machine learning uses it to learn.
- A limit is the value a function approaches as its input approaches a point, whatever happens exactly at that point.
- You can estimate a limit with a table of values from both sides, or simplify the formula first.
- Limits at infinity describe long-run behaviour, such as the sigmoid approaching 0 and 1.
- A continuous function has no holes or jumps.
Next: using limits to measure the slope of a curve, which gives the derivative.
Finished reading? Tick it off to track your progress.