Velamuvi AIAcademy

Lesson 1 of 16 · Change

Limits

Understand limits in plain words: what a function approaches as x gets close to a value or grows without end, holes in graphs, continuity, and why limits are the foundation of derivatives and machine learning.

  • 9 min read
  • Intermediate
  • Reviewed
In this lesson you will learn
  • What calculus is and why machine learning needs it
  • What a limit is, in plain words and in notation
  • How to find a limit with a table of values
  • Limits at a hole in a graph and limits at infinity
  • What a continuous function is

Why calculus?

Calculus is the maths of change. It answers questions like "if I nudge this number a little, how much does that number change?"

That is exactly the question a machine learning model asks millions of times while it learns: "if I nudge this weight a little, does my error go up or down, and by how much?" The answer is a derivative, and derivatives are built on one simple idea: the limit.

Getting closer and closer

Look at f(x) = (x² - 1) / (x - 1). At x = 1 it breaks: the bottom becomes 0, and you cannot divide by zero. But what happens near x = 1?

Try it yourselfPython
def f(x):
    return (x**2 - 1) / (x - 1)

for x in [0.9, 0.99, 0.999, 0.9999]:
    print(f"f({x}) = {f(x):.4f}")
print()
for x in [1.1, 1.01, 1.001, 1.0001]:
    print(f"f({x}) = {f(x):.4f}")

From both sides, the values get closer and closer to 2. We say "the limit of f(x) as x approaches 1 is 2".

lim     f(x) = 2          read: "the limit of f(x), as x approaches 1, is 2"
x → 1

The graph shows why. The function is just the line y = x + 1 with a single point missing, a hole, at x = 1:

-2-101234-1012345hole at (1, 2)
f(x) = (x² - 1)/(x - 1)
The function has no value at x = 1, but as x gets close to 1 from either side, f(x) gets close to 2.

A famous limit

Here is a limit that matters in physics and signal processing. sin(x)/x is undefined at x = 0, but:

Try it yourselfPython
import math

for x in [0.5, 0.1, 0.01, 0.001]:
    print(f"x = {x:<6} sin(x)/x = {math.sin(x) / x:.6f}")

The values approach 1, even though the function has no value at 0.

-10-50510-0.500.51limit = 1
sin(x)/x
sin(x)/x has a hole at x = 0, and the limit there is 1.

Limits at infinity

Sometimes we ask what happens as x grows without end. Take 1/x: as x gets bigger, 1/x gets closer to 0, but never reaches it.

lim    1/x = 0           "as x goes to infinity, 1/x approaches 0"
x → ∞

You have met an important example already: the sigmoid function, used in classification models. It squashes any number into the range 0 to 1:

Try it yourselfPython
import math

def sigmoid(x):
    return 1 / (1 + math.exp(-x))

for x in [0, 2, 5, 10, 20]:
    print(f"sigmoid({x:>2}) = {sigmoid(x):.8f}   sigmoid({-x:>3}) = {sigmoid(-x):.8f}")

As x → ∞, sigmoid(x) → 1, and as x → -∞, sigmoid(x) → 0. Those limits are why a sigmoid output can be read as a probability: it never goes outside 0 and 1.

Continuous functions

Lines, parabolas, eˣ and the sigmoid are continuous everywhere. 1/x breaks at x = 0, and our (x² - 1)/(x - 1) has a hole at x = 1. Most functions used in machine learning are continuous, because training relies on small changes in the input giving small changes in the output.

Why limits lead to derivatives

The next lesson asks: how steep is a curve at a single point? To measure steepness you need two points. The trick is to bring the second point closer and closer to the first and look at the limit of the steepness. That limit is the derivative, the most important idea in this course.

Computers cannot take a true limit, but they can get very close by using a tiny step, such as 0.0001. You will see both approaches.

Common mistakes

Practice exercise

Use a table of values to estimate a limit. (The exact answer is ln 2 ≈ 0.6931, and you will meet this expression again: it is the slope of 2ˣ at x = 0.)

Your solutionPython
# Estimate the limit of (2**h - 1) / h as h approaches 0.
# For each h in the list, print the value rounded to 4 decimal places.

steps = [0.1, 0.01, 0.001, 0.0001]

# Output must be exactly:
# h = 0.1: 0.7177
# h = 0.01: 0.6956
# h = 0.001: 0.6934
# h = 0.0001: 0.6932
Knowledge check

f(x) = (x² - 4)/(x - 2) is undefined at x = 2. What is its limit as x approaches 2?

Summary

  • Calculus is the maths of change, and machine learning uses it to learn.
  • A limit is the value a function approaches as its input approaches a point, whatever happens exactly at that point.
  • You can estimate a limit with a table of values from both sides, or simplify the formula first.
  • Limits at infinity describe long-run behaviour, such as the sigmoid approaching 0 and 1.
  • A continuous function has no holes or jumps.

Next: using limits to measure the slope of a curve, which gives the derivative.

Finished reading? Tick it off to track your progress.