M1.2 — Calculus: Derivatives, Gradients and the Chain Rule

Learning objectives

After this lesson, you should be able to:

• Explain the derivative as a rate of change and compute one by finite differences.
• Use the basic derivative rules and check them numerically.
• Apply the chain rule to a composition.
• Explain why chained small derivatives cause vanishing gradients.
• Explain partial derivatives and the gradient, and why stepping against it lowers a function.
• Perform a gradient check.

The one-sentence idea

The derivative tells you how a small change in an input changes the output, the chain rule multiplies those effects through a stack of steps, and the gradient collects them for every parameter.

Differentiate by hand, then confirm with a small step.

Why a robot-learning course needs calculus

Training a model, as M2.4 and M2.5 showed, means adjusting its parameters to make a loss smaller. To know which way to adjust each parameter you need to know how the loss changes when that parameter changes. That quantity is a derivative, and with many parameters it becomes a gradient. The same tool appears in the Jacobian of M5.3. You will not be asked to do hard integration or symbolic algebra here. The goal is to be able to read “∂L/∂w”, know what it means, and know how to check it.

The derivative: how fast something changes

The derivative of a function f at a point x is its instantaneous rate of change, the slope of the curve there. It is the limit of the change in f over a small step h:

You can approximate it directly by using a small but nonzero h, a finite difference. This is the cheapest check there is on any derivative you work out by hand or any code that claims to compute one:

import numpy as np

def numeric_derivative(f, x, h=1e-5):
    return (f(x + h) - f(x - h)) / (2 * h)          # a symmetric step is more accurate

f = lambda x: x ** 2
print(round(numeric_derivative(f, 3.0), 4))          # the exact derivative of x^2 is 2x, which is 6 at x = 3
print(round(abs((f(3.1) - f(3.0)) / 0.1 - 6), 3), abs(numeric_derivative(f, 3.0) - 6) < 1e-6)

6.0
0.1 True

At x = 3 the slope of x² is 6. The second line compares a coarse forward difference with step 0.1, which is off by 0.1, with the symmetric small-step version, which is accurate to better than a millionth. A step that is too tiny backfires, because rounding errors then dominate, so values like 10⁻⁵ to 10⁻⁶ are typical.

You rarely need the limit definition. A short list of rules covers most functions in machine learning:

g = lambda x: x ** 3 + 2 * x                          # power and sum rules: 3x^2 + 2
print(round(numeric_derivative(g, 2.0), 3), 3 * 2 ** 2 + 2)

h = lambda x: np.sin(x) * x ** 2                      # product rule: cos(x) x^2 + sin(x) 2x
print(round(numeric_derivative(h, 1.0), 4), round(np.cos(1.0) * 1 + np.sin(1.0) * 2, 4))

14.0 14
2.2232 2.2232

Both hand-derived answers agree with the numbers. This habit, differentiate by hand and confirm numerically, is exactly how implementations of backpropagation are tested.

The chain rule: derivatives of compositions

A neural network is a function made of functions: the output of one layer is the input of the next. The chain rule says that the derivative of a composition is the product of the derivatives of its parts. If u = g(x) and y = f(u), then:

Read it as: a small change in x changes u by du/dx times as much, and that change in u changes y by dy/du times as much, so the effects multiply.

x = 2.0
u = 3 * x + 1                      # inner function g(x) = 3x + 1, with du/dx = 3
dy_du = 2 * u                      # outer function f(u) = u^2, with dy/du = 2u
print(dy_du * 3)                   # chain rule: dy/dx = 2u * 3
print(round(numeric_derivative(lambda t: (3 * t + 1) ** 2, 2.0), 3))

42.0
42.0

This is the whole idea of backpropagation (M2.4): to find how the loss depends on an early weight, multiply the local derivatives of every step between that weight and the loss, working backwards. The same pattern shows a well-known problem. The sigmoid’s derivative is σ(z)(1 − σ(z)), which is at most 0.25. Chaining many sigmoid layers multiplies many numbers of at most 0.25:

sigmoid = lambda z: 1 / (1 + np.exp(-z))
d_sigmoid = lambda z: sigmoid(z) * (1 - sigmoid(z))
print(round(float(d_sigmoid(0.0)), 3))             # the largest value the derivative can take
print(0.25 ** 5, f"{0.25 ** 20:.2e}")                 # best case for 5 layers and for 20 layers

0.25
0.0009765625 9.09e-13

Even in the best case a 20-layer chain of sigmoids multiplies the gradient by about 10⁻¹², so the earliest layers barely learn. This vanishing gradient is a main reason ReLU (derivative 1 for positive inputs) and residual connections (M3.4) are used in deep networks.

Many variables: partial derivatives and the gradient

A loss depends on many parameters at once. The partial derivative ∂f/∂xᵢ is the derivative with respect to one variable while holding the others fixed. Collecting all of them into a vector gives the gradient ∇f. Two facts make it the central tool of learning:

• The gradient points in the direction in which f increases fastest, and its length is how fast.
• So stepping against the gradient is the quickest way down, which is gradient descent (M2.4).

def numeric_gradient(f, p, h=1e-5):
    p = np.array(p, dtype=float)
    grad = np.zeros_like(p)
    for i in range(len(p)):
        e = np.zeros_like(p); e[i] = h
        grad[i] = (f(p + e) - f(p - e)) / (2 * h)       # nudge one variable at a time
    return grad

f2 = lambda p: p[0] ** 2 + 3 * p[1] ** 2
p = np.array([1.0, 2.0])
grad = numeric_gradient(f2, p)
print(grad.round(3), np.array([2 * p[0], 6 * p[1]]))    # compare with the hand-derived (2x, 6y)
print(f2(p), round(float(f2(p - 0.05 * grad)), 3))      # a small step against the gradient

[ 2. 12.] [ 2. 12.]
13.0 6.69

At (1, 2) the gradient is (2, 12): f is far more sensitive to y than to x. A small step against it reduces f from 13.0 to 6.69. With a loss that depends on thousands or billions of parameters, the gradient has that many entries, and backpropagation is the efficient way to get them all.

A gradient check applies the numerical test to a real model. Take the one-neuron loss of M2.4, L = (w x + b − t)². The formulas derived there should match the numeric gradient:

x, t = 2.0, 3.0
loss = lambda p: (p[0] * x + p[1] - t) ** 2            # parameters p = [w, b]
p0 = np.array([0.5, 0.0])
y = p0[0] * x + p0[1]
analytic = np.array([2 * (y - t) * x, 2 * (y - t)])    # the formulas from M2.4
print(analytic, numeric_gradient(loss, p0).round(3))

[-8. -4.] [-8. -4.]

The two agree. When the gradients of a new model are wrong, training silently misbehaves rather than failing, so this check is among the most valuable habits in the course. Finally, when a function has several outputs as well as several inputs, the matrix of all the partial derivatives is the Jacobian, which M5.3 used to link small joint motions to hand motion.

Common misconceptions

Engineering takeaways

1. The derivative is the instantaneous rate of change, the slope of the function.
2. A symmetric finite difference approximates it and checks hand-derived results.
3. The chain rule says the derivative of a composition is the product of the parts’ derivatives.
4. Long chains of small derivatives, as with sigmoids, make gradients vanish.
5. The gradient collects the partial derivatives and points toward steepest increase.
6. Gradient descent steps against the gradient.
7. A gradient check compares analytic and numeric gradients.
8. The Jacobian is the gradient generalised to several outputs.

Calculus answers one question over and over in training: if I nudge this number, what happens to the loss?

Knowledge check

Try each question first, then reveal the explanation.

What’s next

You can now follow how a loss changes with its parameters. Next is probability, the language for uncertainty, noise and the outputs of models.

Back to the VLA roadmap prerequisites