VLA · Prerequisite · Python and Deep Learning
M2.4 — Neural Networks and Backpropagation
How a network’s weights are updated from the loss.
A network learns by comparing its output with the answer, measuring the error as a loss, and nudging every weight in the direction that reduces it. Backpropagation is the method that finds that direction for all the weights at once. This lesson builds it up from a single neuron.
Learning objectives
After this lesson, you should be able to:
- Describe a neural network as stacked linear layers with activation functions between them.
- Compute a forward pass and count a network’s parameters.
- Explain what a loss measures and how a gradient-descent step uses a gradient.
- Describe backpropagation as the chain rule applied backwards through the layers.
- Explain why a ReLU unit that is switched off passes no gradient.
- Relate hand-derived gradients to what PyTorch autograd computes.
- Input goes through the network (forward pass)
- The output is compared with the target
- The loss measures the error
- Gradients flow backwards through every layer (backward pass)
- Each parameter is nudged against its gradient
- The loss goes down
The one-sentence idea
Training a network means repeatedly computing the loss, finding how each weight affects it by applying the chain rule backwards, and nudging the weights to reduce it.
Forward to get the answer, backward to learn from it.
What a neural network is for
A neural network is a function with adjustable numbers inside it. You give it an input, such as a camera image, and it returns an output, such as an action. At first the numbers are random, so the output is useless. Training adjusts the numbers, a little at a time, so that the outputs on example data get closer to the answers you want.
Two questions then matter. How does the network compute its output from an input? And how do we know which way to nudge each of its numbers, when there may be millions or billions of them? The first answer is the forward pass. The second is backpropagation.
Layers and activation functions
From M1.1, a linear layer computes y = Wx + b. A neural network stacks such layers, and puts a nonlinear activation function between them so the stack can represent more than a single linear map. Two common activations:
- ReLU
max(0, z): passes positive values through and sets negative values to zero. It is the usual default.- Sigmoid
1 / (1 + e−z): squashes any number into the range 0 to 1, useful for probabilities.
import numpy as np
def relu(z):
return np.maximum(0, z)
z = np.array([-2.0, -0.5, 0.0, 1.5])
print(relu(z))
print((1 / (1 + np.exp(-z))).round(3))Output:
[0. 0. 0. 1.5] [0.119 0.378 0.5 0.818]
A small network with one hidden layer, often called a multilayer perceptron (MLP), computes h = relu(W₁x + b₁) and then y = W₂h + b₂. The shapes follow the rule from M1.1, and the learned parameters are all the entries of W₁, b₁, W₂ and b₂.
rng = np.random.default_rng(0) W1, b1 = rng.normal(size=(4, 3)), np.zeros(4) # 3 inputs -> 4 hidden units W2, b2 = rng.normal(size=(2, 4)), np.zeros(2) # 4 hidden units -> 2 outputs x = np.array([1.0, 0.5, -1.0]) h = relu(W1 @ x + b1) y = W2 @ h + b2 print(h.shape, y.shape) print(W1.size + b1.size + W2.size + b2.size) # number of learned parameters
Output:
(4,) (2,) 26
Real models are the same idea at scale: the vision and language models inside a VLA have millions to billions of parameters, arranged in layers like these.
The loss: measuring how wrong the output is
To adjust the parameters we need a single number that says how bad the current output is. This is the loss. For predicting numbers, a common choice is the squared error between the prediction y and the target t:
The loss is zero when the prediction is exactly right and grows as it gets worse. Training means finding parameters that make the loss, averaged over many examples, small. Different tasks use different losses, and M2.5 returns to this.
The gradient says which way to move
The gradient of the loss with respect to a parameter says how the loss changes if that parameter increases slightly. If it is negative, increasing the parameter lowers the loss. Gradient descent moves every parameter a small step against its gradient:
The learning rate sets the step size. Here is the smallest possible network, one weight and one bias, taken through one complete step. The prediction is y = w·x + b, and by the chain rule the gradients of (y − t)² are 2(y − t)·x for w and 2(y − t) for b:
w, b, x, t = 0.5, 0.0, 2.0, 3.0 y = w * x + b loss = (y - t) ** 2 dw = 2 * (y - t) * x # d loss / d w db = 2 * (y - t) # d loss / d b print(y, loss, dw, db) lr = 0.05 w, b = w - lr * dw, b - lr * db y2 = w * x + b print(round(w, 2), round(b, 2), round((y2 - t) ** 2, 2))
Output:
1.0 4.0 -8.0 -4.0 0.9 0.2 1.0
The prediction started at 1.0 against a target of 3.0, giving a loss of 4.0. Both gradients were negative, so both parameters increased, and the loss fell from 4.0 to 1.0 in a single step. Repeating this is training. The calculus behind “how the loss changes” is covered in M1.2.
Backpropagation: the chain rule through the layers
With many layers, the loss depends on an early weight only through a long chain of later computations. Backpropagation computes every gradient efficiently by applying the chain rule backwards:
- Forward pass. Compute the output layer by layer, and keep the intermediate values.
- Compute the loss from the output and the target.
- Backward pass. Start from the gradient of the loss with respect to the output. Move back one layer at a time, using the stored values to turn the gradient at a layer’s output into the gradient at its input, and into gradients for that layer’s parameters.
- Update each parameter with its gradient.
Each layer only needs a simple local rule. The ReLU rule is worth remembering: its gradient is 1 where the input was positive and 0 where it was not, so a unit that was switched off passes no gradient back. The next example works this through for a tiny two-layer network and checks the result against a numerical estimate that does not use backpropagation (change a weight by a tiny amount and watch the loss):
x, t = np.array([1.0, 2.0]), -6.0
W1 = np.array([[0.5, -1.0],
[1.0, 0.5]])
w2 = np.array([1.0, -2.0])
def loss_fn(W1, w2):
return (w2 @ np.maximum(0, W1 @ x) - t) ** 2
# forward pass, keeping intermediate values
hp = W1 @ x # hidden pre-activation
h = np.maximum(0, hp) # hidden activation (ReLU)
y = w2 @ h
print(hp, h, y, (y - t) ** 2)
# backward pass
dy = 2 * (y - t) # d loss / d y
dw2 = dy * h # gradient for the output weights
dh = dy * w2 # gradient arriving at the hidden activation
dhp = dh * (hp > 0) # ReLU passes gradient only where hp > 0
dW1 = np.outer(dhp, x) # gradient for the first-layer weights
print(dw2)
print(dW1)
# check against a numerical estimate
eps = 1e-6
num = np.zeros_like(W1)
for i in range(2):
for j in range(2):
Wp = W1.copy(); Wp[i, j] += eps
num[i, j] = (loss_fn(Wp, w2) - loss_fn(W1, w2)) / eps
print(np.allclose(dW1, num, atol=1e-3))Output:
[-1.5 2. ] [0. 2.] -4.0 4.0 [0. 8.] [[ 0. 0.] [ -8. -16.]] True
The first hidden unit had a negative pre-activation (−1.5), so ReLU switched it off, and every gradient into its row of W1 is zero. The second unit was active, so its weights receive gradient. The numerical check agrees with the backward pass, which is how implementations of backprop are tested.
You will not write this by hand
PyTorch performs the backward pass for you. Recording the forward computation and calling .backward() gives the same gradients as the hand-derived ones, for networks of any size.
import torch x_t = torch.tensor([1.0, 2.0]) W1_t = torch.tensor([[0.5, -1.0], [1.0, 0.5]], requires_grad=True) w2_t = torch.tensor([1.0, -2.0], requires_grad=True) loss = (w2_t @ torch.relu(W1_t @ x_t) + 6.0) ** 2 loss.backward() print(W1_t.grad) print(w2_t.grad)
Output:
tensor([[ 0., 0.],
[ -8., -16.]])
tensor([0., 8.])These match dW1 and dw2 above. Understanding the hand version still pays off: it explains why a unit can stop learning, why gradients can shrink in very deep networks, and why the forward pass has to remember its intermediate values, which is a large part of the memory a training run uses.
Common misconceptions
Misconception 1 “Backpropagation is a different learning algorithm from gradient descent.”
Correction Backpropagation only computes the gradients. Gradient descent, or a variant, uses them to update the weights. They work together.
Misconception 2 “The network adjusts all its weights by the same amount.”
Correction Each parameter has its own gradient and moves according to it. Some move a lot, some barely, and some, behind a switched-off ReLU, not at all for that example.
Misconception 3 “A higher learning rate always trains faster.”
Correction A step that is too large can overshoot and make the loss rise. The learning rate has to be chosen carefully.
Misconception 4 “Autograd means I do not need to understand backpropagation.”
Correction Autograd does the arithmetic, but knowing what it computes helps you understand why training stalls, diverges or runs out of memory.
Engineering takeaways
- A network is stacked linear layers with nonlinear activations, such as ReLU, between them.
- The parameters are all the weights and biases, and training changes them.
- The loss is a single number that measures how wrong the output is.
- A gradient says how the loss changes when a parameter increases, and gradient descent steps against it.
- Backpropagation applies the chain rule backwards, using values saved in the forward pass.
- A ReLU unit with a negative input passes no gradient back.
- Finite differences give a numerical check on gradients.
- PyTorch autograd computes the same gradients automatically with
backward().
Backpropagation is the chain rule, applied layer by layer, so that every weight learns from the same loss.
Knowledge check
Try each question first, then reveal the explanation.
Question 1
For the single neuron in the lesson, the gradient of the loss with respect to w was −8. What does a negative gradient tell you about how to change w, and what does the update do?
Explanation
It tells you that increasing w lowers the loss. The update w − lr × gradient subtracts a negative number, so w increases. In the example it moved from 0.5 to 0.9.
Question 2
A ReLU hidden unit has a negative input for a particular example. What gradient flows back through it for that example, and why?
Explanation
Zero. The ReLU output is 0 whatever small change is made to its input, so the loss does not respond to that unit, and its gradient is 0. The weights feeding it receive no update from this example.
Question 3
Why does the forward pass have to keep its intermediate values until the backward pass?
Explanation
Because the backward pass needs them to compute the gradients. For example, the gradient for a weight uses the layer’s input, and the ReLU rule uses the sign of its pre-activation. Storing them is a large part of the memory a training run needs.
What’s next
You now know how a network computes an output and how backpropagation finds the gradients. Next we put the forward pass, loss, backward pass and update together into a training loop.