M5.6 — Basic Feedback Control

Learning objectives

After this lesson, you should be able to:

• Distinguish open-loop from feedback control.
• Explain proportional control and why too high a gain is unstable.
• Explain what the derivative term adds for a joint with inertia.
• Explain steady-state error and how the integral term removes it.
• Write the PID law and describe each term.
• Explain how delay can turn a stable loop into an oscillating one.

The one-sentence idea

A feedback controller measures the error between target and actual position and commands a correction in proportion to it, to its history and to its rate of change.

Measure, compare, correct, repeat.

Why feedback

Suppose you tell a joint to move to 1.0 rad. If you just send a fixed command and hope, you are running open loop: any error, a heavier load, friction, a slightly wrong model, goes uncorrected. Feedback control closes the loop. It measures where the joint actually is, computes the error (target minus measurement), and uses the error to decide the next command. Nearly every motor on every robot runs a feedback loop like this, usually at hundreds or thousands of times per second.

This matters for learned policies because they sit on top of these loops. A VLA outputs joint or hand targets (M5.5), and a feedback controller makes the joints actually go there. The policy decides where. The controller makes the motors get there despite loads and disturbances. The site’s PID tuning lab and its oscillation lab let you try the ideas below hands-on. Here they are worked through with code.

Proportional control

The simplest law is proportional (P) control: the command is a constant Kp times the error. A big error gives a big push and a small one a gentle push. For a joint whose velocity we can command directly, with new commands every dt seconds:

import numpy as np

def simulate_p(Kp, dt=0.1, steps=8, target=1.0):
    q, history = 0.0, []
    for _ in range(steps):
        error = target - q
        q = q + Kp * error * dt            # velocity command = Kp * error
        history.append(round(q, 3))
    return history

print(simulate_p(2.0))
print(simulate_p(25.0))

[0.2, 0.36, 0.488, 0.59, 0.672, 0.738, 0.79, 0.832]
[2.5, -1.25, 4.375, -4.062, 8.594, -10.391, 18.086, -24.629]

With a gain of 2, the joint approaches the target smoothly: each step closes a fixed fraction of the remaining error. With a gain of 25 the correction per step is bigger than the error itself, so the joint overshoots, then overshoots the other way by more, and the error grows every step. The loop has become unstable. In this simple discrete loop, the rule is that each step multiplies the error by 1 − Kp dt, which shrinks only if Kp dt is between 0 and 2. A higher gain is not better, and the safe limit depends on how often the loop runs.

Real joints have inertia: add damping

A real joint has mass and responds to a command with acceleration, not velocity. Now a pure P controller behaves like a spring: it accelerates the joint toward the target, but nothing slows it, so it swings past and keeps oscillating. The fix is a derivative (D) term, a command proportional to the joint’s velocity, opposing it. Together they form PD control, u = Kp · error − Kd · velocity:

def simulate_mass(Kp, Kd, Ki=0.0, disturbance=0.0, dt=0.01, T=10.0, target=1.0):
    q, v, integral, peak = 0.0, 0.0, 0.0, 0.0
    for _ in range(int(T / dt)):
        error = target - q
        integral += error * dt
        u = Kp * error - Kd * v + Ki * integral          # control force
        v += (u + disturbance) * dt                      # unit mass: acceleration = force
        q += v * dt
        peak = max(peak, q)
    return round(max(0.0, peak - target), 3), round(target - q, 4)

print(simulate_mass(Kp=16, Kd=0))       # P only: overshoot, final error
print(simulate_mass(Kp=16, Kd=8))       # PD

(1.0, -0.6838)
(0.0, 0.0)

With P alone the mass overshoots the target by 1.0, as far as the whole distance it travelled, and it never settles: the printed final error of −0.68 is simply where it happens to be in its endless swing. Adding damping removes the overshoot, and the mass settles on the target. In each result the first number is how far the mass went past the target, and the second is the error left at the end.

The derivative term also has a cost. It uses the velocity or the rate of change of the error, and as M5.1 showed, velocity computed from noisy position readings is very noisy. Real controllers filter the derivative or use a velocity measurement directly.

Steady-state error and the integral term

A joint carries a load, and gravity pulls with a roughly constant force. With a constant disturbance, a P or PD controller settles where its push exactly cancels it, and that requires a nonzero error, because the push is proportional to the error. This leftover is the steady-state error, and it equals the disturbance divided by Kp. The integral (I) term accumulates the error over time, so any lasting error keeps increasing the command until it vanishes:

d = -1.0                                           # a constant pull against the motion, like gravity
print(simulate_mass(Kp=16, Kd=8, disturbance=d))
print(simulate_mass(Kp=16, Kd=8, Ki=10, disturbance=d))

(0.0, 0.0625)
(0.164, 0.0)

The PD controller ends 0.0625 short of the target, which is exactly |d| / Kp = 1 / 16. Adding the integral term brings the final error to zero, at the price of an overshoot of 0.164, about 16% of the step. That is the usual trade: the integral term removes lasting error but adds overshoot, and a larger Ki would overshoot more. Together the three terms give PID control:

Read the terms as: P responds to the present error, I to the accumulated past error, and D to how fast the error is changing, a prediction of the near future. The common tuning intuitions: raising K_p speeds the response but risks oscillation; raising K_d adds damping; raising K_i removes lasting error but can cause overshoot, and an integral that keeps growing while a joint is blocked (“windup”) needs to be limited.

Delay makes feedback harder

Feedback assumes the measurement is current and the command acts immediately. A delay, from sensing, computation or communication, means the controller reacts to stale information, and a gain that was perfectly stable can start to oscillate. This is the same trade-off as before in a different guise, and it is directly relevant when a large model, which is slow, tries to produce commands:

def simulate_delayed(Kp, delay_steps, dt=0.1, steps=30, target=1.0):
    q = 0.0
    queue = [0.0] * delay_steps                    # commands still on their way to the motor
    out = []
    for _ in range(steps):
        queue.append(Kp * (target - q))            # command computed from the current error
        q += queue.pop(0) * dt                     # the motor receives an old command
        out.append(q)
    return round(max(out), 2), round(out[-1], 2)

print(simulate_delayed(Kp=4.0, delay_steps=0))
print(simulate_delayed(Kp=4.0, delay_steps=6))

(1.0, 1.0)
(3.12, -1.18)

With no delay, a gain of 4 reaches the target (peak equal to the target, 1.0, and it ends at 1.0). With the command arriving six steps late, the same gain makes the joint swing far past the target, up to 3.1, and then back below zero (−1.2 at step 30). The swings keep growing: the loop is unstable. The practical lesson for robot learning is that an action sent to a controller should be treated as a target for a fast, local loop that corrects continuously, and that the slow policy shouldn’t try to close that fast loop itself. This division of labour, learned policy above and classical control below, is how most systems are built, and it returns in Phase 9, where safety monitors and fallback behaviour are added around the same structure.

Common misconceptions

Engineering takeaways

1. Feedback uses the measured error to correct motion, and open loop does not.
2. P control commands Kp × error. Too high a gain, or too slow a loop, becomes unstable.
3. A joint with inertia needs a D term to damp oscillation.
4. A constant disturbance leaves a steady-state error of about |d| / Kp under P or PD control.
5. The I term accumulates error and removes it, with possible overshoot and windup.
6. PID is Kp e + Ki ∫e dt + Kd de/dt.
7. Delay reduces the stable range of gains and can cause oscillation.
8. Learned policies set targets, and fast local controllers track them.

The policy chooses where to go, and feedback is what keeps the motor honest on the way there.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

That completes the robotics prerequisites. The last module covers robot learning: supervised learning, imitation and the vocabulary of policies.

Back to the VLA roadmap prerequisites