M1-T2 — Nested Loops and Separation of Timescales

Learning objectives

After this lesson, you should be able to:

• Order the loops of a vehicle by bandwidth and say which signal links each pair.
• Distinguish a loop's bandwidth from its update rate.
• Compute the phase lag an inner loop adds to an outer loop from the ratio of their bandwidths.
• Predict the overshoot of a two-loop cascade from that ratio, and state the usual minimum ratio.

The intuition

The loop of M1-T1: Three Questions: Where Am I, Where Should I Go, How Do I Get There hides a ladder. Guidance does not talk to the engine. It hands a command to a loop beneath it, which hands a command to the next, and so on down to the actuator. Hopper's chain for holding a position on the range runs like this:

1. Guidance gives a position command to the lateral-position loop.
2. That loop turns the position error into a velocity command for the lateral-velocity loop.
3. That loop turns the velocity error into a pitch command, because tilting the thrust is how a rocket accelerates sideways.
4. The pitch-angle loop turns the pitch error into a pitch-rate command.
5. The pitch-rate loop turns the rate error into a gimbal command.
6. The gimbal actuator moves the engine.

Each loop makes sure the signal it is given is delivered, and each is designed to be quicker than the loop above it. The reason is easiest to see from the point of view of one loop. The loop above sends a command and watches the result. If the loop beneath needs ten seconds to carry out an instruction but the one above checks every two, the upper loop concludes that nothing is happening and piles on more instruction. When the lower loop finally acts, it carries out the whole pile, and the vehicle overshoots. Make the lower loop fast compared with the upper one and the problem disappears: the upper loop sees what it asked for almost at once.

The rule of thumb

Designers usually make each loop 3 to 10 times faster than the loop it serves, and Hopper's ratios (3 to 4) sit at the low end of that range. The lower end saves hardware and noise, because a faster loop amplifies more sensor noise and asks more of the actuator. The upper end makes the design easy: the outer loop can be tuned as if the inner one were not there. The derivation below shows where the number comes from and why it is a rule of thumb and not a law.

Bandwidth is not update rate

Two different things are called speed here, and mixing them up is a common source of mistakes.

• The bandwidth of a closed loop is how fast the loop can follow a changing command. A loop with a bandwidth of 4 rad/s follows commands that change over about a second and ignores much faster ones. It is a property of the dynamics and the gains.
• The update rate is how often the computer recomputes the loop's output. It is a property of the software and the hardware.

The update rate must be much higher than the bandwidth, or the sampling itself becomes a lag inside the loop. A common guide is a sample rate (in rad/s, that is 2π times the rate in hertz) of 10 to 20 times the loop's bandwidth. Hopper's pitch-rate loop has a bandwidth of 12 rad/s (about 2 Hz) and runs at 200 Hz, 100 times faster than it needs.

Navigation sits inside every loop

The last row of the table matters. The estimate $\hat{x}$ feeds the controllers, so any lag in producing it is a lag inside their loops. The navigation block therefore follows the same rule as the controllers: it must respond faster than any loop that uses its output. A filter that smooths heavily, or one that updates a few times per second, can be the slowest element in a fast loop without anyone noticing. Hopper's filter propagates the estimate with the IMU at 200 Hz and corrects it whenever a beacon fix arrives.

The cost of separation

The ratios multiply. Six loops with a ratio of about 3.3 each span a factor of 400, from 0.1 to 40 rad/s. The outermost loop sets how fast the whole vehicle can change what it is doing, so Hopper's guidance lives on a timescale of about ten seconds. If you want a vehicle that reacts faster, every loop beneath guidance must speed up too, and sensor noise, actuator limits and the update rate put a ceiling on how far that can go. Choosing the bandwidths of a vehicle is a budget of speed.

The UAV track shows the symptoms of a planner that outruns its controller, see UAV M5-T15: Guidance and control integration, and the cascade of PID loops is in UAV M5-T6. This lesson adds the quantity those pages leave out: how much faster the inner loop must be.

Definitions

Closed-loop bandwidth $\omega_b$

The highest frequency, in rad/s, at which the closed loop still follows a sinusoidal command with at least 70% of its amplitude. A frequency in hertz is $f = \omega / 2\pi$. The next module defines it from the frequency response.

Update rate $f_s$ and period $T$

How often a computed loop produces a new output, in hertz, and its inverse $T = 1/f_s$ in seconds.

Inner and outer loop

The inner loop is the one that carries out the outer loop's command. Their bandwidths are $\omega_i$ and $\omega_o$.

Separation ratio $r$

The ratio of the two: $r = \omega_i / \omega_o$. The rule of thumb asks for $r$ between 3 and 10.

The simplest model of a loop that is closed and well behaved is a first-order lag. Its response to a command is a smooth rise with time constant $1/\omega_i$.

At a sinusoid of angular frequency $\omega$ this lag passes only part of the amplitude and delays it. The part passed and the delay (a phase lag, in degrees or radians) are:

Now close an outer position loop around it. The plant of the outer loop is a velocity that is integrated into a position, and the outer loop commands velocity in proportion to the position error:

Finally, a computer that holds its output constant between updates delays it on average by half a period. At the frequency $\omega$ of a loop that is a further phase lag:

Derivation: why the ratio matters, and the number behind it

The cascade above has two states and one command. Its closed loop is a second-order system, and the question is how its damping depends on the ratio $r$. Work with the four lines of the cascade.

1. $$L(s) = \frac{K\,\omega_i}{s\,(s + \omega_i)}$$

   The open loop is the gain $K$, then the inner loop $\omega_i/(s + \omega_i)$ from command to velocity, then the integrator $1/s$ from velocity to position.

2. $$1 + L(s) = 0 \;\Rightarrow\; s^2 + \omega_i\,s + K\,\omega_i = 0$$

   The closed-loop poles are the roots of $1 + L$. Clearing the denominator gives a quadratic.

3. $$\omega_n = \sqrt{K\,\omega_i}, \qquad \zeta = \frac{\omega_i}{2\,\omega_n} = \frac12\sqrt{\frac{\omega_i}{K}} = \frac12\sqrt{r}$$

   Compare with the standard form $s^2 + 2\zeta\omega_n s + \omega_n^2$ to read off the natural frequency and the damping ratio. The damping ratio depends on the separation ratio and on nothing else.

4. $$\zeta \ge 1 \iff r \ge 4, \qquad M_p = e^{-\pi\zeta/\sqrt{1-\zeta^2}}\ \ (\zeta < 1)$$

   A damping ratio of 1 or more means no overshoot, which needs a ratio of 4. Below that the step response overshoots by the peak $M_p$, a standard result for second-order systems.

5. $$\zeta = \tfrac{1}{\sqrt{2}} \iff r = 2, \qquad M_p = e^{-\pi} \approx 4.3\%$$

   At a ratio of 2 the overshoot is already down to 4.3%, which most designs accept.

6. $$\varphi\!\left(\omega_o\right) = -\arctan\!\left(\frac{K}{\omega_i}\right) = -\arctan\!\left(\frac1r\right)$$

   The phase lag the inner loop adds at the outer loop's bandwidth, from the second definition above. For $r = 3$ it is 18.4 degrees, for 5 it is 11.3 and for 10 it is 5.7.

Two things follow. First, the idealized floor is about 2: with a first-order inner loop and nothing else, a ratio of 2 gives acceptable overshoot, and a ratio below 1 is a bad design. Second, the real floor is higher. The derivation counted only the inner loop's lag. A real loop also has an actuator lag, a transport delay, sampling (the formula for $\varphi_{\text{sample}}$) and the lag of the navigation filter, and each takes phase away from the same budget. A ratio of 3 to 10 leaves room for them. That is why the rule of thumb is a range and not a number.

A fast inner loop is not a luxury. It is what lets the outer loop be designed as if the inner one were not there, and the lag it still adds, arctan(1/r), is the price of that simplification.

A worked example

Take the cascade above with an outer gain $K = 2$ per second, a position step of 1 m, and four different inner loops. Predict the overshoot from the formula, then simulate the cascade and compare.

1. Fix the outer loop. $K = 2\ \text{s}^{-1}$ sets the outer bandwidth. It is the same in all four cases.
2. Choose four inner bandwidths: 20, 8, 2 and 0.5 rad/s. The separation ratios $r = \omega_i / K$ are 10, 4, 1 and 0.25.
3. Predict. From $\zeta = \tfrac12\sqrt{r}$: 1.58, 1.00, 0.50 and 0.25. The first two do not overshoot. The third overshoots by $e^{-\pi \cdot 0.5/\sqrt{0.75}} = 16.3\%$ and the fourth by 44.4%. The phase lag at $K$ is $\arctan(K/\omega_i)$, which is 5.7, 14.0, 45.0 and 76.0 degrees.
4. Simulate. Step the cascade with a fourth-order Runge-Kutta integrator and a step of 1 ms, and record the peak position above 1 m.
5. Compare. The simulation agrees with the formula to the digit shown, because for this linear second-order system the formula is exact. A ratio of 10 gives a sluggish, over-damped loop with almost no lag. A ratio of 4 is critically damped. A ratio of 1 is the case of a designer who sets both loops to the same speed: 16% overshoot. At 0.25 the outer loop is four times faster than the loop that carries out its commands, and the vehicle overshoots by almost half the step.

The same experiment as a program that uses only the standard library:

import math

K = 2.0   # outer (position) loop gain in 1/s: its bandwidth if the inner loop were instant

def overshoot(w_inner, K, t_end=15.0, dt=0.001):
    """Unit step. Plant: x' = v. Inner velocity loop: v' = w_inner * (v_cmd - v). Outer loop: v_cmd = K * (1 - x)."""
    def f(x, v):
        return v, w_inner * (K * (1.0 - x) - v)
    x = v = peak = 0.0
    for _ in range(int(t_end / dt)):
        k1 = f(x, v)
        k2 = f(x + 0.5 * dt * k1[0], v + 0.5 * dt * k1[1])
        k3 = f(x + 0.5 * dt * k2[0], v + 0.5 * dt * k2[1])
        k4 = f(x + dt * k3[0], v + dt * k3[1])
        x += dt / 6 * (k1[0] + 2 * k2[0] + 2 * k3[0] + k4[0])
        v += dt / 6 * (k1[1] + 2 * k2[1] + 2 * k3[1] + k4[1])
        peak = max(peak, x)
    return max(0.0, peak - 1.0)

print(' ratio  zeta  lag at K  overshoot (formula)  overshoot (simulated)')
for w_inner in (20.0, 8.0, 2.0, 0.5):
    r = w_inner / K
    zeta = 0.5 * math.sqrt(r)
    formula = math.exp(-math.pi * zeta / math.sqrt(1 - zeta ** 2)) if zeta < 1 else 0.0
    lag = math.degrees(math.atan(K / w_inner))
    print(f'{r:6.2f} {zeta:6.2f} {lag:6.1f} deg {100 * formula:14.1f} % {100 * overshoot(w_inner, K):18.1f} %')

 ratio  zeta  lag at K  overshoot (formula)  overshoot (simulated)
 10.00   1.58    5.7 deg            0.0 %                0.0 %
  4.00   1.00   14.0 deg            0.0 %                0.0 %
  1.00   0.50   45.0 deg           16.3 %               16.3 %
  0.25   0.25   76.0 deg           44.4 %               44.4 %

In robotics

Every flight controller you will meet is built as a cascade, and its documentation names the loops in the order of the ladder.

• Multicopters. PX4 and ArduPilot run an angular-rate loop inside an attitude loop inside a velocity and position loop, with mission logic on top. The controller diagrams in the PX4 documentation show the chain, and UAV M5-T6: PID controllers tunes it loop by loop. Tuning guides always say the same thing: tune the innermost loop first, and only then the one above it.
• Aircraft. An autopilot is closed by successive loop closure: a pitch-rate or yaw-damper loop first, then the attitude loops, then airspeed, altitude and path on top. Each outer loop is designed treating the inner one as a unity gain, which is the assumption of this lesson (module 10).
• Spacecraft. The separation can be extreme. A satellite's attitude loop settles in seconds while its orbit changes over minutes to hours, so guidance and attitude control are effectively decoupled (module 11).
• Estimators. A navigation filter has a timescale too. If an estimator smooths a measurement for a second, then any control loop with a bandwidth near 1 rad/s that uses its output has the smoothing lag inside the loop. That is why the filter's rate, and the latency from sensor to estimate, appear on every timing budget (see UAV M7-T6: End-to-end latency).

The practical use of the rule is in the other direction too. When a vehicle oscillates, a quick check is to list the loop bandwidths and look for a neighbouring pair with a ratio below about 3. Oscillation at a frequency near the crossover of an outer loop usually means that the loop beneath it is too slow, or that something (a filter, a delay, a saturated actuator) has made it so.

Common misconceptions

Key takeaways

1. A vehicle is a cascade of loops: each loop's output is the command of the loop beneath it, down to the actuator.
2. Each loop is designed 3 to 10 times faster than the loop it serves, so that the outer loop sees the inner one as an almost instant servant.
3. For a first-order inner loop under an integrating outer loop the damping ratio is $\zeta = \tfrac12\sqrt{r}$: a ratio of 4 gives no overshoot, a ratio of 2 about 4%, a ratio of 1 about 16%.
4. At the outer loop's bandwidth the inner loop adds a phase lag of $\arctan(1/r)$, which is 18 degrees at a ratio of 3.
5. Bandwidth (how fast the closed loop responds) is not update rate (how often the computer runs it). The rate must be much higher.
6. The navigation filter is inside every loop that uses its estimate, so it must be faster than all of them.

References and further reading

Each source is marked free or paid. Free to read does not mean free to reuse: the course resources page lists every source with its licence.

• R. W. Beard and T. W. McLain, Small Unmanned Aircraft: Theory and Practice, Princeton University Press, 2012, the chapter on autopilot design by successive loop closure. Paid book; the authors' slides and code are free.
• K. J. Åström and R. M. Murray, Feedback Systems, 2nd edition, Princeton University Press, 2021 , the chapters on transfer functions and on frequency-domain design, for the second-order step response. Free to read as the authors' PDF; the book is copyrighted.
• B. Douglas, “Classical Control Theory” (video series) . Short lectures that prepare module 3. Free to watch.
• PX4 documentation, “Controller Diagrams” . The cascade of a fielded autopilot, drawn out. Free documentation.
• MATLAB Tech Talks, “Control Systems in Practice” (video series) . Free to watch.

Knowledge check

Three questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

Related content

Nothing below is required to finish this lesson.

• Before
  M1-T1: Three Questions: Where Am I, Where Should I Go, How Do I Get There

  Define navigation, guidance and control by the question each answers, and name the signals that pass between them.

• Lab
  M1-L1: Who Missed the Pad?

  Hopper lands 4 m from its pad in three different flights. From the logged truth, estimate and command signals, decide in each flight whether navigation, guidance or control is responsible.

Other courses on this site treat neighbouring topics from their own angle:

• UAV M5-T15: Guidance and control integration
• UAV M1-T12: Complete UAV system architecture and data flow
• Navigation M1-T2: The Navigation Stack at a Glance
• Physical AI: The Physical AI stack

This course leaves the following to the pages that already teach them:

• The ground-robot navigation stack (costmaps, global and local planners)
• UAV hardware and companion-computer architecture