M4-T5 — LQG and Why the Guarantees Disappear

Learning objectives

After this lesson, you should be able to:

• Assemble an LQG controller from an LQR gain and a Kalman gain and write its transfer function from measurement to input.
• Explain why LQR and Kalman margins do not carry over to the combination.
• Measure the gain range and peak sensitivity of an LQG loop.
• Apply loop-transfer recovery and state its price.

The intuition

UAV M3-T4: Kalman Filter estimated the state and UAV M5-T7: LQR control regulated it. Putting one behind the other is the obvious design, and it is called LQG: linear model, quadratic cost, Gaussian noise. The preceding lessons of this module gave each half a strong property. The regulator has a return difference of at least 1 at the plant input. The Kalman filter, by duality, has the same property at the plant output. It is natural to expect the combination to be robust. It is not.

Why the guarantee is lost

The regulator margin is for the loop cut at the plant input with the true state fed back. Once an observer is inserted the controller is the estimator and the gain together: it takes the measurement and produces the input through a dynamic system of its own. The loop that matters is now measurement to input to plant to measurement, and its return difference is no longer the one the Riccati equation constrained. The theory says only that the poles are right; the margins are free to be anything.

How bad can it be

Doyle's example is a two-state plant with the weights made large together. As they grow, the LQG loop stays stable for a range of plant gain that shrinks to a point: a change of a fraction of a percent in the gain destabilizes it. The loop is optimal for the model and has no robustness to anything else. On Hopper, with only the beacon position measured, the Kalman observer designed with plausible noise levels leaves a gain range of ×0.69 to ×1.5 and a peak sensitivity of 4.8, where LQR alone allowed ×0.19 to ×1000 with a peak of 1.0.

The repair

The remedy is loop-transfer recovery. The observer is designed on purpose for a plant that is noisier than the real one: extra fictitious noise is injected at the plant input, which the filter reads as an unknown input and must be fast enough to track. As the fictitious noise grows the observer becomes faster and its dynamics disappear from the loop in the limit, which restores the return difference of the regulator. On Hopper, a noise factor of 10 restores a range of ×0.23 to ×10.8 and a peak of 1.20. The price is a high observer gain, and with it more amplification of the real sensor noise.

The separation principle is about poles and says nothing about margins. An optimal estimator and an optimal regulator can make a fragile loop. Check the margins of the combined loop at the plant input, always.

Definitions

LQG controller

The LQR gain $\mathbf{K}$ acting on the Kalman estimate $\hat{\mathbf{x}}$, which is formed from the measurement $y$ and the input $u$.

Controller $C(s)$ (measurement to input)

The transfer function from $y$ to $u$ of the compensator: $C(s) = \mathbf{K}(s\mathbf{I} - \mathbf{A} + \mathbf{B}\mathbf{K} + \mathbf{L}\mathbf{C})^{-1}\mathbf{L}$.

Loop gain $L(s) = G(s)C(s)$

Cut at the plant input: the plant followed by the compensator. Its return difference $1 + L$ is what the margins are read from.

Loop-transfer recovery

Design of the observer with fictitious process noise $\rho^2\mathbf{B}\mathbf{B}^{\mathsf{T}}$ at the plant input so that $L(s)$ approaches the loop gain of the regulator as $\rho \to \infty$.

Derivation: the separation principle, and what it leaves out

The poles of the combination are the union of the regulator poles and the observer poles. The margins are a separate matter.

1. $$\mathbf{e} = \mathbf{x} - \hat{\mathbf{x}}, \qquad \dot{\mathbf{e}} = (\mathbf{A} - \mathbf{L}\mathbf{C})\mathbf{e}$$

   The estimation error evolves on its own, whatever the input, because the observer sees the same input as the plant.

2. $$\dot{\mathbf{x}} = (\mathbf{A} - \mathbf{B}\mathbf{K})\mathbf{x} + \mathbf{B}\mathbf{K}\mathbf{e}$$

   Substitute $u = -\mathbf{K}(\mathbf{x} - \mathbf{e})$ into the plant. The error enters as a disturbance.

3. $$\begin{bmatrix}\dot{\mathbf{x}} \\ \dot{\mathbf{e}}\end{bmatrix} = \begin{bmatrix}\mathbf{A} - \mathbf{B}\mathbf{K} & \mathbf{B}\mathbf{K} \\ 0 & \mathbf{A} - \mathbf{L}\mathbf{C}\end{bmatrix}\begin{bmatrix}\mathbf{x} \\ \mathbf{e}\end{bmatrix}$$

   The system is block triangular, so its eigenvalues are those of the two diagonal blocks: the separation principle.

Why the margins are not separate

1. $$L(s) = \mathbf{K}(s\mathbf{I} - \mathbf{A} + \mathbf{B}\mathbf{K} + \mathbf{L}\mathbf{C})^{-1}\mathbf{L}\;\mathbf{C}(s\mathbf{I} - \mathbf{A})^{-1}\mathbf{B}$$

   Cut the loop at the plant input: the compensator $C(s)$ from measurement to input multiplied by the plant.

2. $$L(s) = \mathbf{K}(s\mathbf{I} - \mathbf{A})^{-1}\mathbf{B} \quad \text{if } \mathbf{e} \text{ were exactly zero}$$

   With perfect estimates the compensator would act as the plain state feedback and the loop would be the regulator's, with its guaranteed margins.

3. $$L(s) \to \mathbf{K}(s\mathbf{I} - \mathbf{A})^{-1}\mathbf{B} \quad \text{as } \rho \to \infty$$

   With fictitious input noise of intensity $\rho^2$ the Kalman gain tends to $\rho\,\mathbf{B}/\sqrt{V}$ in the input direction. The observer cancels the plant dynamics seen from $u$ and the loop gain tends to the regulator's. This holds for plants with no zeros in the right half-plane.

The LQG loop has the right poles by construction, but its return difference at the plant input is that of a dynamic compensator, which nothing constrains. Recovery restores the regulator's margins by making the observer fast enough to disappear, at the price of noise amplification.

A worked example

First Doyle's example, then Hopper with position measured alone.

1. Doyle. The plant $\dot{\mathbf{x}} = \begin{bmatrix}1&1\\0&1\end{bmatrix}\mathbf{x} + \begin{bmatrix}0\\1\end{bmatrix}u$, $y = x_1$, with LQR weight and Kalman noise both $q\begin{bmatrix}1&1\\1&1\end{bmatrix}$. The LQG loop is stable for a plant gain from ×0.923 to ×1.047 at $q = 1$ and from ×0.977 to ×1.000 at $q = 1000$: at $q = 10^4$ the window is a point.
2. Hopper, regulator alone. The LQR of the previous lesson is stable for gains from ×0.19 to beyond ×1000 and has a peak sensitivity of 1.00.
3. Hopper, with a Kalman observer. Only the beacon position is measured (noise variance 5·10⁻⁴ m²); the process noise acts on velocity and pitch rate with variance 0.01. The loop is stable only from ×0.69 to ×1.5, and the peak sensitivity rises to 4.76.
4. Recovery. Adding fictitious input noise with $\rho = 3, 10, 100$ widens the range to ×0.26 to ×6.8, ×0.23 to ×10.8 and ×0.20 to ×30.5, and lowers the peak sensitivity to 1.34, 1.20 and 1.07.
5. The price. The compensator gain at high frequency grows with $\rho$: the sensor noise reaches the gimbal through a larger gain. The choice of $\rho$ is a compromise between margin and noise, made in the lab M4-L2: The Observer That Spoils the Loop.

import cmath, math
import gnckit as g

def lqg_matrix(A, B, C, K, L, k=1.0):
    """Plant state and estimate, with the plant input gain scaled by k (the estimator does not know about it)."""
    n = len(A)
    M = [[0.0] * (2 * n) for _ in range(2 * n)]
    for i in range(n):
        for j in range(n):
            M[i][j] = A[i][j]
            M[i][n + j] = -k * B[i] * K[j]
            M[n + i][j] = L[i] * C[j]
            M[n + i][n + j] = A[i][j] - B[i] * K[j] - L[i] * C[j]
    return M

def gain_range(A, B, C, K, L):
    ks = [10 ** (i / 200) for i in range(-400, 400)]
    ok = [k for k in ks if max(complex(p).real for p in g.mat_eigvals(lqg_matrix(A, B, C, K, L, k))) < 0]
    return (min(ok), max(ok)) if ok else None

print("Doyle's example: x' = [[1,1],[0,1]] x + [0,1] u, y = x1; LQR weight q [[1,1],[1,1]], Kalman noise q [[1,1],[1,1]]")
A, B, C = [[1.0, 1.0], [0.0, 1.0]], [0.0, 1.0], [1.0, 0.0]
for q in (1, 10, 100, 1000, 10000):
    K = g.lqr(A, [[v] for v in B], [[q, q], [q, q]], [[1.0]])[0][0]
    L = [row[0] for row in g.kalman_gain(A, [C], [[q, q], [q, q]], [[1.0]])[0]]
    lo, hi = gain_range(A, B, C, K, L)
    print(f'  q = {q:5d}: stable only for plant gain between x{lo:.3f} and x{hi:.3f}')

print('Hopper, lateral channel, position measured by the beacons only:')
gr, lam, b = 9.81, 0.7, 19.62
A = [[0, 1, 0, 0], [0, 0, gr, 0], [0, 0, 0, 1], [0, 0, lam ** 2, 0]]
B = [0, gr, 0, b]
C = [1, 0, 0, 0]
K = g.lqr(A, [[v] for v in B], [[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 25, 0], [0, 0, 0, 4]], [[1 / 0.15 ** 2]])[0][0]
W0 = [[0, 0, 0, 0], [0, 0.01, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0.01]]
ws = g.logspace(-2, 3, 3000)

def loop(L, w):
    Acc = [[A[i][j] - B[i] * K[j] - L[i] * C[j] for j in range(4)] for i in range(4)]
    return g.ss_freq(Acc, L, K, 0.0, w) * g.ss_freq(A, B, C, 0.0, w)       # controller y -> u times plant u -> y

def report(name, L):
    lo, hi = gain_range(A, B, C, K, L)
    print(f'  {name:34s} stable for gain x{lo:.2f} to x{hi:.1f}, peak |S| = {max(1 / abs(1 + loop(L, w)) for w in ws):.2f}')

ks = [10 ** (i / 100) for i in range(-300, 301)]
ok = [k for k in ks if max(complex(p).real for p in g.mat_eigvals(g.ss_closed_loop(A, B, [k * c for c in K]))) < 0]
Lq = lambda w: g.ss_freq(A, B, K, 0.0, w)
print(f'  {"LQR alone (full state known)":34s} stable for gain x{min(ok):.2f} up to the largest tried (x{max(ok):.0f}), peak |S| = {max(1 / abs(1 + Lq(w)) for w in ws):.2f}')
Lk = [row[0] for row in g.kalman_gain(A, [C], W0, [[5e-4]])[0]]
report('with the Kalman observer', Lk)
for rho in (3, 10, 100):
    W = [[W0[i][j] + rho ** 2 * B[i] * B[j] for j in range(4)] for i in range(4)]
    report(f'Kalman with input noise rho = {rho}', [row[0] for row in g.kalman_gain(A, [C], W, [[5e-4]])[0]])

Doyle's example: x' = [[1,1],[0,1]] x + [0,1] u, y = x1; LQR weight q [[1,1],[1,1]], Kalman noise q [[1,1],[1,1]]
  q =     1: stable only for plant gain between x0.923 and x1.047
  q =    10: stable only for plant gain between x0.923 and x1.023
  q =   100: stable only for plant gain between x0.944 and x1.000
  q =  1000: stable only for plant gain between x0.977 and x1.000
  q = 10000: stable only for plant gain between x1.000 and x1.000
Hopper, lateral channel, position measured by the beacons only:
  LQR alone (full state known)       stable for gain x0.19 up to the largest tried (x1000), peak |S| = 1.00
  with the Kalman observer           stable for gain x0.69 to x1.5, peak |S| = 4.76
  Kalman with input noise rho = 3    stable for gain x0.26 to x6.8, peak |S| = 1.34
  Kalman with input noise rho = 10   stable for gain x0.23 to x10.8, peak |S| = 1.20
  Kalman with input noise rho = 100  stable for gain x0.20 to x30.5, peak |S| = 1.07

In robotics

LQG-type designs fly in launchers, aircraft and spacecraft, which is why the weakness is known and managed.

• Always check the loop at the plant input. The margins of an observer-based controller are read from the loop broken at the actuator, as in the worked example. A design review that shows only the regulator and the filter separately has not shown stability margins.
• Recovery as a design knob. Raising $\rho$ until the margins are acceptable is routine. A designer reads the cost off the noise gain of the compensator at high frequency and stops when the two are balanced.
• Limits of recovery. It requires a plant without right half-plane zeros. A plant with one cannot be recovered fully, and the delay of M3-T6: Limits of Performance: Delay, Right-Half-Plane Zeros and the Bode Integral is the same obstacle: the margin limits there apply to LQG too.
• Beyond LQG. Robust methods (H-infinity, μ-synthesis, covered by the references) optimize a measure that includes the margin, and structured controllers (PID in the loop, lead-lag) are tuned by direct search against the margins. Module 5 gives the tools to measure them.
• Relation to the estimator you already have. The same Kalman filter fuses the sensors of M1-T1: Three Questions: Where Am I, Where Should I Go, How Do I Get There's navigation. Tuning it for navigation accuracy (the real noise levels) and tuning it for loop margins (fictitious noise) are different design problems and often have different answers, which is why flight code carries both.

The math behind the filter is in Probability for Robotics (module 7) and Linear Algebra M9-T7: Linear Kalman Filter Mathematics.

Common misconceptions

Key takeaways

1. LQG is the LQR gain applied to a Kalman estimate; its poles are those of the regulator and of the observer together.
2. The margins of the regulator and the filter do not carry over: the loop at the plant input is a different loop.
3. Doyle's example is stable only in a window of plant gain that shrinks to a point as the weights grow.
4. On Hopper, a plain Kalman observer reduces the gain range from ×0.19–×1000 to ×0.69–×1.5, with peak sensitivity 4.76.
5. Loop-transfer recovery adds fictitious input noise $\rho^2\mathbf{B}\mathbf{B}^{\mathsf{T}}$ to the filter design; $\rho = 10$ gives ×0.23–×10.8 and a peak of 1.20.
6. Recovery costs a high observer gain and more sensor noise at the actuator, and it needs a plant without right half-plane zeros.

References and further reading

Each source is marked free or paid. Free to read does not mean free to reuse: the course resources page lists every source with its licence.

• J. C. Doyle, "Guaranteed margins for LQG regulators", IEEE Transactions on Automatic Control 23(4), 1978, the one-page paper with the counterexample. Paid (IEEE); the abstract is free to read.
• MIT 16.323 Principles of Optimal Control (OpenCourseWare) , the lectures on the LQG and on loop-transfer recovery. Free; licence CC BY-NC-SA 4.0.
• K. J. Åström and R. M. Murray, Feedback Systems, 2nd edition, Princeton University Press, 2021 , the chapter on output feedback and the discussion of robustness of observer-based control. Free to read as the authors' PDF; the book is copyrighted.
• S. Skogestad and I. Postlethwaite, Multivariable Feedback Control, 2nd edition, Wiley, 2005, the chapters on LQG and on loop-transfer recovery. Paid.

Knowledge check

Three questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

Related content

Nothing below is required to finish this lesson.

• Before
  M4-T4: The Linear-Quadratic Regulator from Dynamic Programming

  Derive the Riccati equation from the principle of optimality, choose weights with Bryson's rule, and state the margins LQR guarantees.

• Lab
  M4-L2: The Observer That Spoils the Loop

  Pitch rate is no longer measured. Add an observer, find the observer bandwidth at which the margins collapse, and choose one that keeps them.

Other courses on this site treat neighbouring topics from their own angle:

• UAV M5-T7: LQR control
• UAV M5-L2: Implement LQR
• Navigation M5-T6: LQR and MPC for Path Tracking
• UAV M3-T4: Kalman Filter
• UAV M3-T12: Observability and consistency
• Linear Algebra M6-T1: Eigenvalues and Eigenvectors
• Linear Algebra M6-T4: Positive Definite Matrices
• Linear Algebra M9-T7: Linear Kalman Filter Mathematics
• Robot Learning M4-T4: Dynamic Programming and Value Iteration
• Probability for Robotics (module 7 derives the Kalman filter)

This course leaves the following to the pages that already teach them:

• The derivation of the Kalman filter
• LQR applied to a linearized quadrotor
• LQR applied to path tracking of a ground robot