14.An introduction to H2 optimal control

and, furthermore, exactly one of these solutions is positive-definite. The equation (14.10) is called the algebraic Riccati equation.

Proof Let

˜ ˜ ˜ ˜t −1 −t t

Σ = (A, b, c , 01) = (T AT , T b, T c , 01)

˜ −1

then P = T P T is a solution to (14.10). Therefore, without loss of generality we assume that (A, b) is in controller canonical form. Let TΣ(s) = ct(sIn − A)−1b be the transfer function for Σ with (N, D) its c.f.r. Then we have

TΣt (−s)TΣ(s) + 1 = bt(−sIn − A)cct(sIn − A)−1b + 1,

which is nonnegative for s = iω. Let us denote this rational function by R. Defining

and of degree n. Because A is in controller canonical form, a simple computation along the lines of that in the proof of Theorem 6.49 shows that if f Rn is defined by

Hurwitz. By Theorem 5.32(i) there exists a unique positive-definite solution to the equation

Let us denote this solution by P 1. A straightforward computation shows that the previous equation is equivalent to

Thus the existence part of the theorem will follow if we can show that P 1b = f. Let us show this.

The argument is outlined as follows:

2. let Q1 R[s] be defined by

btP 1(sIn − A)−1b = Q1(s);

D(s)

− ˜

3. let P = Q1 F + D;

4. substitute the above definitions into the equation ( ) and the resulting expression turns out to be

Thus we have shown that (14.10) has a solution, and the solution P 1 we found was positive-definite. Let us show that this is the only positive-definite solution. Let P 2 be positive-definite and suppose that

(At − fbt)P 2 + P 2(A − bft) = −cct − fft;

5. by Theorem 5.32(i), P 2 = P 1.

During the course of the proof of the theorem, we arrived at the relationship between the solution to the algebraic Riccati equation and the optimal state feedback vector f. Let us record this.

14.30 Corollary Consider a controllable SISO linear system Σ = (A, b, ct, 01). If f is the optimal state feedback vector of Corollary 14.26 and P is the unique positive-definite solution to the algebraic Riccati equation (14.10), then f = P b.

Proof By Corollary 14.26 the vector f = (f0, f1, . . . , fn−1) is defined by

fn−1sn−1 + · · · + f1s + f0 = [D(s)D(−s) + N(s)N(−s)]+ − D(s).

However, this is exactly the relation (14.11) in the proof of Theorem 14.29, since in the statement of the theorem we had assumed (A, b) to be in controller canonical form. During the course of the same proof, the relation f = P b was also shown to be true when (A, b) is in controller canonical form. The result now follows from the transformation property for

respectively. Let us denote by P 1 and P 2 the two corresponding positive-definite matrices guaranteed by Theorem 14.29. Referring to the equation (14.12) in the proof of Theorem 14.29, we see that the matrices P 1 and P 2 satisfy

(At − fjbt)P j + P j(A − bftj) = −cjctj − fjftj, j = 1, 2.

Now we refer to the proof of part (i) of Theorem 5.32 to see that

Z ∞

P j = eAtj tQjeAj t dt, j = 1, 2,

0

where

Aj = A − bftj, Qj = cjctj + fjftj, j = 1, 2.

One may do the integration (I used Mathematica®, of course) to obtain

In each case we readily verify that fj = P jb, j = 1, 2, just as predicted by Corollary 14.30.

14.32 Remarks 1. The algebraic Riccati equation must generally be solved numerically. Indeed, note that Theorem 14.25 provides essentially the same information as the algebraic Riccati equation (as made precise in Corollary 14.30). In the former case, we must find the left half-plane spectral factor of an even polynomial, and this involves finding the roots of this polynomial. This itself is something that typically must be done numerically.

2.Note that the matrix on the right-hand side of the algebraic Riccati equation is the matrix that determines the penalty given to states (as opposed to control) in the cost function of Problems 14.1 and 14.3. One can easily imagine using more general symmetric matrices to define this cost, thus looking at cost functions of the form

where Q Rn×n is symmetric and positive-semidefinite, and R > 0. This can also be seen to be generalisable to multiple inputs by making the cost associated to the input be of the form ut(t)Ru(t) for an m × m symmetric matrix Q. Making the natural extrapolation gives the analogue of equation (14.10) to be

AtP + P A − P BR−1BtP = −Q.

This is indeed the form of the algebraic Riccati equation that gets used in MIMO generalisations of our Theorem 14.25.

3. The optimal feedback vector f determined in this section is often referred to as the linear quadratic regulator (LQR).

14.3.3 Optimal state estimation results With the optimal control results of the preceding two sections, and with the (now known) solution of the model matching problem of Section 14.2.6, we can prove the following result which characterises the solution to the optimal state estimation problem.

14.33 Theorem For Σ = (A, b, ct, D) a complete SISO linear system, the following statements regarding ` Rn are equivalent:

˜ t˜ ˜ t ˜ −1 ˜

we have ` = T `. Now we note that (T A T , T c) is in controller canonical form if and

˜ −t ˜ t ˜ ˜ −t

only if (T AT , T c) is in observer canonical form. From this we infer that T = T . The

˜ ˜

equivalence of parts (ii) and (iii) now follow from the fact that (N, D) = (N, D).

22/10/2004 14.3 Solutions of optimal control and state estimation problems 549

We now show the equivalence of parts (i) and (iii). First we note that by Parseval’s

posed without changing anything. That is to say, we have

TΣ(s) = ct(sIn − A)−1b = bt(sIn − At)−1ct

TΣ` (s) = ct(sIn − A + `ct)−1` = `t(sIn − At + c`t)−1c.

We shall think of these transfer functions as being in this form for the moment. Now, thinking of TΣ` as the unknown, we wish to choose this unknown to minimise J(`). This, however, is exactly the model matching problem posed in Section 14.2.6. The solution to the model matching problem is given, if we know the solution to the optimal control problem Problem 14.3. But since we now know this, we do indeed know the solution to the model matching problem, and let us translate the solution into our current notation.

AP 22 + P 22At − P 22cctP 22 = −bbt.

Since P is symmetric and positive-definite, so too is P 22 (since xtP xt > 0 for all x of the form xt = [ 0 xt2 ]). The last of equations (14.18) then uniquely prescribes P 22 as being the matrix prescribed in part (iii) of the theorem. Adding the last two of equations (14.18) gives

A(P 12 + P 22) + (P 12 + P 22)At − P 22cct(P 12 + P 22) = 0.

This gives P 12 + P 22 = 0 by . We then have

and we take ` = `2. Now we obtain

T˜ (s) = −`t(sIn − At)−1c = −ct(sIn − A)−1`

Σ1

T˜ (s) = `t(sIn − At + c`t)−1c = ct(sIn − A + `ct)−1`,

Σ2

550 14 An introduction to H2 optimal control 22/10/2004

thus giving

(−1 + T˜ (s))T˜ (s) = (1 − ct(sIn − A + `ct)−1`)ct(sIn − A)−1`

Σ2 Σ1

=ct(sIn − A)−1(In − `ct(sIn − A + `ct)−1)`

=ct(sIn − A + `ct − `ct)(sIn − A + `ct)`

=ct(sIn − A + `ct)`,

14.4 The linear quadratic Gaussian controller

In Section 10.5.3 we showed how one could combine state estimation via a Luenberger observer with static state feedback to obtain a controller that worked by using the estimated states in the state feedback law.

14.4.1LQR and pole placement We know that if the state feedback vector f is chosen according to Corollary 14.26, then the matrix A − bft will be Hurwitz. However, we have said nothing about the exact nature of the eigenvalues of this matrix. In this section we address this issue.

14.4.2Frequency domain interpretations Our above presentations for the linear quadratic regulator and the optimal state estimator are presented in the time-domain. While we present their solutions in terms of spectral factorisation of polynomials, we also see that these solutions are obtainable using the algebraic Riccati equation, and this is how these problems are typically solved in the MIMO case. However, it is also possible to give frequency response formulations and solutions to these problems, and in doing so we make a connection to the H2 model matching problem of Section 14.2.6.

14.4.3H2 model matching and LQG In this section we show that LQG control may be posed as an H2 model matching problem. This will provide us with a natural segue to the next chapter where we discuss a more di cult model matching problem, that of H∞ model matching. By representing the somewhat easily understood LQG control in the context of model matching, we hope to motivate the less easily understood material in the next chapter.

14.5 Stability margins for optimal feedback

In this section we wish to investigate some properties of our optimal feedback law. We shall work in the setting of Corollary 14.26. It is our desire to talk about the gain and phase margin for the closed-loop system in this case. However, it is not quite clear that it makes sense to do this as gain and phase margins are defined in the context of unity gain feedback loops, not in the context of static state feedback. Therefore, the first thing we shall is recall from Exercise E7.11 the connection between static state feedback and unity gain feedback loop concepts. In particular, recall that A − bft is Hurwitz if and only if

This result tells us that closed-loop stability of the closed-loop system Σf is equivalent to IBIBO stability of a unity gain feedback loop with loop gain RL(s) = ft(sIn − A)−1b. Note that this enables us to employ our machinery for these interconnections, thinking of f(sIn − A)−1b as being the loop gain.

14.5.1 Stability margins for LQR In particular, as is made clear in Exercise E7.11, we may employ the Nyquist criterion to determine closed-loop stability under static state feedback. Recall that rather than the poles of the loop gain in C+, one uses the eigenvalues of A in C+ to compare with the encirclements of −1 + i0. Let us illustrate the Nyquist criterion on an unstable system.

indicates that any stabilising state feedback vector f will have the property that the Nyquist plot for the loop gain Rf (s) = ft(sIs − A)−1b will encircle the origin twice in the clockwise direction. Let us test this, not for just any stabilising state feedback vector, but for the optimal one. Without going through the details as we have done this for one example already, the optimal feedback state vector is

In Figure 14.2 is shown the Nyquist plot for the loop gain Rf (s) = ft(sI2 − A)−1b. One

Re

Figure 14.2 Nyquist plot for optimal state feedback with two unstable eigenvalues

The discussion to this point has largely been with respect to general feedback vectors. However, notice that the Nyquist plot of Example 14.34, done for the optimal state feedback vector of Corollary 14.26, has an interesting feature: the Nyquist contour remains well clear of the critical point −1 + i0. The following result tells us that we can generally expect this to happen when we use the optimal state feedback vector.

14.35 Theorem Let Σ = (A, b, ct, 01) be a controllable SISO linear system and let f Rn be the optimal state feedback vector of Corollary 14.26. We have

−1 − f(iωIn − A)−1b ≥ 1

for every ω R. In other words, the point Rf (iω) is at least distance 1 away from the point

−1 + i0.

Proof We assume without loss of generality that (A, b) is in controller canonical form. We let P be the unique positive-definite matrix guaranteed by Theorem 14.29. Thus

Now perform the following computations:

1.to (14.19), add and subtract iωP ;

2.multiply the resulting equation on the left by bt(−iωIn − At)−1 and on the right by (iωIn − A)−1b;

3.use the identity f = P b from Corollary 14.30.

The result of these computations is

1 + ft(iωIn − A)−1b 2 = 1 + bt(−iωIn − At)−1cct(iωIn − A)−1b.

Since the matrix cct is positive-semidefinite, we have

bt(−iωIn − At)−1cct(iωIn − A)−1b ≥ 0.

This interesting result tells us that the Nyquist contour remains outside the circle of radius 1 in the complex plane with centre −1 + i0. Thus, it remains well clear of the critical point.

14.36 Example (Example 14.28 cont’d) We resume looking at the system with

full knowledge of the state. Doyle [1978] shows that as soon as one tries to estimate the state from the output using a Kalman filter, the stability margin of Theorem 14.35 disappears. Although state estimation, and in particular Kalman filtering, is something we do not cover in this book, the reader should be aware of these lurking dangers.

14.6 Summary

554 14 An introduction to H2 optimal control 22/10/2004

k > 0.

(b)Now use the fact that in the statement of Theorem 14.25, Σ is said to be observable to show that limt→∞ x(t) = 0.

In Exercise E10.5 you provided a characterisation of a stabilising state feedback vector using a linear matrix inequality (LMI). In the following exercise, you will further characterise the optimal state feedback vector using LMI’s. The characterisation uses the algebraic Riccati equation of Theorem 14.29.

E14.6

E14.7 For the pendulum on a cart of Exercises E1.5 and E2.4, choose parameter values for the mass of the cart, the mass of the pendulum, the gravitational constant, and the length of the pendulum arm to be M = 112 , m = 1, g = 9.81, and ` = 12 . For each of the following linearisations:

(a)the equilibrium point (0, π) with cart position as output;

(b)the equilibrium point (0, π) with cart velocity as output;

(c)the equilibrium point (0, π) with pendulum angle as output;

(d)the equilibrium point (0, π) with pendulum angular velocity as output,

do the following:

1.construct the optimal state feedback vector of Problem 14.3;

2.compute the closed-loop eigenvalues;

3.plot a few trajectories of the full system with the control determined by the state feedback vector of part (1).

E14.8 For the double pendulum system of Exercises E1.6 and E2.5, choose parameter values for the first link mass, the second link mass, the first link length, and the second link length to be m1 = 1, m2 = 2, `1 = 12 , and `2 = 13 . For each of the following linearisations:

(a)the equilibrium point (0, π, 0, 0) with the pendubot input;

(b)the equilibrium point (π, 0, 0, 0) with the pendubot input;

(c)the equilibrium point (π, π, 0, 0) with the pendubot input;

(d)the equilibrium point (0, π, 0, 0) with the acrobot input;