Robustness of Suboptimal Model Predictive Control

James B. Rawlings
Department of Chemical and Biological Engineering
University of WisconsinMadison

Insitut f
ur Systemtheorie und Regelungstechnik
Universitat Stuttgart
Stuttgart, Germany
June 24, 2011

Stuttgart June 2011

Distributed MPC

1 / 50

Outline
1

Inherent robustness of suboptimal MPC

Exponential stability for difference inclusions
Nominal exponential stability of suboptimal MPC
Robust exponential stability of suboptimal MPC
Feasibility issue
Main results
Robust stability in absence of state constraints

Proposal for nonlinear, distributed MPC

Model
Objective Function
Terminal Controller
Removing the terminal constraint
Cooperative control algorithm
Stability of distributed nonlinear cooperative control
Example

Stuttgart June 2011

Distributed MPC

2 / 50

Why study robustness of suboptimal MPC?

Cooperative, distributed MPC is a special case of suboptimal MPC.
Anything we establish about suboptimal MPC can be applied to
cooperative, distributed MPC (and optimal MPC!)
Suboptimal MPC has an interesting feature: a nonunique,
point-to-set control law u N (x).
Optimal solution of nonconvex
PN (x) :

min VN (x, u)

uUN

cannot be computed online for any nonlinear model. Practitioners

implement only suboptimal MPC.
We should know something about its inherent robustness properties.1

Pannocchia, Rawlings, and Wright (2011)

Stuttgart June 2011

Distributed MPC

3 / 50

For suboptimal MPC; again, the basic MPC setup

The system model

x + = f (x, u)

(1)

State and input constraints

x(k) X ,

u(k) U

for all k I0

Terminal constraint (and penalty)

(N; x, u) Xf X

Stuttgart June 2011

Distributed MPC

4 / 50

Feasible sets
The set of feasible initial states and associated control sequences
ZN = {(x, u) | u(k) U, (k; x, u) X for all k I0:N1 ,
and (N; x, u) Xf }
and Xf X is the feasible terminal set.
The set of feasible initial states is
XN = {x Rn | u UN such that (x, u) ZN }

(2)

For each x XN , the corresponding set of feasible input sequences is

UN (x) = {u | (x, u) ZN }

Stuttgart June 2011

Distributed MPC

5 / 50

Cost function and control problem

For any state x Rn and input sequence u UN , we define

VN (x, u) =

N1
X

`((k; x, u), u(k)) + Vf ((N; x, u))

k=0

`(x, u) is the stage cost; Vf (x(N)) is the terminal cost

Consider the finite horizon optimal control problem
PN (x) :

Stuttgart June 2011

min VN (x, u)

uUN

Distributed MPC

6 / 50

Suboptimal MPC

Rather then solving PN (x) exactly, we consider using any

(unspecified) suboptimal algorithm having the following properties.
Let u UN (x) denote the (suboptimal) control sequence for the
denote a warm start for the successor initial
initial state x, and let u
+
state x = f (x, u(0; x)), obtained from (x, u) by
:= {u(1; x), u(2; x), . . . , u(N 1; x), u+ }
u

(3)

u+ U is any input that satisfies the invariance conditions of

Assumption 3 for x = (N; x, u) Xf , i.e., u+ f ((N; x, u)).

Stuttgart June 2011

Distributed MPC

7 / 50

Suboptimal MPC
UN (x + ).
The warm start satisfies u
The suboptimal input sequence for any given x + XN is defined as
any u+ UN that satisfies:
u+ UN (x + )

(4a)

)
VN (x + , u+ ) VN (x + , u
VN (x + , u+ ) Vf (x + )

(4b)
when x + r B

(4c)

in which r is a positive scalar sufficiently small that r B Xf .

Notice that constraint (4c) is required to hold only if x + r B, and it
implies that |u+ | 0 as |x + | 0.
Condition (4b) ensures that the computed suboptimal cost is no
larger than that of the warm start.

Stuttgart June 2011

Distributed MPC

8 / 50

Suboptimal MPC

Any u0 (x + ), optimal solution to PN (x + ), satisfies conditions (4a),

(4b) for all x + XN . Moreover, the inequality in condition (4c) is
satisfied by u0 (x + ) for all x + Xf .
Therefore, for any x + XN , there exists a u+ UN (x + ) satisfying all
UN (x + ).
conditions (4) for all u
We now observe that u+ is a set-valued map of the state x + , and so
is the associated first component u(0; x + ).
If we, again, denote the latter map as N (), we can write the
evolution of the closed-loop system as the following difference
inclusion:
x + {f (x, u) | u N (x)}

Stuttgart June 2011

Distributed MPC

(5)

9 / 50

Inherent robustness of the suboptimal controller

Consider a process disturbance d, x + = f (x, (x)) + d

A measurement disturbance xm = x + e
Nominal controller with disturbance
x + f (x, N (xm )) + d
x + f (x, N (x + e)) + d
x + Fed (x)

(6)

Robust stability; is the system x + Fed (x) input-to-state stable

considering (d, e) as the input.

Stuttgart June 2011

Distributed MPC

10 / 50

Basic MPC assumptions

Assumption 1 (Continuity of system and cost)

The functions f : Rn Rm Rn , ` : Rn Rm R0 and
Vf : Rn R0 are continuous, f (0, 0) = 0, `(0, 0) = 0, and Vf (0) = 0.

Assumption 2 (Properties of constraint sets)

The set U is compact and contains the origin. The sets X and Xf are
closed and contain the origin in their interiors, Xf X.
The set U is compact and contains the origin. The sets X = Rn and
Xf = lev Vf = {x Rn | Vf (x) }, with > 0.

Stuttgart June 2011

Distributed MPC

11 / 50

Basic MPC assumptions

Assumption 3 (Basic stability assumption)
For any x Xf there exists u := f (x) U such that f (x, u) Xf and
Vf (f (x, u)) + `(x, u) Vf (x).

Assumption 4
There exist positive constants a, a10 , a20 , af and r , such that the cost
function satisfies the inequalities
`(x, u) a10 |(x, u)|a
VN (x, u) a20 |(x, u)|a
Vf (x) af |x|a

Stuttgart June 2011

for all (x, u) X U

for all (x, u) r B
for all x X

Distributed MPC

12 / 50

Definition 5 (Exponential stability)

The origin of the difference inclusion z + H(z) is exponentially stable
(ES) on Z, 0 Z, if there exist scalars b > 0 and 0 < < 1, such that
for any z Z, all solutions (k; z) satisfy:
(k; z) Z,

|(k; z)| bk |z|

Stuttgart June 2011

for all k I0

Distributed MPC

13 / 50

Exponential stability for difference inclusions

Definition 6 (Exponential Lyapunov function)

V is an exponential Lyapunov function on the set Z for the difference
inclusion z + H(z) if there exist positive scalars a, a1 , a2 , a3 such that
the following holds for all z Z
a1 |z|a V (z) a2 |z|a ,

Stuttgart June 2011

max V (z + ) V (z) a3 |z|a

z + H(z)

Distributed MPC

14 / 50

Nominal exponential stability

Theorem 7 (ES)
Under Assumptions 1, 2.1, 3, and 4, the origin of the closed-loop system
x + F (x) = {f (x, u) | u N (x)}
is ES on (arbitrarily large) compact subsets of XN .

Stuttgart June 2011

Distributed MPC

15 / 50

Robust exponential stability of suboptimal MPC

Definition 8 (RES)
The origin of the closed-loop system (6) is robustly exponentially stable
(RES) on int(XN ) if there exist scalars b > 0 and 0 < < 1 such that for
all compact sets C XN , with 0 int(C), the following property holds:
Given any  > 0, there exists > 0 such that for all sequences {d(k)} and
{e(k)} with x(0) = x C satisfying:
max |d(k)| , max |e(k)|
k0

k0

xm (k) = x(k) + e(k) XN , x(k) XN , for all k I0 ,

it follows that
|ed (k; x)| bk |x| + , for all k I0 .

Stuttgart June 2011

Distributed MPC

(8)

16 / 50

Robust exponential stability of suboptimal MPC

Definition 9 (SRES)
The origin of the closed-loop system (6) is strongly robustly exponentially
stable (SRES) on a compact set C XN , 0 int(C), if there exist scalars
b > 0 and 0 < < 1 such that the following property holds: Given any
 > 0, there exists > 0 such that for all sequences {d(k)} and {e(k)}
satisfying
|d(k)| and |e(k)| for all k I0 ,
and all x C, we have that
xm (k) = x(k) + e(k) XN , x(k) XN , for all k I0 ,

(9a)

|ed (k; x)| bk |x| + , for all k I0 .

(9b)

Stuttgart June 2011

Distributed MPC

17 / 50

Feasibility issue
is feasible for the predicted successor state
The warm start u
x+ = f (xm , u(0; xm ))
) ZN ,
i.e., (
x +, u
But it may not be feasible for the measured successor state
+
xm
= f (x, u(0; xm )) + d + e +

(The true successor state, which is unknown, is

x + = f (x, u(0; xm )) + d.)
So we have to resolve this infeasibility online. We make the following
assumption

Assumption 10
For any x, x 0 XN and u UN (x), there exists u0 UN (x 0 ) such that
|u u0 | (|x x 0 |) for some K-function ().
Stuttgart June 2011

Distributed MPC

18 / 50

Feasibility issue

Among various options for finding p, we consider the following feasibility

problem (notice that x+ is known):
+
+
+ p UN (xm
Find p s.t. u
) and |p| (|xm
x+ |),

Proposition 11
+ X , the set of
) ZN and xm
Under Assumption 10, for any (
x +, u
N
solutions to (10) is nonempty.

Stuttgart June 2011

Distributed MPC

19 / 50

Main results

Theorem 12 (SRES of suboptimal MPC)

Under Assumptions 1, 2.1, 3, 4, and 10, the origin of the perturbed
closed-loop system
x + Fed (x)
is SRES on C .

Stuttgart June 2011

Distributed MPC

20 / 50

(10)

Main results
u
S
S

ZN

UN

u0(x)

XN

Figure: Sketch of the main sets involved in SRES.

Stuttgart June 2011

Distributed MPC

21 / 50

Robust stability in absence of state constraints

To this aim, we replace Assumption 2.1 with 2.2.

Assumption 10 will not be necessary, whereas Assumptions 1, 3 and 4
(with VN () replaced by VN () later defined) are required.
We modify the cost function as follows:
VN (x, u)

N1
X

`((k; x, u), u(k)) + Vf ((N; x, u))

k=0

Stuttgart June 2011

Distributed MPC

22 / 50

Modified suboptimal MPC algorithm in absence of state

constraints

u+ UN

(11a)

)
VN (x + , u+ ) VN (x + , u

(11b)

VN (x + , u+ ) Vf (x + )

when x + r B

(11c)

,
Zr = {(x, u) Rn UN | VN (x, u) V
and VN (x, u) Vf (x) if x r B}
X0 = {x Rn |u UN such that (x, u) Zr }

Stuttgart June 2011

Distributed MPC

(12)

23 / 50

Nominal exponential stability

We have the following nominal stability result.

Theorem 13 (ES without state constraints)

Under Assumptions 1, 2.2, 3, and 4, the origin of the closed-loop system
x + {f (x, u) | u N (x)}
is ES on X0 .

Stuttgart June 2011

Distributed MPC

24 / 50

Robust stability in absence of state constraints

+ +
+
),
Ged (z) = {u+ | u+ UN , VN (xm
, u ) VN (xm
,u
+ +
+
+
VN (xm
, u ) Vf (xm
) if xm
r B}

(zm ) = max V (z)

V
N
N
eB

s.t. z = zm (e, 0)

(13a)

(zm ) V
}
S = {zm Zr | V
N

(13b)

C = {x Rn | x = xm e, e B, u s.t. (xm , u) S }

Stuttgart June 2011

Distributed MPC

(14)

25 / 50

Robust stability in absence of state constraints

Theorem 14 (SRES without state constraints)

Under Assumptions 1, 2.2, 3 and 4, the origin of the closed-loop system
x + Fed (x)
is SRES on C .

Stuttgart June 2011

Distributed MPC

26 / 50

The robust region of attraction

The ideal, but unachievable, result would be that the robust region of
attraction under nonzero disturbances is the entire nominal feasible
set, XN .
This ideal result is approached reasonably closely, however, when
excluding state constraints!
When |d|, |e| 0, C X0 and SRES holds over a set approaching
the admissible set of initial conditions.
These results for robustness of suboptimal nonlinear MPC all apply to
cooperative, distributed MPC.

Stuttgart June 2011

Distributed MPC

27 / 50

Requirements for distributed, nonlinear control

Now lets return to the setting of distributed, nonlinear MPC, and

deal with the nonconvexity issue.
Two criteria in design:
1
2

the optimizers should not rely on a central coordinator

the exchange of information between the subsystems and the iteration
of the subsystem optimizations should be able to terminate before
convergence without compromising closed-loop properties.

Stuttgart June 2011

Distributed MPC

28 / 50

Distributed nonconvex optimization

Consider the optimization

min V (u)
u

s.t.

uU

We require approximate solutions to the following suboptimizations at

iterate p 0 for all i I1:M
p
u pi = arg min V (ui , ui
)
ui Ui

in which ui = (u1 , . . . , ui1 , ui+1 , . . . , uM ).

Define the step ip = u pi uip .

Stuttgart June 2011

Distributed MPC

29 / 50

Algorithm

To choose the stepsize ip , each suboptimizer initializes the stepsize2

with i
p
V (u p ) V (uip + ip ip , ui
) ip i V (u p )0 ip

in which (0, 1).

After all suboptimizers finish the backtracking process, they exchange
steps. Each suboptimizer forms a candidate step
uip+1 = uip + wi ip ip

i I1:M

Armijo rule: (Bertsekas, 1999, p.230)

Stuttgart June 2011

Distributed MPC

30 / 50

Algorithm
Check the following inequality, which tests if V (u p ) is convex-like
X
p
V (u p+1 )
wi V (uip + ip ip , ui
)
(15)
iI1:M

in which

iI1:M

wi = 1 and wi > 0 for all i I1:M .

If the condition above is not satisfied, then we find the direction with
the worst cost improvement
p
imax = arg max{V (uip + ip ip , ui
)}
i

and eliminate this direction by setting wimax to zero and repartitioning

the remaining wi so that they sum to 1.
At worst, condition (15) is satisfied with one direction only.

Stuttgart June 2011

Distributed MPC

31 / 50

Distributed nonconvex optimization Properties

Lemma 15 (Feasibility)
Given a feasible initial condition, the iterates u p are feasible for all p 0.

Lemma 16 (Objective decrease)

The objective function decreases at every iterate, that is,
V (u p+1 ) V (u p ).

Lemma 17 (Convergence)
Every accumulation point of the sequence {u p } is stationary.

Stuttgart June 2011

Distributed MPC

32 / 50

Distributed nonconvex optimization

4

u2

0
0

2
u1

Figure: Nonconvex function optimized with Distributed nonconvex optimization

algorithm
Stuttgart June 2011

Distributed MPC

33 / 50

Distributed nonlinear cooperative control

We have the following local models
x1+ = f1 (x1 , x2 , u1 , u2 )

x2+ = f2 (x1 , x2 , u1 , u2 )

(16)

Collect these models to form the plantwide model

x + = f (x1 , x2 , u1 , u2 ) = f (x, u)
in which
 
x
x= 1
x2

 
u
u= 1
u2



f1 (x1 , x2 , u1 , u2 )
f (x, u) =
f2 (x1 , x2 , u1 , u2 )

for which x Rn , u Rm , and f : Rn Rm Rn .

At each time step k, we require the inputs to satisfy
u1 (k) U1

u2 (k) U2

k I0:N1

in which each Ui Rmi is compact, convex, and contains the origin

in its interior.
Stuttgart June 2011

Distributed MPC

34 / 50

Distributed nonlinear cooperative control

The objective function for each subsystem i I1:2 is defined

Vi x(0), u1 , u2 =

N1
X

`i xi (k), ui (k) + Vif x(N)

k=0

We define plantwide objective

V x1 (0), x2 (0), u1 , u2 = 1 V1 x(0), u1 , u2 + 2 V2 x(0), u1 , u2
To simplify notation we use V (x, u) for the plantwide objective.

Assumption 18
For each i I1:2 , there exists a K function i () such that
`i (xi , ui ) i (|xi |)
Stuttgart June 2011

(xi , ui ) Rni Ui
Distributed MPC

(17)
35 / 50

Distributed nonlinear cooperative control

Denote the plantwide terminal penalty Vf (x) = 1 V1f (x) + 2 V2f (x).
We define the terminal region Xf to be a sublevel set of Vf . For
a > 0, define
Xf = {x | Vf (x) a}

Assumption 19
The plantwide terminal penalty Vf () satisfies
f (|x|) Vf (x) f (|x|) x Xf
in which f () and f () are K functions.

Stuttgart June 2011

Distributed MPC

36 / 50

Distributed nonlinear cooperative control

Defining `(x, u) = 1 `1 (x1 , u1 ) + 2 `2 (x2 , u2 ), we require the following
stability assumption.

Assumption 20
The terminal cost Vf () satisfies
n


min
Vf f (x, u1 , u2 ) + `(x, u)
(u1 ,u2 )U1 U2
o
s.t. f (x, u1 , u2 ) Xf Vf (x)

x Xf

This assumption implies that there exists a if (x) Ui for all i I1:2
such that




Vf f (x, 1f (x), 2f (x)) + ` x, 1f (x), 2f (x) Vf (x)
(18)
s.t. f (x, 1f (x), 2f (x)) Xf
Stuttgart June 2011

Distributed MPC

37 / 50

Removing the terminal constraint

To avoid coupled input constraints, must remove the terminal state
constraint.
For some 1, define the modified objective function

V (x, u) =

N1
X

`(x(k), u(k)) + Vf (x(N))

(19)

k=0

Define the set of admissible initial (x, u) pairs as

Z0 = {(x, u) X UN | V (x, u) V , (N; x, u) Xf }
The set of initial states X0 is the projection of Z0 onto X
X0 = {x X | u such that (x, u) Z0 }

Stuttgart June 2011

Distributed MPC

38 / 50

(20)

Distributed nonlinear cooperative control

Proposition 21 (Terminal constraint satisfaction)

Let {(x(k), u(k)) | k I0 } denote the set of states and control
sequences generated by the suboptimal system. There exists a > 1 such
that for all , if (x(0), u(0)) Z0 , then (x(k), u(k)) Z0 with
(N; x(k), u(k)) Xf for all k I0 .
For the remainder of the talk, we replace the plantwide objective with the
modified objective V () V () and hence the terminal constraint is
satisfied.

Stuttgart June 2011

Distributed MPC

39 / 50

Cooperative control algorithm

U be the initial condition for the cooperative MPC algorithm

Let u
) Xf .
such that (N; x(0), u
At each iterate p, an approximate solution of the following
optimization problem is found


min V x1 (0), x2 (0), u1 , u2
u

s.t.

x1+ = f1 (x1 , x2 , u1 , u2 )

(21b)

x2+ = f2 (x1 , x2 , u1 , u2 )

(21c)

ui UN
i

i I1:2

|ui | i (|xi (0)|)

Stuttgart June 2011

(21a)

if x(0) Br

Distributed MPC

(21d)
i I1:2

40 / 50

(21e)

Cooperative control algorithm

Let the input sequence returned by distributed nonconvex

).
optimization algorithm be up (x, u
) is injected
The first input of this sequence p (x(0)) = u p (0; x(0), u
into the plant and the state is moved forward.
To reinitialize the algorithm at the next sampling time, we define the
warm start


+
u
=
{u
(1),
u
(2),
.
.
.
,
u
(N

1),

x(N)
}
1
1
1
1f
1


+
u
2 = {u2 (1), u2 (2), . . . , u2 (N 1), 2f x(N) }
in which x(N) = (N; x(0), u1 , u2 ).

Stuttgart June 2011

Distributed MPC

41 / 50

Nominal stability results

Theorem 22 (Asymptotic stability)

Let Assumptions 18-20 hold and let V () V () by Proposition 21.
Then for every x(0) XN , the origin is asymptotically stable for the
closed-loop system x + = f (x, p (x)).

Stuttgart June 2011

Distributed MPC

42 / 50

A nonlinear example
Consider the unstable nonlinear system
x1+ = x12 + x2 + u13 + u2
x2+ = x1 + x22 + u1 + u23
with initial condition (x1 , x2 ) = (3, 3).
For this example, we use the stage cost
1
`1 (x1 , u1 ) = (x10 Q1 x1 + u10 R1 u1 )
2
1
`2 (x2 , u2 ) = (x20 Q2 x2 + u20 R2 u2 )
2
For the simulation we choose the parameters
Q=I

R=I

N = 2 p = 3 Ui = [2.5, 2.5] i I1:2

Stuttgart June 2011

Distributed MPC

43 / 50

Distributed nonlinear cooperative control

4
4

x1
x1

x2

x2

2
1
1
x
x

0
-1
-1
-2
-2
-3
-3
-4
0

-4
0

10

Figure: State trajectory (p = 3)

Stuttgart June 2011

Figure: Centralized state trajectory

(p = 10)

Distributed MPC

44 / 50

10

Distributed nonlinear cooperative control

3
3
2
2

u2

u2

u
u

u1

u1

0
-1
-1
-2
-2
-3
0

-3
0

10

Figure: Input trajectory (p = 3)

Stuttgart June 2011

Figure: Centralized input trajectory

(p = 10)

Distributed MPC

45 / 50

Distributed nonlinear cooperative control

104

p = 3
p = 10

10

100
V (x(k), u(k))

102
104
106
108

1010

10

k
Figure: Open-loop cost to go versus time on the closed-loop trajectory for
different numbers of iterations.
Stuttgart June 2011

Distributed MPC

46 / 50

10

Distributed nonlinear cooperative control

1
0.5
0
-0.5
u2 -1
-1.5
-2
-2.5
-3
-3

-2

-1

u1
Figure: Contours of V with N = 1 for k = 0 with (x1 (0), x2 (0)) = (3, 3).
Iterations of the subsystem controllers with initial condition (u10 , u20 ) = (0, 0).
Stuttgart June 2011

Distributed MPC

47 / 50

Distributed nonlinear cooperative control

1
Xt
0.5
Xf
0

x2

-0.5

-1

-1

-0.5

0
x1

0.5

Figure: Terminal region. Xt are the points in which the terminal controller is
stabilizing and Xf = {x | Vf (x) 0.485} Xt is the terminal region.
Stuttgart June 2011

Distributed MPC

48 / 50

Further Reading I

D. P. Bertsekas. Nonlinear Programming. Athena Scientific, Belmont,

MA, second edition, 1999.
G. Pannocchia, J. B. Rawlings, and S. J. Wright. Conditions under which
suboptimal nonlinear MPC is inherently robust. Sys. Cont. Let., 2011.
Accepted for publication.

Stuttgart June 2011

Distributed MPC

49 / 50

Acknowledgments

The author is indebted to Luo Ji of the University of Wisconsin and

Ganzhou Wang of the Universitat Stuttgart for their help in organizing the
material and preparing the overheads.

Stuttgart June 2011

Distributed MPC

50 / 50