Professional Documents
Culture Documents
i
to Maryan, Federica and Marina
and all our families
ii
Preface
Dynamic optimization has become a standard tool for decision making in a wide
range of areas. The search for the most fuel-efficient strategy to send a rocket into
orbit or the most economical way to start-up a chemical production facility can
be expressed as dynamic optimization problems that are solved almost routinely
nowadays.
The basis for these dynamic optimization problems is a dynamic model, for
example,
x(k + 1) = g(x(k), u(k)), x(0) = x0
that describes the evolution of the state x(k) with time, starting from the initial
condition x(0), as it is affected by the manipulated input u(k). Here g(x, u) is
some nonlinear function. Throughout the book we are assuming that this discretetime description, i.e., the model of the underlying system, is available. The goal
of the dynamic optimization procedure is to find the vector of manipulated inputs
UN = [u(0) , ..., u(N 1) ] such that the objective function is optimized over some
time horizon N , typically
minUN
N
1
X
k=0
The terms q(x, u) and and p(x) are referred to as the stage cost and terminal cost,
respectively. Many practical problems can be put into this form and many algorithms and software packages are available to determine the optimal solution vector
UN
, the optimizer. The various algorithms exploit the structure of the particular
problem, e.g., linearity and convexity, so that even large problems described by
complex models and involving many degrees of freedom can be solved efficiently
and reliably.
One difficulty with this idea is that, in practice, the sequence of u(0), u(1), ...,
which is obtained by this procedure cannot be simply applied. The model of the
system predicting its evolution is usually inaccurate and the system may be affected
by external disturbances that may cause its path to deviate significantly from the
one that is predicted. Therefore, it is common practice to measure the state after
some time period, say one time step, and to solve the dynamic optimization problem
again, starting from the measured state x(1) as the new initial condition. This
feedback of the measurement information to the optimization endows the whole
procedure with a robustness typical of closed-loop systems.
What we have described above is usually referred to as Model Predictive Control
(MPC), but other names like Open Loop Optimal Feedback and Reactive Scheduling have been used as well. Over the last 25 years MPC has evolved to dominate
the process industry, where it has been employed for thousands of problems [232].
The popularity of MPC stems from the fact that the resulting operating strategy
respects all the system and problem details, including interactions and constraints,
which would be very hard to accomplish in any other way.
Indeed, often MPC is used for the regulatory control of large multivariable
linear systems with constraints, where the objective function is not related to an
economical objective, but is simply chosen in a mathematically convenient way,
iii
namely quadratic in the states and inputs, to yield a good closed-loop response.
Again, there is no other controller design method available today for such systems,
that provides constraint satisfaction and stability guarantees.
One limitation of MPC is that running the optimization algorithm on-line at
each time step requires substantial time and computational resources. Today, fast
computational platforms together with advances in the field of operation
research and optimal control have enlarged in a very significant way
the scope of applicability of MPC to fast-sampled applications. One
approach is to use tailored optimization routines which exploit both the
structure of the MPC problem and the architecture of the embedded
computing platform to implement MPC in the order of milliseconds.
The second approach is to have the result of the optimization pre-computed
and stored for each x in the form of a look-up table or as an algebraic function
u(k) = f (x(k)) which can be easily evaluated. In other words, we want to determine
the (generally nonlinear) feedback control law f (x) that generates the optimal
u(k) = f (x(k)) explicitly and not just implicitly as the result of an optimization
problem. It requires the solution of the Bellman equation and has been a long
standing problem in optimal control. A clean simple solution exists only in the case
of linear systems with a quadratic objective function, where the optimal controller
turns out to be a linear function of the state (Linear Quadratic Regulator, LQR).
For all other cases a solution of the Bellman equation was considered prohibitive
except for systems of low dimension (2 or 3), where a look-up table can be generated
by gridding the state space and solving the optimization problem off-line for each
grid point.
The major new contribution of this book is to show how the nonlinear optimal feedback controller can be calculated efficiently for some important classes of
systems, namely linear systems with constraints and switched linear systems or,
more generally, hybrid systems. Traditionally, the design of feedback controllers for
linear systems with constraints, for example anti-windup techniques, was ad hoc
requiring both much experience and trial and error. Though significant progress
has been achieved on anti-windup schemes over the last decade, these techniques
deal with input constraints only and cannot be extended easily.
The classes of constrained linear systems and linear hybrid systems treated in
this book cover many, if not most, practical problems. The new design techniques
hold the promise to lead to better performance and a dramatic reduction in the
required engineering effort.
The book is structured in four parts. Chapters marked with a star () are
considered advanced material and can be skipped without compromising the
understanding of the fundamental concepts presented in the book.
In the first part of the book (Part I) we recall the main concepts and results of
convex and discrete optimization. Our intent is to provide only the necessary
background for the understanding of the rest of the book. The material of this
part has been extracted from the following books and lecture notes: Convex optimization by Boyd and Vandenberghe [61], Nonlinear Programming
Theory and Algorithms by Bazaraa, Sherali, Shetty [24], LMIs in Control
by Scherer and Weiland [248] and Lectures on Polytopes by Ziegler [287].
iv
Continuous problems as well as integer and mixed-integer problems are presented in Chapter 1. Chapter 2 discusses the classical results of Lagrange
duality. Since polyhedra are the fundamental geometric objects used in this
book, in Chapter 5 we introduce the main definitions and the algorithms,
which describe standard operations on polyhedra. Part I closes with Chapter 3 where Linear and Quadratic programs are presented together with their
properties and some fundamental results.
The second part of the book (Part II) is a self-contained introduction to
multi-parametric programming. In our framework, parametric programming
is the main technique used to study and compute state feedback optimal
control laws. In fact, we formulate the finite time optimal control problems
as mathematical programs where the input sequence is the optimization vector. Depending on the dynamical model of the system, the nature of the
constraints, and the cost function used, a different mathematical program is
obtained. The current state of the dynamical system enters the cost function
and the constraints as a parameter that affects the solution of the mathematical program. We study the structure of the solution as this parameter changes
and we describe algorithms for solving multi-parametric linear, quadratic and
mixed integer programs. They constitute the basic tools for computing the
state feedback optimal control laws for these more complex systems in the
same way as algorithms for solving the Riccati equation are the main tools for
computing optimal controllers for linear systems. In Chapter 6 we introduce
the concept of multiparametric programming and we recall the main results
of nonlinear multiparametric programming. Then, in Chapter 7 we describe
three algorithms for solving multiparametric linear programs (mp-LP), multiparametric quadratic programs (mp-QP) and multiparametric mixed-integer
linear programs (mp-MILP).
In the third part of the book (Part III) we introduce the general class of
optimal control problems studied in the book. Chapter 8 contains the basic
definitions and essential concepts. Chapter 9 presents standard results on
Linear Quadratic Optimal Control while in Chapter 10 unconstrained optimal
control problems for linear systems with cost functions based on 1 and
norms are analyzed.
In the fourth part of the book (Part IV) we focus on linear systems with
polyhedral constraints on inputs and states. We study finite time and infinite time optimal control problems with cost functions based on 2, 1 and
norms. In Chapter 12 we show that the solution to all these optimal
control problems can be expressed as a piecewise affine state feedback law.
Moreover, the optimal control law is continuous and the value function is
convex and continuous. The results form a natural extension of the theory
of the Linear Quadratic Regulator to constrained linear systems. Chapter 12
also presents a self-contained introduction to invariant set theory as well as
dynamic programming for constrained systems.
Chapter 13 presents the concept of Model Predictive Control. Classical feasibility and stability issues are shown through simple examples and explained
v
by using invariant set methods. Finally we show how they can be solved with
a proper choice of the terminal constraints and the cost function.
Chapter 16 addresses the robustness of the optimal control laws. We discuss
min-max control problems for uncertain linear systems with polyhedral constraints on inputs and states and present an approach to compute their state
feedback solutions. Robustness is achieved against additive norm-bounded
input disturbances and/or polyhedral parametric uncertainties in the statespace matrices.
The result in Chapter 12 have important consequences for the implementation of MPC laws. Precomputing off-line the explicit piecewise affine feedback
policy reduces the on-line computation for the receding horizon control law to
a function evaluation, therefore avoiding the on-line solution of a mathematical program. In Chapter 15 the evaluation of the piecewise affine optimal
feedback policy is carefully studied and we present algorithms to reduce its
storage demands and computational complexity.
In the fifth part of the book (Part V) we focus on linear hybrid systems. We
give an introduction to the different formalisms used to model hybrid systems
focusing on computation-oriented models (Chapter 17). In Chapter 18 we
study finite time optimal control problems with cost functions based on 2, 1
and norms. The optimal control law is shown to be, in general, piecewise
affine over non-convex and disconnected sets. Along with the analysis of the
solution properties we present algorithms that efficiently compute the optimal
control law for all the considered cases.
Berkeley
Lucca
Zurich
Francesco Borrelli
Alberto Bemporad
Manfred Morari
April 2012
vi
Acknowledgements
Large parts of the material presented in this book is extracted from the work of the
authors Francesco Borrelli, Alberto Bemporad and Manfred Morari. Several sections have also been extracted from the PhD Theses of Miroslav Baric, Mato Baotic,
Fabio Torrisi, Domenico Mignone, Pascal Grieder, Eric Kerrigan, Tobias Geyer and
Frank J. Christophersen. A special thanks goes to Michal Kvasnica, Stefan Richter,
Valerio Turri and Thomas Besselmann for their help on the construction of the examples. A special thanks goes to Martin Herceg for his meticulous work on the
figures.
Our gratitude also goes to the several people who have carefully read preliminary versions of the book and gave us suggestions on how to improve them.
They include David Mayne, Per Olof Gutman, Diethard Klatte, Paul Goulart, Bill
Levine and all the students of our class on Model Predictive Control (ME290J at
UC Berkeley, 227-0221-00 at ETH).
The authors of the LaTex book style files and macros are Stephen Boyd and
Lieven Vandenberghe. We are most grateful to Stephen and Lieven for sharing
them with us.
Contents
I
Basics of Optimization
1 Main Concepts
1.1 Optimization Problems . . . . . . . . . . . . . . . .
1.1.1 Continuous Problems . . . . . . . . . . . .
Active, Inactive and Redundant Constraints
Problems in Standard Forms . . . . . . . .
Eliminating Equality Constraints . . . . . .
Problem Description . . . . . . . . . . . . .
1.1.2 Integer and Mixed-Integer Problems . . . .
1.2 Convexity . . . . . . . . . . . . . . . . . . . . . . .
Operations preserving convexity . . . . . .
Linear and quadratic convex functions . . .
Convex optimization problems . . . . . . .
1
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3
3
4
5
5
5
6
6
6
7
8
8
2 Optimality Conditions
2.1 Optimality Conditions for Unconstrained Problems .
2.2 Lagrange Duality Theory . . . . . . . . . . . . . . .
2.2.1 Strong Duality and Constraint Qualifications
2.2.2 Certificate of Optimality . . . . . . . . . . . .
2.3 Complementary Slackness . . . . . . . . . . . . . . .
2.4 Karush-Kuhn-Tucker Conditions . . . . . . . . . . .
2.4.1 Geometric Interpretation of KKT Conditions
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
11
11
12
14
15
15
16
18
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
21
21
22
22
24
25
25
28
29
30
31
31
32
viii
Contents
3.4
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
35
35
37
37
38
38
40
41
42
43
43
45
45
46
46
46
47
49
49
50
54
59
59
60
63
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
67
67
69
72
73
74
74
75
75
76
78
79
80
82
82
84
85
85
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Contents
5.5.1
5.5.2
5.5.3
II
ix
Set-Difference . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
Polytope Covering . . . . . . . . . . . . . . . . . . . . . . . . 88
Union of P-collections . . . . . . . . . . . . . . . . . . . . . . 89
Multiparametric Programming
91
III
Optimal Control
.
.
.
.
.
.
105
105
105
106
107
108
108
.
.
.
.
.
.
.
110
113
115
117
120
125
125
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
126
129
131
133
137
137
137
138
138
139
139
140
141
141
142
145
Contents
8.2
8.3
8.4
8.5
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
149
150
152
153
154
156
156
159
161
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
163
163
164
165
167
168
169
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
171
172
174
176
177
IV
181
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. .
. .
. .
. .
In. .
. .
. .
. .
.
.
.
.
183
183
185
190
196
.
.
.
.
197
199
202
205
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
211
211
214
218
218
220
221
223
225
226
.
.
.
.
.
.
.
.
.
Contents
xi
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
227
230
233
234
235
236
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
243
244
250
251
254
257
260
262
267
274
.
.
.
.
.
.
.
.
.
277
277
278
280
281
282
284
285
285
286
.
.
.
.
.
286
288
289
290
291
xii
Contents
16 Constrained Robust Optimal Control
311
16.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
16.1.1 State Feedback Solutions Summary . . . . . . . . . . . . . . . 317
16.2 Feasible Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
16.3 State Feedback Solution, Nominal Cost . . . . . . . . . . . . . . . . 323
16.4 State Feedback Solution, Worst-Case Cost, 1-Norm and -Norm Case324
16.4.1 Batch Approach: Open-Loop Predictions . . . . . . . . . . . 325
16.4.2 Recursive Approach: Closed-Loop Predictions . . . . . . . . . 326
16.4.3 Solution to CROC-CL and CROC-OL via mp-MILP* . . . . 327
16.5 Parametrizations of the Control Policies . . . . . . . . . . . . . . . . 329
16.6 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
16.7 Robust Receding Horizon Control . . . . . . . . . . . . . . . . . . . . 336
16.8 Literature Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338
339
. . .
. . .
. . .
Pro. . .
. . .
. . .
. . .
.
.
.
.
.
.
.
.
.
.
.
.
.
341
341
342
344
345
348
349
349
350
351
352
352
353
353
.
.
.
.
.
354
355
357
357
360
367
. 367
. 369
. 375
.
.
.
.
376
379
380
380
Contents
xiii
Bibliography
383
385
388
389
390
391
391
404
xiv
Contents
Sets
R (R+ )
N (N+ )
Rn
Rnm
Set
Set
Set
Set
of
of
of
of
A closed interval in R
A row vector
Transpose of matrix A
Inverse of matrix A
Generalized Inverse of A, A = (A A)1 A
Determinant of matrix A
A symmetric positive (semi)definite matrix, x Ax > ()0, x 6= 0
A symmetric negative (semi)definite matrix, x Ax < ()0, x 6= 0.
i-th row of matrix A
i-th element of vector x
True iff xi > 0 (xi 0) i = 1, . . . , n
True iff xi < 0 (xi 0) i = 1, . . . , n
Absolute value of x
Any vector norm of x
pPn
2
Euclidian norm of vector x Rn , kxk2 =
i|
i=1 |xP
n
Sum of absolute elements of vector x Rn , kxk1 = i=1 |xi |
Largest absolute value of the vector x Rn , kxk = maxP
i{1,...,n} |xi |
Matrix -norm of S Cmn ,i.e., kSk = maxi{1,...,m} nj=1 |si,j |
P
Matrix 1-norm of S Cmn , i.e., kSk1 = maxj{1,...,n} m
i=1 |si,j |
Identity matrix
Vector of ones, 1 = [1 1 . . . 1]
Vector of zeros, 0 = [0 0 . . . 0]
Contents
xv
|P|
P Q
P
S Q
r{1,...,R}
Pc
P \Q
PQ
PQ
PQ
PQ
P Q
P Q
P
int(P)
:
f (x)
Pr
xvi
Contents
Acronyms
ARE
BMI
CLQR
CFTOC
CITOC
DP
LMI
LP
LQR
LTI
MILP
MIQP
MPC
mp-LP
mp-QP
PWA
PPWA
PWP
PWQ
QP
RHC
rhs
SDP
Dynamical Systems
x(k)
xk
x+
F
O
C
K
LQR
O
Part I
Basics of Optimization
Chapter 1
Main Concepts
In this chapter we recall the main concepts and definitions of continuous and discrete optimization. Our intent is to provide only the necessary background for
the understanding of the rest of the book. The notions of feasibility, optimality,
convexity and active constraints introduced in this chapter will be widely used in
the rest of the book.
1.1
Optimization Problems
An optimization problem is generally formulated as
inf z
subj. to
f (z)
z S Z,
(1.1)
where the vector z collects the decision variables, Z is the optimization problem
domain, S Z is the set of feasible or admissible decisions. The function f : Z R
assigns to each decision z a cost f (z) R. We will often use the following shorter
form of problem (1.1)
inf f (z).
(1.2)
zSZ
The number f is the optimal value of problem (1.2), i.e. the greatest lower bound
of f (z) over the set S:
f (z) f z S and ( > 0 z S : f (z) f + ).
If f = we say that the problem is unbounded below. If the set S is empty
then the problem is said to be infeasible and we set f = + by convention. If
S = Z the problem is said to be unconstrained.
Main Concepts
(1.3)
and z is called an optimizer or global optimizer or optimal solution. The set of all
optimal solutions is denoted by
argminzS f (z) , {z S : f (z) = f } .
A problem of determining whether the set of feasible decisions is empty and, if not,
to find a point which is feasible, is called a feasibility problem.
1.1.1
Continuous Problems
In continuous optimization the problem domain Z is a subset of the finite-dimensional
Euclidian vector-space Rs and the subset of admissible vectors is defined through
a list of equality and inequality constraints:
inf z
subj. to
f (z)
gi (z) 0
hj (z) = 0
z Z,
for i = 1, . . . , m
for j = 1, . . . , p
(1.4)
(1.6)
1.1
Optimization Problems
such that
f (
z ) = inf z
subj. to
f (z)
gi (z) 0 for i = 1, . . . , m
hi (z) = 0 for i = 1, . . . , p
kz zk R
z Z.
(1.7)
Roughly speaking, this means that z is the minimizer of f (z) in a feasible neighborhood of z defined by kz zk R. The point z is called a local optimizer.
Active, Inactive and Redundant Constraints
Consider a feasible point z. We say that the i-th inequality constraint gi (z) 0 is
active at z if gi (
z ) = 0. If gi (
z ) < 0 we say that the constraint gi (z) 0 is inactive
at z. Equality constraints are always active for all feasible points.
We say that a constraint is redundant if removing it from the list of constraints
does not change the feasible set S. This implies that removing a redundant constraint from problem (1.4) does not change its solution.
Problems in Standard Forms
Optimization problems can be cast in several forms. In this book we use the
form (1.4) where we adopt the convention to minimize the cost function and to
have the right-hand side of the inequality and equality constraints equal to zero.
Any problem in a different form (e.g., a maximization problem or a problem with
box constraints) can be transformed and arranged into this form. The interested
reader is referred to Chapter 4 of [61] for a detailed discussion on transformations
of optimization problems into different standard forms.
Eliminating Equality Constraints
Often in this book we will restrict our attention to problems without equality
constraints, i.e., p = 0
inf z
subj. to
f (z)
gi (z) 0
for i = 1, . . . , m.
(1.8)
The simplest way to remove equality constraints is to replace them with two inequalities for each equality, i.e., hi (z) = 0 is replaced by hi (z) 0 and hi (z) 0.
Such a method, however, can lead to poor numerical conditioning and may ruin
the efficiency and accuracy of a numerical solver.
If one can explicitly parameterize the solution of the equality constraint hi (z) =
0, then the equality constraint can be eliminated from the problem. This process
can be described in a simple way for linear equality constraints. Assume the equality constraints to be linear, Az b = 0, with A Rps . If Az = b is inconsistent
then the problem is infeasible. The general solution of the equation Az = b can be
expressed as z = F x + x0 where F is a matrix of full rank whose spanned space
coincides with the null space of the A matrix, i.e., R(F ) = N (A), F Rsk ,
where k is the dimension of the null space of A. The variable x Rk is the new
Main Concepts
f (F x + x0 )
gi (F x + x0 ) 0
for i = 1, . . . , m.
(1.9)
We want to point out that in some cases the elimination of equality constraints
can make the problem harder to analyze and understand and can make a solver
less efficient. In large problems it can destroy useful structural properties of the
problem such as sparsity. Some advanced numerical solvers perform elimination
automatically.
Problem Description
The functions f, gi and hi can be available in analytical form or can be described
through an oracle model (also called black box or subroutine model). In an
oracle model f, gi and hi are not known explicitly but can be evaluated by querying
the oracle. Often the oracle consists of subroutines which, called with the argument
z, return f (z), gi (z) and hi (z) and their gradients f (z), gi (z), hi (z). In
the rest of the book we assume that analytical expressions of the cost and the
constraints of the optimization problem are available.
1.1.2
(1.10)
1.2
Convexity
A set S Rs is convex if
z1 + (1 )z2 S for all z1 S, z2 S and [0, 1].
1.2
Convexity
f (z1 ) + (1 )f (z2 )
for all z1 S, z2 S and [0, 1].
0 for all z S.
PN
i=1
Main Concepts
1.2
Convexity
to determine its global minimum. Convexity also plays a major role in most nonconvex optimization problems which are solved by iterating between the solutions
of convex sub-problems.
It is difficult to determine whether the feasible set S of the optimization problem (1.4) is convex or not except in special cases. For example, if the functions
g1 (z), . . . , gm (z) are convex and all the hi (z) (if any) are affine in z, then the feasible
region S in (1.6) is an intersection of convex sets and therefore convex. Moreover
there are non-convex problems which can be transformed into convex problems
through a change of variables and manipulations of cost and constraints. The discussion of this topic goes beyond the scope of this overview on optimization. The
interested reader is referred to [61].
Remark 1.1 With the exception of trivial cases, integer and mixed-integer optimization problems are always non-convex problems because {0, 1} is not a convex set.
10
Main Concepts
Chapter 2
Optimality Conditions
In general, an analytical solution to problem (1.4), restated below, does not exist.
inf z
subj. to
f (z)
gi (z) 0
hj (z) = 0
z Z.
for i = 1, . . . , m
for j = 1, . . . , p
(2.1)
Solutions are usually computed by iterative algorithms which start from an initial
guess z0 and at step k generate a point zk such that the sequence {f (zk )}k=0,1,2,...
converges to f as k increases. These algorithms iteratively use and/or solve conditions for optimality, i.e., analytical conditions that a point z must satisfy in order
to be an optimizer. For instance, for convex, unconstrained optimization problems with a smooth cost function the most commonly used optimality criterion
requires the gradient to vanish at the optimizer, i.e., z is an optimizer if and only
if f (z) = 0. In this chapter we summarize necessary and sufficient optimality
conditions for unconstrained and constrained optimization problems.
2.1
12
Optimality Conditions
2.2
bound on f .
Starting from the standard nonlinear program (2.1) we construct another problem with different variables and constraints. The original problem (2.1) will be
called the primal problem while the new one will be called the dual problem. First,
we augment the objective function with a weighted sum of the constraints. In this
way the Lagrange dual function (or Lagrangian) L is obtained
L(z, u, v) =
(2.2)
(2.3)
2.2
13
L(z, u, v)
inf
f (z)
(2.5)
in order to reconstruct the original problem on the right-hand side of the expression.
Since for arbitrary u 0 and v
inf
(2.6)
we obtain
inf L(z, u, v)
zZ
inf
f (z).
(2.7)
(2.8)
zZ
provides us with a lower bound to the original problem. The best lower bound
is obtained by maximizing problem (2.8) over the dual variables
sup
inf L(z, u, v)
(u,v), u0 zZ
inf
f (z).
(2.9)
(u, v),
(2.10)
(u,v), u0
and its optimal solution is denoted by (u , v ). The dual cost (u, v) is the optimal value of an unconstrained optimization problem. Problem (2.9) is called the
Lagrange dual subproblem. Only points (u, v) with (u, v) > are interesting
for the Lagrange dual problem. A point (u, v) will be called dual feasible if u 0
and (u, v) > . (u, v) is always a concave function since it is the pointwise
infimum of a family of affine functions of (u, v). This implies that the dual problem
is a convex optimization problem (max of a concave function over a convex set)
even if the original problem is not convex. Therefore, it is easier in principle to
solve the dual problem than the primal (which is in general non-convex). However,
14
Optimality Conditions
in general, the solution to the dual problem is only a lower bound of the primal
problem:
sup (u, v)
inf
f (z).
zZ, g(z)0, h(z)=0
(u,v), u0
Such a property is called weak duality. In a simpler form, let f and d be the
primal and dual optimal value, respectively,
f
inf
sup
f (z),
(2.11a)
(u, v),
(2.11b)
(u,v), u0
f d
(2.12)
and the difference f d is called the optimal duality gap. The weak duality
inequality (2.12) holds also when d and f are infinite. For example, if the primal
problem is unbounded below, so that f = , we must have d = , i.e., the
Lagrange dual problem is infeasible. Conversely, if the dual problem is unbounded
above, so that d = +, we must have f = +, i.e., the primal problem is
infeasible.
Remark 2.1 We have shown that the dual problem is a convex optimization problem
even if the original problem is non-convex. As stated in this section, for non-convex
optimization problems it is easier to solve the dual problem than the primal problem. However the evaluation of (
u, v) at a point (
u, v) requires the solution of the
non-convex optimization problem (2.9), which, in general, is not easy.
2.2.1
(u, v) =
inf
f (z).
(2.13)
This means that the best lower bound obtained by solving the dual problem coincides with the optimal cost of the primal problem. In general, strong duality does
not hold, even for convex primal problems. Constraint qualifications are conditions on the constraint functions which imply strong duality for convex problems.
A detailed discussion on constraint qualifications can be found in Chapter 5 of [24].
A well known simple constraint qualification is Slaters condition:
Definition 2.1 (Slaters condition) Consider problem (2.1). There exists z
Rs which belongs to the relative interior of the problem domain Z, which is feasible
(g(
z ) 0, h(
z ) = 0) and for which gj (
z ) < 0 for all j for which gj is not an affine
function.
Remark 2.2 Note that Slaters condition reduces to feasibility when all inequality
constraints are linear and Z = Rn .
2.3
Complementary Slackness
15
Theorem 2.4 (Slaters theorem) Consider the primal problem (2.11a) and its
dual problem (2.11b). If the primal problem is convex, Slaters condition holds and
f is bounded then d = f .
2.2.2
Certificate of Optimality
Consider the (primal) optimization problem (2.1) and its dual (2.10). Any feasible
point z gives us information about an upper bound on the cost, i.e., f f (z).
If we can find a dual feasible point (u, v) then we can establish a lower bound
on the optimal value of the primal problem: (u, v) f . In summary, without
knowing the exact value of f we can give a bound on how suboptimal a given
feasible point is. In fact, if z is primal feasible and (u, v) is dual feasible then
(u, v) f f (z). Therefore z is -suboptimal, with equal to the primal-dual
gap, i.e., = f (z) (u, v).
The optimal value of the primal (and dual) problems will lie in the same interval
f [(u, v), f (z)] and d [(u, v), f (z)].
For this reason (u, v) is also called a certificate that proves the (sub)optimality
of z. Optimization algorithms make extensive use of such criteria. Primal-dual
algorithms iteratively solve primal and dual problems and generate a sequence of
primal and dual feasible points zk , (uk , vk ), k 0 until a certain is reached. The
condition
f (zk ) (uk , vk ) < ,
for terminating the algorithm guarantees that when the algorithm terminates, zk
is -suboptimal. If strong duality holds the condition can be met for arbitrarily
small tolerances .
2.3
Complementary Slackness
Consider the (primal) optimization problem (2.1) and its dual (2.10). Assume that
strong duality holds. Suppose that z and (u , v ) are primal and dual feasible
with zero duality gap (hence, they are primal and dual optimal):
f (z )
= (u , v ).
Therefore
f (z ) f (z ) + u g(z ) + v h(z ),
(2.14)
f (z ) f (z ) + u g(z ) f (z ).
(2.15)
16
2
From the last equation we can conclude that u g(z ) =
since ui 0 and gi (z ) 0, we have
Optimality Conditions
Pm
i=1
ui gi (z ) = 0 and
ui gi (z ) = 0, i = 1, . . . , m.
(2.16)
Conditions (2.16) are called complementary slackness conditions. Complementary slackness conditions can be interpreted as follows. If the i-th inequality constraint of the primal problem is inactive at the optimum (gi (z ) < 0), then the
i-th dual optimizer has to be zero (ui = 0). Vice versa, if the i-th dual optimizer
is different from zero (ui > 0), then the i-th constraint is active at the optimum
(gi (z ) = 0).
Relation (2.16) implies that the inequality in (2.14) holds as equality
X
X
X
X
f (z ) +
ui gi (z ) +
vj hj (z ) = min f (z) +
ui gi (z) +
vj hj (z) .
i
zZ
(2.17)
Therefore, complementary slackness conditions implies that z is a minimizer of
L(z, u , v ).
2.4
Karush-Kuhn-Tucker Conditions
Consider the (primal) optimization problem (2.1) and its dual (2.10). Assume that
strong duality holds. Assume that the cost functions and constraint functions f , gi ,
hi are differentiable. Let z and (u , v ) be primal and dual optimal points, respectively. Complementary slackness conditions implies that z minimizes L(z, u , v )
under no constraints (equation (2.17)). Since f , gi , hi are differentiable, the gradient of L(z, u , v ) must be zero at z
f (z ) +
X
i
ui gi (z ) +
X
j
vj hj (z ) = 0.
m
X
i=1
ui gi (z ) +
p
X
j=1
vj hi (z ) = 0,
ui gi (z ) = 0,
ui 0,
gi (z ) 0,
hj (z ) = 0,
(2.18a)
i = 1, . . . , m
i = 1, . . . , m
(2.18b)
(2.18c)
i = 1, . . . , m
j = 1, . . . , p
(2.18d)
(2.18e)
2.4
Karush-Kuhn-Tucker Conditions
where equations (2.18d)-(2.18e) are the primal feasibility conditions, equation (2.18c)
is the dual feasibility condition and equation (2.18b) are the complementary slackness conditions.
Conditions (2.18a)-(2.18e) are called the Karush-Kuhn-Tucker (KKT) conditions. We have shown that the KKT conditions are necessary conditions for any
primal-dual optimal pair if strong duality holds and the cost and constraints are
differentiable, i.e., any primal and dual optimal points z , (u , v ) must satisfy
the KKT conditions (2.18). If the primal problem is also convex then the KKT
conditions are sufficient, i.e., a primal dual pair z , (u , v ) which satisfies conditions (2.18a)-(2.18e) is a primal dual optimal pair with zero duality gap.
There are several theorems which characterize primal and dual optimal points
z and (u , v ) by using KKT conditions. They mainly differ on the type of
constraint qualification chosen for characterizing strong duality. Next we report
just two examples.
If a convex optimization problem with differentiable objective and constraint
functions satisfies Slaters condition, then the KKT conditions provide necessary
and sufficient conditions for optimality.
Theorem 2.5 [24, p. 244] Consider problem (2.1) and let Z be a nonempty set of
Rs . Suppose that problem (2.1) is convex and that cost and constraints f , gi and
hi are differentiable at a feasible z . If problem (2.1) satisfies Slaters condition
then z is optimal if and only if there are (u , v ) that, together with z , satisfy
the KKT conditions (2.18).
If a convex optimization problem with differentiable objective and constraint functions has a linearly independent set of active constraints gradients, then the KKT
conditions provide necessary and sufficient conditions for optimality.
Theorem 2.6 (Section 4.37 in [24]) Consider problem (2.1) and let Z be a nonempty
open set of Rs . Let z be a feasible solution and A = {i : gi (z ) = 0} be the set of
active constraints at z . Suppose cost and constraints f , gi are differentiable at z
for all i and that hj are continuously differentiable at z for all j. Further, suppose
that gi (z ) for i A and hj (z ) for j = 1, . . . , p, are linearly independent. If
z , (u , v ) are primal and dual optimal points, then they satisfy the KKT conditions (2.18). In addition, if problem (2.1) is convex, then z is optimal if and only
if there are (u , v ) that, together with z , satisfy the KKT conditions (2.18).
The KKT conditions play an important role in optimization. In a few special cases
it is possible to solve the KKT conditions (and therefore, the optimization problem)
analytically. Many algorithms for convex optimization are conceived as, or can be
interpreted as, methods for solving the KKT conditions as Boyd and Vandenberghe
observe in [61].
The following example [24] shows a convex problem where the KKT conditions
are not fulfilled at the optimum. In particular, both the constraint qualifications
of Theorem 2.6 and Slaters condition in Theorem 2.5 are violated.
17
18
Optimality Conditions
z2
(1, 1)
g2 (1, 0)
f (1, 0)
z1
g1 (1, 0)
(1, 1)
z1
(z1 1)2 + (z2 1)2 1
(z1 1)2 + (z2 + 1)2 1
(2.19)
From the graphical interpretation in Figure 2.1 it is immediate that the feasible set is a
single point z = [1, 0] . The optimization problem does not satisfy Slaters conditions
and moreover z does not satisfy the constraint qualifications in Theorem 2.6. At
the optimum z Equation (2.18a) cannot be satisfied for any pair of nonnegative real
numbers u1 and u2 .
2.4.1
where A = {1, 2} at z1 and A = {2, 3} at z2 . This means that the negative gradient
of the cost at the optimum f (z ) (which represents the direction of steepest
2.4
Karush-Kuhn-Tucker Conditions
19
g1 (z
)
(z
g2
g1 (z1 ) f (z1 )
g2 (z1 )
z1
f (z2 )
g2 (z2 )
g(z) 0
)
(z
g3
z2
0
g3 (z2 )
descent) has to belong to the cone spanned by the gradients of the active constraints
gi , (since inactive constraints have the corresponding Lagrange multipliers equal
to zero). In Figure 2.2, condition (2.20) is not satisfied at z2 . In fact, one can
move within the set of feasible points g(z) 0 and decrease f , which implies that
z2 is not optimal. At point z1 , on the other hand, the cost f can only decrease
if some constraint is violated. Any movement in a feasible direction increases the
cost. Conditions (2.20) are fulfilled and hence z1 is optimal.
20
Optimality Conditions
Chapter 3
3.1
(3.1)
(3.2)
a3
x
b3
a1 x
b1
a
2x
b2
a4 x
22
Figure 3.1 Polytope. The planes (here lines) defining the halfspaces are
ai x bi = 0.
3.2
Linear Programming
When the cost and the constraints of the continuous optimization problem (1.4)
are affine, then the problem is called a linear program (LP). The most general form
of a linear program is
inf z
c z
(3.3)
subj. to Gz w,
where G Rms , w Rm . Linear programs are convex optimization problems.
Two other common forms of linear programs include both equality and inequality constraints:
inf z
c z
subj. to Gz w
(3.4)
Geq z = weq ,
where Geq Rps , weq Rp , or only equality constraints and positive variables:
inf z
subj. to
c z
Geq z = weq
z 0.
(3.5)
3.2.1
3.2
Linear Programming
23
P
z
k4
k3
k2
k1
k4
(a) Case 1.
k3
k2
k1
(b) Case 2.
z
P
k4
k3
k2
k1
(c) Case 3.
not empty. Denote by f the optimal value and by Z the set of optimizers of
problem (3.3)
Z = argminzP c z.
Three cases can occur.
Case 1. The LP solution is unbounded, i.e., f = .
Case 2. The LP solution is bounded, i.e., f > and the optimizer is unique.
z , Z is a singleton.
Case 3. The LP solution is bounded and there are multiple optima. Z is an
uncountable subset of Rs which can be bounded or unbounded.
The two-dimensional geometric interpretation of the three cases discussed above
is depicted in Figure 3.2.
The level curves of the cost function c z are represented by the parallel lines.
All points z belonging both to the line c z = ki and to the polyhedron P are feasible
points with an associated cost ki , with ki < ki1 . Solving (3.3) amounts to finding
24
a feasible z which belongs to the level curve with the smallest cost ki . Since the
gradient of the cost is c, the direction of steepest descent is c/kck.
Case 1 is depicted in Figure 3.2(a). The feasible set P is unbounded. One can
move in the direction of steepest descent c and be always feasible, thus decreasing
the cost to . Case 2 is depicted in Figure 3.2(b). The optimizer is unique and
it coincides with one of the vertices of the feasible polyhedron. Case 3 is depicted
in Figure 3.2(c). The whole facet of the feasible polyhedron P is optimal, i.e., the
cost for any point z belonging to the facet equals the optimal value f . In general,
the optimal facet will be a facet of the polyhedron P parallel to the hyperplane
c z = 0.
From the analysis above we can conclude that the optimizers of any bounded
LP always lie on the boundary of the feasible polyhedron P.
3.2.2
Dual of LP
Consider the LP (3.3)
inf z
c z
subj. to Gz w,
(3.6)
u w
if G u = c
if G u 6= c.
Since we are interested only in cases where is finite, from the relation above we
conclude that the dual problem is
supu
subj. to
u w
G u = c
u 0,
(3.7)
inf u
subj. to
w u
G u = c
u 0.
(3.8)
Note that for LPs feasibility implies strong duality (Remark 2.2).
3.2
3.2.3
Linear Programming
25
=
=
c,
0, j = 1, . . . , m
(3.9a)
(3.9b)
0,
w.
(3.9c)
(3.9d)
3.2.4
(3.11)
26
5
6
P
z
Definition 3.3 The LP (3.3) is said to be dual degenerate if its dual problem is
primal degenerate.
The LICQ condition is invoked so that the optimization problem is well behaved
in a way we will explain next. Let us look at the equality constraints of the
dual problem (3.8) at the optimum G u = c or equivalently GA uA = c since
uN A = 0. If LICQ is satisfied then the equality constraints will allow only a unique
optimizer u . If LICQ is not satisfied, then the dual may have multiple optimizers.
Thus we have the following Lemma.
Lemma 3.2 If the primal problem (3.3) is not degenerate then the dual problem
has a unique optimizer. If the dual problem (3.8) is not degenerate, then the primal
problem has a unique optimizer.
Multiple dual optimizers imply primal degeneracy and multiple primal optimizers
imply dual degeneracy but the reverse is not true as we will illustrate next. In
other words LICQ is only a sufficient condition for uniqueness of the dual.
Example 3.1 Primal and dual degeneracies
Consider the following pair of primal and dual LPs
Primal
inf [1 1]x
0
1
1
1
x
1
0
0
1
1
1
0
0
(3.12)
3.2
Linear Programming
27
Dual
inf [1 1 0 0]u
1
0 1 1
0
u=
1
1 1
0
1
u0
(3.13)
Substituting for u3 and u4 from the equality constraints in (3.13) we can rewrite the
dual as
Dual
u1
inf [1 1]
u2
(3.14)
0
1 0
1
0 1 u1
1 1 u2 1
0
0 1
The situation is portrayed in Figure 3.4.
u2
x2
dual solution
primal solution
max
min
1
(a) Case 2 - Primal LP.
u1
x1
(b) Case 2 - Dual LP.
Figure 3.4 Example 3.1 LP with Primal and Dual Degeneracy. The vectors
max and min point in the direction for which the objective improves. The
feasible regions are shaded.
Consider the two solutions for the primal LP denoted with 1 and 2 in Figure 3.4(a) and referred to as basic solutions. Basic solution 1 is primal nondegenerate, since it is defined by exactly as many active constraints as there
are variables. Basic solution 2 is primal degenerate, since it is defined by three
active constraints, i.e., more than two. Any convex combination of optimal solutions 1 and 2 is also optimal. This continuum of optimal solutions in the primal
problem corresponds to a degenerate solution in the dual space, that is, the dual
problem is primal-degenerate. Hence the primal problem is dual-degenerate. In
conclusion, Figures 3.4(a) and 3.4(b) show an example of a primal problem with
28
multiple optima and the corresponding dual problem being primal degenerate.
Next we want to show that the statement if the dual problem is primal degenerate then the primal problem has multiple optima is, in general, not true.
Switch dual and primal problems, i.e., call the dual problem primal problem
and the primal problem dual problem (this can be done since the dual of
the dual problem is the primal problem). Then, we have a dual problem which
is primal degenerate in solution 2 while the primal problem does not present
multiple optimizers.
3.2.5
(3.15)
(3.16)
S
where {Ri }pi=1 are polyhedral sets with disjoint interiors, R , pi=1 Ri is a polyhedron and ci Rs , di R. Then, f (z) can be rewritten as [247]
i=1,...,k
or [61]:
f (z) =
min,z
ci z + di , i = 1, . . . , k
z R.
(3.17)
f (z)
Gz w
z R,
(3.18)
where the cost function has the form (3.15). Substituting (3.17) this becomes
f =
minz,
subj. to
Gz w
ci z + di , i = 1, . . . , k
z R.
(3.19)
The previous result can be extended to the sum of continuous and convex
piecewise affine functions. Let f : R Rs R be defined as:
f (z) =
r
X
f j (z),
(3.20)
j=1
with
(3.21)
3.3
Quadratic Programming
29
f (z)
c4 z + d4
c1 z + d1
c2 z
R1
c3 z + d3
+ d2
R2
R3
R4
Figure 3.5 Convex PWA Function Described as the Max of Affine Functions.
f (z)
Gz w
z R,
(3.22)
where the cost function has the form (3.20), can be solved by the following linear
program:
f = minz,1 ,...,r 1 + + r
subj. to
Gz w
c1i z + d1i 1 , i = 1, . . . , k 1
c2i z + d2i 2 , i = 1, . . . , k 2
(3.23)
..
.
cri z + dri r , i = 1, . . . , k r
z R.
Remark 3.1 Note that the results of this section can be immediately applied to the
minimization of one or infinity norms. For any y R, |y| = max {y, y}. Therefore
for any Q Rks and p Rk :
kQz pk = max{Q1 z + p1 , Q1 z p1 , . . . , Qk z + pk , Qk z pk }
and
kQz pk1 =
3.3
k
X
i=1
|Qi z + pi | =
k
X
i=1
max{Qi z + pi , Qi z pi }
Quadratic Programming
The continuous optimization problem (1.4) is called a quadratic program (QP) if the
constraint functions are affine and the cost function is a convex quadratic function.
30
P
(a) Case 1, optimizer in interior of P.
1
2 z Hz
+ q z + r
Gz w,
(3.24)
3.3.1
1
2 z Hz
+ q z + r
Gz w
Geq z = weq .
(3.25)
3.3
Quadratic Programming
31
(3.26)
Since H 0 we obtain z = H 1 q.
3.3.2
Dual of QP
Consider the QP (3.24)
minz
subj. to
1
2 z Hz
+ q z
Gz w.
1
L(z, u) = { z Hz + q z + u (Gz w)}.
2
The the dual cost is
1
(u) = min{ z Hz + q z + u (Gz w)}
z
2
and the dual problem is
1
max min{ z Hz + q z + u (Gz w)}.
u0 z
2
(3.27)
(3.28)
1
1
G )u
2 u (GH
u 0.
+ u (w + GH 1 q) + 12 q H 1 q
(3.29)
(3.30)
Note that for convex QPs feasibility implies strong duality (Remark 2.2).
3.3.3
=0
(3.31a)
ui (Gi z wi )
u
=0
0
(3.31b)
(3.31c)
Gz w
0.
(3.31d)
32
3.3.4
3.3.5
(3.32)
is an (unconstrained) QP. It arises in many fields and has many names, e.g., linear
regression or least-squares approximation. From (3.26) we find the minimizer
z = (A A)1 A b , A b,
where A is the generalized inverse of A. When linear inequality constraints are
added, the problem is called constrained linear regression or constrained leastsquares, and there is no longer a simple analytical solution. As an example we can
consider regression with lower and upper bounds on the variables, i.e.,
minz
subj. to
kAz bk22
li zi ui , i = 1, . . . , n,
(3.33)
which is a QP. In Chapter 7.3.1 we will show how to compute an analytical solution to the constrained least-squares problem. In particular we will show how to
compute the solution z as a function of A, b, ui s and li s.
3.4
Mixed-Integer Optimization
As discussed in Section 1.1.2, if the decision set Z in the optimization problem (1.2)
is the Cartesian product of a binary set and a real Euclidian space, i.e., Z
{[zc , zb ] : zc Rsc , zb {0, 1}sb }, then the optimization problem is said to be
3.4
Mixed-Integer Optimization
33
2 z Hz + q z
Gc zc + Gb zb
+r
w
Geq,c zc + Geq,b zb = weq
zc Rsc , zb {0, 1}sb
z = [zc , zb ],
(3.34)
2 zc Hc zc + qc (zb ) zc
Gc zc w Gb zb
+ k(zb )
(3.36)
Therefore the most obvious way to interpret and solve an MIQP is to enumerate
all the 2sb integer values of the variable zb and solve the corresponding QPs. By
comparing the 2sb optimal costs one can derive the optimizer and the optimal cost
of the MIQP (3.34). Although this approach is not used in practice, it gives a
simple way for proving what is stated next. Let Pzb be the feasible set (1.6) of
problem (3.35) for a fixed zb = zb . The cost is a quadratic function defined over
Rsc and Pzb is a polyhedron defined by the inequality constraints
Gc zc w Gb zb .
(3.37)
Denote by f the optimal value and by Z the set of optimizers of problem (3.34)
Z = arg minzb {0,1}sb ,zc Pz
1
z Hc zc + qc (zb ) zc + k(zb )
2 c
If Pzb is empty for all zb , then the problem (3.35) is infeasible. Five cases can occur
if Pzb is not empty for at last one zb {0, 1}sb :
34
{(z1,c
, z1,b
), . . . , (zN,c
, zN,b
)}.
Case 5. The union of Case 3 and Case 4.
Chapter 4
4.1
Introduction
In this chapter, we present numerical methods for finding a minimizer z to the
optimization problem
minimize f (z)
(P)
subject to z S
where both the objective function f : Rs R and the feasible set S are convex,
hence (P) is said to be convex. Most important examples that appear in the context
of model predictive control are instances of the form
min q z
s.t. Az = b
z0
(LP)
min
s.t.
1
2 z Hz
Az = b
z0
+ q z
(QP)
with polyhedral feasible set Q and H 0. In all but the simplest cases, analytically
finding a minimizer z is impossible. Hence, one has to resort to numerical methods
that compute an approximate minimizer that is good enough (we give a precise
definition of this term below). Any such method proceeds in an iterative manner,
36
max
, where
i.e. from an initial guess z0 , it computes a sequence {zi }i=i
i=0
zi+1 = (zi , f, S) ,
(4.1)
with being an update rule depending on the method. For a method to be useful,
it should terminate after imax iterations and return an approximate minimizer zimax
that satisfies
|f (zimax ) f (z ) |
and dist(zimax , S) ,
(4.2)
where
dist(z, S) , min ky zk
yS
(4.3)
is the shortest distance between a point and a set in Rs measured by the norm k k.
The parameters , define the required accuracy of the approximate minimizer.
There exist mainly two classes of optimization methods for solving (P): firstorder methods that make use of first-order information (subgradients or gradients)
and second-order methods using in addition second-order information (Hessians).
When presenting these methods, we will touch on important aspects such as
Convergence: is imax finite for every , > 0?
Convergence speed: how do the errors ei , |f (zi ) f (z ) | and dist(zi , S)
depend on the iteration counter i?
Feasibility: does the method produce feasible iterates, i.e. dist(zi , S) = 0 i?
Numerical robustness: Do the properties above change in presence of finite
precision arithmetics using fixed- or floating point number representations?
Warm-starting: Can the method take advantage of z (0) being close to the
minimizer z ?
Preconditioning: Is it possible to transform (P) into an equivalent problem (P ) that can be solved in fewer iterations?
Computational complexity: How many floating point operations are needed
for computing the approximate solution zimax ?
Rates of convergence
The speed or rate of convergence relates to the second point in the list above, and is
an important distinctive feature between various methods. We present a definition
of convergence rates in the following.
Definition 4.1 (Rates of convergence [160, 7.2.1]) Let {ei } be a converging
sequence of reals with limit point 0, i.e. limi ei = 0. A sequence is said to
converge with rate r if, for some C > 0 and finite,
ei+1
(4.4)
lim sup r = C.
i
ei
If r = 1, the rate of convergence is called linear, if r < 1 the rate is sublinear, if
r > 1 the rate is superlinear and if r = 2, the rate is quadratic. For sublinear and
linear convergence, the constant C must be strictly smaller than 1.
4.2
Unconstrained Optimization
37
Higher rates of convergence are desirable; for example, a quadratically convergent method computes a solution to 16 bits accuracy in only five steps. The
constant C may be important for the behavior of the algorithm in practice. A linearly converging method with a constant C 1 can be slower for practical purposes
than a method with sublinear convergence but C 0.
Outline
In the remainder of this chapter, we first introduce methods that solve the unconstrained problem (i.e. S = Rs ) in Section 4.2. These are essential in the
understanding of constrained optimization methods, which are then presented in
Section 4.3.
4.2
Unconstrained Optimization
In unconstrained smooth convex optimization, we are interested in solving the
following problem:
min f (z),
(4.5)
z
4.2.1
Descent Methods
For the purpose of this chapter, we focus on so-called descent methods for unconstrained optimization. These methods obtain the next iterate zi+1 from the current
iterate zi by taking a step of size hi > 0 along a certain direction pi :
zi+1 = zi + hi pi .
(4.6)
Particular methods differ in the way pi and hi are computed. In general, pi has to
be a descent direction, i.e. zi 6= z it must satisfy
f (zi ) pi < 0.
(4.7)
The step sizes hi have to be chosen carefully in order to ensure a sufficiently large
decrease in the function value, which is crucial for convergence of descent methods.
Since solving for the optimal step size,
hi arg min f (zi + hpi ),
h0
(4.8)
is generally too expensive, inexact line search methods are used instead. They
compute hi cheaply and achieve a reasonably good decrement in the function value
while ensuring convergence of the algorithm. We discuss line search methods in
Section 4.2.4.
38
4.2.2
Gradient Methods
Classic Gradient Method
One property of the gradient of f evaluated at zi , f (zi ), is that it points into
the direction of steepest local ascent. The main idea of gradient methods for
minimization is therefore to use the anti-gradient direction as a descent direction,
pi = f (zi ),
(4.9)
trivially satisfying (4.7). Before turning to the question of appropriate step sizes
hi , we introduce an additional assumption on f . For the remainder of this section,
we assume that f has a Lipschitz continuous gradient, i.e. that it is L-smooth. We
state the definition of L-smoothness below.
Definition 4.2 (Lipschitz Continuity of the Gradient (L-Smoothness)) Let
f : Rn R be once continuously differentiable on Rn . The gradient f is Lipschitz
continuous on Rn with Lipschitz constant L > 0 if for all pairs (z, y) Rn Rn
kf (z) f (y)k L kz yk .
In general it is difficult to obtain the Lipschitz constant L for the gradient f
from Definition 4.2 only. In this context, the next lemma is much more useful, as
it relates the largest eigenvalue of the Hessian of f to the Lipschitz constant L.
Lemma 4.1 (Second-Order Characterization of L-Smoothness) Let f : Rn
R be twice continuously differentiable on Rn . Then the gradient f is Lipschitz
continuous on Rn with Lipschitz constant L > 0 if and only if
2
f (z)
L , z Rn .
The most important consequence of Lipschitz continuity of the gradient is that
f can be upper-bounded by a quadratic, cf. 4.1. This is expressed in the next
lemma.
Lemma 4.2 (Descent Lemma) Let f be L-smooth on Rn . Then for any pair
(z, y) Rn Rn we have
f (z) f (y) + f (y) (z y) +
L
2
kz yk .
2
(4.10)
4.2
Unconstrained Optimization
39
f (z)
2.5
f (y) + f (y)T (z y) +
L
kz
2
yk2
1.5
f (y)
0.5
1
f (y) + f (y)T (z y) +
0.5
0
z
y=0.25
0.5
kz
2
yk2
1
Figure 4.1 Upper and lower bounds on functions: The quadratic upper
bound stems from L-smoothness of f , while the quadratic lower bound
is obtained from -strong convexity of f (cf. 4.2.2). In this example,
f (z) = z 2 + 12 z + 1 and we have chosen L = 3 and = 1 for illustration.
where we define
1
kz zi k2 ,
f(z, zi ) , f (zi ) + f (zi ) (z zi ) +
2hi
(4.12)
(4.13)
(4.15)
In fact, the inequalities are strict for zi 6= z . The gradient method summa
rized in Algorithm 4.1 converges sublinearly with imax O Lkz0 z k2 / [215,
Theorem 1.1, Theorem 1.2]. Note that the step size is fixed, which results in an
extremely simple, division-free algorithm (if the gradient f can be computed
without divisions).
Example 4.1 [Quadratic objective function] Consider f : R2 R defined by
f (z) =
1
z Hz + g z
2
(4.16)
40
Figure 4.2 First 10 iterations of the gradient method with step size rule (4.14)
using the exact, slightly underestimated, and strongly underestimated Lipschitz constant L, as well as the fast gradient method applied to Example 4.1
(without taking strong convexity into account). The contour lines indicate
the function value of each iterate. Note that the gradients are orthogonal to
the contour lines.
0.2
2
and
g=
0.1
.
0.1
(4.17)
For quadratic functions, the Lipschitz constant L of the gradient can be exactly
determined by computing the largest eigenvalue of the Hessian H, i.e.
L = max (H),
(4.18)
here L = 2.022. The upper left plot in Figure 4.2 shows the first 10 iterations of the
gradient method with step size rule (4.14) when the exact L is employed. The upper
right and lower left plot show the first 10 iterations using a wrong L for the step
size calculation. An underestimated L leads to zig-zagging of the iterates, which can
potentially slow down convergence. It can be shown that if L is underestimated by
a factor of 2, convergence is lost as seen in the lower left plot. The lower right plot
shows the iterates of the fast gradient method, which generates a non-monotonically
decreasing error sequence.
(4.19)
p
This two-step method can be shown to converge sublinearly with imax O( Lkz0 z k2 /),
which is the best convergence rate for L-smooth problems using only first-order information. The mathematical reasoning for why using two converging sequences
{zi } and {yi } leads to a better theoretical convergence rate is quite involved and
beyond the scope of this chapter, but can be found in [215, Constant Step Scheme
II, 2.2.4].
Algorithm 4.1 Gradient Method for (4.5)
input: Initial iterate z0 Rn , Lipschitz constant L of f (Def. 4.2)
output: Point close to z
repeat
zi+1 = zi
1
L
f (zi )
4.2
Unconstrained Optimization
41
i (1i )
2
+i+1
i
kz yk2 .
2
L
1
(4.20)
follows intuitively. The constant is the so-called condition number of the Lsmooth, -strongly convex function f .
With this additional assumption, it can be shown that the gradient method (4.1)
converges linearly with imax O( ln(1/)). To achieve this convergence rate, the
gradient method in Algorithm 4.1 does not need to be modified. In contrast, the
sequences {i } and {yi } used in the fast gradient method in Algorithm 4.2 need
42
1
z Hz + g z
2
(4.21)
Algorithm 4.3 [t]Fast Gradient Method for (4.5) (strongly convex case)
n
input:
p Initial iterates z0 R , y0 = z0 ,
0 < /L 0 < 1, Lipschitz constant L of f (cf. 4.2), convexity parameter of
f (cf. 4.1)
output: Point close to z
repeat
zi+1 = yi L1 f (yi )
Solve 2i+1 = (1 i+1 )2i +
i =
i (1i )
2
i +i+1
i+1
L
Preconditioning
In the previous section, we have seen that the theoretical convergence rates for
gradient methods depend on the condition number . In practice, a high condition number usually indicates that many iterations of Algorithms ?? are needed to
achieve the specified accuracy. To improve the conditioning of the problem, and
thus to lower the amount of computation needed to obtain an approximate minimizer, one can transform problem (4.5) into an equivalent problem by a variable
transformation
y , Pz
(4.23)
with P invertible. The aim is to choose the preconditioner P such that the resulting
problem
min h(y)
(4.24)
y
4.2
Unconstrained Optimization
43
has a smaller condition number, i.e. h < f . Intuitively, the best choice for P is
one that achieves a circular shape of the level sets of the objective function, or at
least comes close to it. Mathematically speaking, P should be such that h 1.
Finding optimal preconditioners for which h = 1 is impossible for all but the
simplest cases. We give one such example in the following.
Example 4.3 [Optimal Preconditioner for Unconstrained Quadratic Program] Let
f (z) = 1/2z Hz. It is easily verified that P = H 1/2 is the optimal preconditioner
for which h = 1, since h(y) = 1/2y y. However, computing H 1/2 is as expensive
as solving the unconstrained quadratic program, so nothing has been gained in this
example.
An optimal preconditioner P for constrained quadratic programs can be calculated using semi-definite programming; for non-quadratic functions one has to
resort to heuristics.
Summary
In this section, we have discussed the concept of descent methods and presented
one of the most important instances, the classic gradient method for smooth convex
optimization. Although not a descent method in its basic form, Nesterovs fast gradient method was included due to its similarity to the classic gradient method. Both
methods are particularly useful in the strongly convex case, where they converge
linearly, with the fast gradient method attaining the best theoretical convergence
rate possible for first-order methods. Despite a slower theoretical (and most of the
time also practical) convergence, the classic gradient method is well-suited for processors with fixed-point arithmetic, for instance, field-programmable gate arrays
(FPGAs), because of its numerical robustness against inaccurate computations.
For the fast gradient method, recent research (cf. [91]) indicates that care has to
be taken when, e.g., inexact gradients are used. When using gradient methods in
practice, preconditioning should be applied to lower the condition number of the
problem, in order to decrease the number of iterations needed. The latter can also
be achieved by warm-starting, i.e. starting the methods from a point z0 close to
a minimizer z (this follows immediately from the dependence of the number of
iterations, imax , on the distance kz0 z k).
4.2.3
Newtons Method
The (fast) gradient method uses only first-order information of the function f . A
much more quickly converging algorithm, at least locally, can be obtained when
using second-order information, i.e. the Hessian 2 f . In the following, we assume f to be -strongly convex; hence the Hessian 2 f is positive definite on Rn .
Furthermore we assume that the Hessian 2 f is Lipschitz continuous on Rn with
Lipschitz constant M, i.e.
k2 f (z) 2 f (y)k M kz yk z, y Rs .
(4.25)
44
(4.26)
(4.28)
1
for all zi 6= z since, by strong convexity, the inverse of the Hessian 2 f (z)
exists for all z Rs and is positive definite. If the Lipschitz constant of the Hessian,
M , is small, this method works very well, since the quadratic model remains a good
approximation even if a full Newton step is taken. However, the quadratic model
m(p) is in general neither an upper nor a lower bound on f , and consequently a
line search (cf. Section 4.2.4) must be used to ensure a sufficient decrease in the
function value, which results in global convergence of the algorithm. Newtons
method is summarized in Algorithm 4.4. It can be shown to converge globally in
two phases (for technical details see [61, 9.5] and [215, 1.2.4]):
1. Damped Newton phase: When the iterates are far away from the optimal
point, the function value is decreased sublinearly, i.e. it can be shown that
there exist constants , (with 0 < < 2 /M and > 0) such that f (zi+1 )
f (zi ) for all zi for which kf (zi )k2 . The term damped is used in
this context since during this phase, the line search oftentimes returns step
sizes hi < 1, i.e. no full Newton steps are taken.
2. Quadratic convergence phase: If an iterate is sufficiently close to the optimum, i.e. kf (zi )k2 < , a full Newton step is taken, i.e. the line search
4.2
Unconstrained Optimization
45
returns hi = 1. The set {z | kf (z)k2 < } is called the quadratic convergence zone, and once an iterate zi is in this zone, all subsequent iterates
remain in it. The function value f (zi ) converges quadratically (i.e. with
imax O (ln ln )) to f .
The quadratic convergence property makes Newtons method one of the most
powerful algorithms for unconstrained optimization of twice continuously differentiable functions. (The method is also widely employed for solving nonlinear equations.) As a result, it is used in interior point methods for constrained optimization
to solve the (unconstrained or equality constrained) subproblems in barrier methods or to generate primal-dual search directions. We discuss these in detail in
Section 4.3.2.
A disadvantage of Newtons method is the need to form the Hessian 2 f (zi ),
which can be numerically ill-conditioned, and to solve the linear system
2 f (zi )pN = f (zi )
(4.29)
for the Newton direction pN , which costs O(n3 ) floating point operations in general.
Each iteration is therefore significantly more expensive than one iteration of any
of the gradient methods.
Preconditioning and Affine Invariance
Newtons method is affine invariant, i.e. a linear (or affine) transformation of
variables as discussed in Section 4.2.2 will lead to the same iterates subject to
transformation, i.e.
zi = P yi i = 0, . . . , imax .
(4.30)
Therefore, the number of steps to reach a desired accuracy of the solution does not
change. Consequently, preconditioning has no effect when using Newtons method.
Newtons Method with Equality Constraints
Equality constraints are naturally handled in Newtons method as follows. Assume
we want to solve the problem
minimize
subject to
f (z)
Az = b
(4.31)
with A Rps (assuming full rank p) and b Rp . As discussed above, the Newton
direction is given by the minimization of the quadratic model m(p) at the current
feasible iterate zi (i.e. Azi = b) with an additional equality constraint:
1 2
p f (zi )p + f (zi ) p + f (zi )
2
s.t. Ap = 0
pN = arg min
p
(4.32)
Since p is restricted to lie in the nullspace of A, we ensure that any iterate generated
by Newtons method will satisfy Azi+1 = A(zi +hi pN (zi )) = b. From the optimality
46
conditions of (4.32), it follows that pN is obtained from the solution to the linear
system of dimension n + p:
2
f (z) A p
f (z)
=
,
(4.33)
A
0 y
0
where y Rp are the Lagrange multipliers associated with the equality constraint
Ap = 0.
4.2.4
4.3
4.3.1
Constrained Optimization
Gradient Projection Methods
Gradient methods presented in Section 4.2.2 for unconstrained smooth (strongly)
convex optimization have a natural extension to the constrained problem
minimize
subject to
f (z)
zS
(4.34)
L
kz zi k2 ,
2
(4.35)
1
f (zi )) ,
L
(4.36)
where Q is the projection operator for Q. For the projection under the Euclidean
norm,
1
Q (y) , arg min kz yk2 .
(4.37)
zQ 2
To see that (??) and (4.36) are indeed equivalent, set y zi L1 f (zi ) in (4.37)
and observe that the optimality conditions coincide, while strong convexity of k k2
ensures uniqueness of zi+1 . So, Algorithms 4.1, 4.2, 4.3 remain the same in the
4.3
Constrained Optimization
Figure 4.3 Classic gradient method (left) and fast gradient method (right)
for the constrained case. After computing the standard update rule from
the unconstrained case, a projection on the feasible set is applied.
constrained case, except that the right hand sides in line 2 are encompassed by a
projection Q (), resulting in Algorithms 4.5, 4.6. This principle is also depicted
in 4.3.
Algorithm 4.5 Gradient Method for (4.34)
input: Initial iterate z0 Q, Lipschitz constant L of f (cf. 4.2)
output: Point close to z
repeat
zi+1 = Q zi L1 f (zi )
until stopping criterion
47
48
Table 4.1 Euclidean norm projection operators of selected closed convex sets
in Rn .
Set Name
Set Definition
n
o
n
z R | Az = b
Q =
affine set
with (A, b) R
nonnegative
orthant
Q =
rectangle
(e.g. norm
ball)
2-norm ball
Q =
ms
z R
z R
z R
, A 6= 0
o
|z 0
o
|l z u
with (l, u) R
Q =
| kzk r
Projection
z + A (AA )1 (b Az)
Q (z) =
z + A AT
A bz
Q (z)
=
k
Q (z)
with r 0
(
zk
0
lk
zk
uk
Q (z) =
if zk 0,
otherwise
if rank A = m,
otherwise
k = 1, . . . , n
if zk < lk ,
if lk zk uk ,
if zk > uk
(
r
z
z
kzk
k = 1, . . . , n
if kzk > r,
otherwise
domain and then recovering the primal solution from the dual solution. In the
following, we assume that the feasible set Q can be written as
Q = {z Rn | Az = b, z K} ,
(4.38)
where K is a set that allows an efficient projection according to e.g. 4.1, and A
Rms and b Rm , respectively. Most practically relevant feasible sets can be
defined in terms of (4.38), for example polytopes. We can now rewrite (4.34):
min f (z)
z
s.t. Az = b
zK
(4.39)
Using partial Lagrange relaxation [46, 4.2.2] by eliminating only the complicating
equality constraints, we define the dual function as
d() , min f (z) + (Az b),
zK
(4.40)
(4.41)
which we call the outer problem. In order to apply any of the gradient methods in
Algorithms ??, two requirements have to be fulfilled: differentiability of the dual
function d defined in (4.40), i.e. d() must exist for all , and Lipschitz continuity
of d. Due to strong convexity of f , which ensures uniqueness of z () defined as
z () , arg min f (z) + (Az b),
zK
(4.42)
it can be shown that the dual function is differentiable [244, Corollary 28.1.1], with
d() , Az () b.
(4.43)
4.3
Constrained Optimization
49
2
max
(A)
,
(4.44)
where max (A) is the largest singular value of A. For quadratic functions f (z) =
1/2z Hz + g z, Ld can be exactly computed as
Ld = kAH 1/2 k2 .
(4.45)
4.3.2
50
f (z)
Az = b
g(z) 0
(P-IPM)
(4.46)
f (z) + g (z)
Az = b
(P ())
where > 0 is called barrier parameter. The purpose of the barrier function g
is to trap an optimal solution of P (), which we denote by z (), in the set Q
(consequently, g (z) must take on the value + whenever gi (z) > 0 for some i,
and a finite value otherwise). As a result, z () is feasible with respect to Q, but it
differs from z since we have perturbed the objective function by the barrier term.
With the above idea in mind, we are now ready to outline the (primal) barrier
method summarized in Algorithm 4.7. Starting from (a possibly large) 0 , a solution z (0 ) to problem P (0 ) is computed, for example by any of the methods for
unconstrained optimization presented in Section 4.2. For this reason, g should be
twice continuously differentiable, allowing one to apply Newtons method. After
z (0 ) has been computed, the barrier parameter is decreased by a constant factor, 1 = 0 / (with > 1), and z (1 ) is computed. This procedure is repeated
until has been sufficiently decreased. It can be shown under mild conditions
that z () z from the interior of Q as 0. The points z () form the
so-called central path, which is a continuously differentiable curve in Q that ends in
the solution set (cf. Example 4.4). Solving a subproblem P () is therefore called
centering.
If the decrease factor is not too large, z (i ) will be a good starting point for
solving P (i+1 ), and therefore Newtons method is applied to P (i+1 ) to exploit the
local quadratic convergence rate. Note that the equality constraints in (P-IPM)
have been preserved in P (), since they do not cause substantial difficulties for
unconstrained methods as shown in Section 4.2. In particular, we refer the reader
to Section 4.2.3 for the equality constrained Newton method.
4.3
Constrained Optimization
51
Logarithmic barrier
So far, we have not specified which function to use as a barrier. For example, the
indicator function
(
0
if g(z) 0
Ig (z) ,
,
(4.47)
+ otherwise
trivially achieves the purpose of a barrier, but it is not useful since methods for
smooth convex optimization as discussed in Section 4.2 require the barrier function
to be convex and continuously differentiable. A twice continuously differentiable
convex barrier function that approximates Ig well is the logarithmic barrier function
g (z) ,
m
X
ln (gi (z)) ,
(4.48)
i=1
1
gi (z)
gi (z)
(4.49)
1
1
gi (z)gi (z) +
2 gi (z),
gi (z)2
gi (z)
(4.50)
g (z) =
i=1
g (z) =
m
X
i=1
which makes it possible to use Newtons method for solving the barrier subproblems. The logarithmic barrier function is used in almost all implementations of
barrier methods. (In fact, the existence of polynomial-time algorithms for convex
optimization is closely related to the existence of barrier functions for the underlying feasible sets [216].)
z2
z2
52
z1
z1
z2
z2
z1
z1
Figure 4.4 Illustration of interior point method for example problem (4.51)
min
subj. to
1
z
2
"
[ 20 01 ] z + [ 4
1
1
3.9669
1.1997
1.1673
2.0588
1.7527
1
1.3622
1.3111
1
2
]z
" 1.0890 #
1.0623
2.0187
1
1
(4.51)
The solutions of P () for this Quadratic Program for different values of are
depicted in Fig. 4.4. For large , the level sets are almost identical to the level sets of
g (z), i.e. z (5) is very close to the analytic center z a , arg min g (z) of the feasible
set. As decreases, z () approaches z , by following the central path.
4.3
Constrained Optimization
53
Example 4.5 [Barrier method for QPs] In the following, we derive the barrier method
for quadratic programs of the form
minimize
subject to
1
z Hz
2
+ q z
Az = b,
Gz h
(4.52)
m
X
X
1
1
gi , 2 g (z) =
g g
2 i i
h
(h
i gi z
i gi z)
i=1
i=1
i=1
(4.53)
where gi denotes the ith row of G and hi the ith element of h. Hence P () takes the
form
m
X
1
ln(hi gi z)
min z Hz + q z
2
(4.54)
i=1
g (z) =
s.t. Az = b
which we solve by Newtons method. Using the formula for the Newton direction pN
with equality constraints (4.33), we arrive at the following linear system of size n + p,
which has to be solved in each Newton step at the current iterate z:
"
#
P
P
1
1
H + m
A pN
Hz + f + m
i=1 (hi g z)2 gi gi
j=1 hi gi z gi
i
. (4.55)
=
y
0
A
0
The resulting barrier method for solving (4.52) is summarized in Algorithm 4.8. Lines
3 8 form the inner Newton iterations of the algorithm, computing z (). Note that
a primal feasible line search must be employed to ensure that the iterates remain in
the interior of Q.
54
We conclude this section with a few remarks on the barrier method. The main
computational burden is the computation of the Newton directions by solving linear
systems of type (4.55). This is usually done using direct methods (matrix factorizations), but iterative linear solvers can be used as well. In most applications the
matrices involved have a special sparsity pattern, which can be exploited to significantly speed up the computation of the Newton direction. We discuss structure
exploitation for solving (4.55) arising from MPC problems in Section 15.3.
If the barrier parameter is decreased too quickly, many inner iterations are
needed, but only a few outer iterations. The situation is reversed if is decreased
only slowly, which leads to a quick convergence of the inner problems (usually in
one or two Newton steps), but more outer iterations are needed. This tradeoff is
not very prominent in practice. Usually, the total number of Newton steps is in
the range of 20 40, essentially independent of the conditioning of the problem.
The barrier method can be shown to converge linearly; more details can be found
in [61, 11.3].
Finally, we would like to point out that the barrier method is a feasible method,
which means that it starts with a feasible point and all iterates remain feasible.
This complicates the implementation in practice, as a line search must be employed
that maintains feasibility, and a strictly feasible initial iterate z0 has to be supplied
by the user or computed first by what is called a Phase I method. See [61] for
more details.
Computational experience has shown that modern primal-dual methods are
significantly more effective with only little additional computational cost when
compared to the barrier method. As a result, primal-dual methods are now predominant in commercial codes. They also allow infeasible iterates, that is, the
equality and inequality constraints on the primal variables are satisfied only at
convergence, which alleviates the necessity for finding a feasible initial point. We
discuss this class of methods next.
Primal-dual methods
The general idea of primal-dual interior point methods is to solve the KarushKuhn-Tucker (KKT) conditions by a modified version of Newtons method. The
nonlinear KKT equations represent necessary and sufficient conditions for optimality for convex problems. For (P-IPM), the KKT conditions are
f (z) + A v + G(z) u = 0
Az b = 0
(4.56a)
(4.56b)
(s, u) 0
(4.56e)
g(z) + s = 0
si ui = 0, i = 1, . . . , m
(4.56c)
(4.56d)
4.3
Constrained Optimization
55
and the matrix G(z) Rms is the Jacobian of g evaluated at z. The variables z
and s are from the primal, the variables v and u from the dual space.
If a primal-dual pair of variables (z , v , u , s ) is found that satisfies these
conditions, the corresponding primal variables (z , s ) are optimal for (P-IPM) (and
the dual variables (v , u ) are optimal for the dual of (P-IPM)). Since Newtons
method is a powerful tool for solving nonlinear equations, the idea of applying it to
the system of nonlinear KKT equations is apparent. However, certain modifications
to a pure Newton method are necessary to obtain a useful method for constrained
optimization. The name primal-dual indicates that the algorithm operates in both
the primal and dual space.
There are many variants of primal-dual interior point methods. For a thorough
treatment of the theory we refer the reader to the book by Wright [281], which
gives a comprehensive overview of the different approaches. In this section, after
introducing the general framework of primal-dual methods, we restrict ourselves
to the presentation of a variant of Mehrotras predictor-corrector method [201],
because it forms the basis of most existing implementations of primal-dual interior
point methods and has proven particularly effective in practice. The method we
present here allows infeasible starting points.
Central Path
We start by outlining the basic idea. Most methods track the central path, which we
have encountered already in barrier methods, to the solution set. These methods
are called path-following methods. In the primal-dual space, the central path
is defined as the set of points (z, v, u, s), for which the following relaxed KKT
conditions hold:
f (z) + A v + G(z) u = 0
Az b = 0
g(z) + s = 0
si ui = , i = 1, . . . , m
(s, u) > 0
(4.57a)
(4.57b)
(4.57c)
(4.57d)
(4.57e)
56
(4.58)
z
f (z) + A v + G(z) u
H(z, u) A G(z) 0
A
Az b
0
0
0
v =
(4.59)
G(z)
g(z) + s
0
0
I u
s
Su
0
0
S
Z
where S , diag(s1 , . . . , sm ) and Z , diag(u1 , . . . , um ), the (1,1) block in the coefficient matrix is
m
X
ui 2 gi (z)
(4.60)
H(z, u) , 2 f (z) +
i=1
and the vector R allows a modification of the right-hand side, thereby generating different search directions that depend on the particular choice of . For
brevity, we denote the solution to (4.59) by [z, v, u, s]().
The the vector can takes a value between the following two extremes:
If = 0, the right hand side in (4.59) corresponds to the residual of the KKT
conditions (4.56), i.e. [z, v, u, s](0) is the pure Newton direction that aims
at satisfying the KKT conditions (4.56) in one step based on a linearization.
This direction is called affine-scaling direction. However, pure Newton steps
usually decrease some products si ui prematurely towards zero, i.e. the generated iterate is close to the boundary of the feasible set (if si ui 0, either
si or ui must be small). This is disadvantageous since from such points only
very small steps can be taken, and convergence would be prohibitively slow.
Hence the iterates must be centered by bringing them closer to the central
path.
If = 1, the complementarity condition (4.57d) is modified such that the
resulting search direction aims at making the individual products si ui equal
4.3
Constrained Optimization
57
to their average value . This is called centering. A centering step does not
decrease the average value, so no progress towards the solution is made.
A tradeoff between the two goals of progressing towards the solution and centering
is achieved by computing search directions [z, v, u, s]() by solving (4.59) for
= 1
(4.61)
where (0, 1) is called the centering parameter. This is the key difference to pure
Newton steps, and the main ingredient (together with tracking the central path)
in making Newtons method work efficiently for solving the nonlinear KKT system (4.56). Figure 4.5 depicts this combination of search directions schematically,
and a principle framework for primal-dual methods is given in Algorithm (4.9). We
will instantiate it with Mehrotras method below.
In the discussion above, we have excluded almost all theoretical aspects concerned with convergence of primal-dual methods. For example, the terms sufficient
close to the central path and sufficient reduction in have precise mathematical
counterparts that allow one to analyze the convergence properties. The book by
Wright [281] is a good starting point for the interested reader.
Algorithm 4.9 Primal-dual interior point methods for (P-IPM)
input: Initial iterates z0 , v0 , u0 > 0, s0 > 0, Centering parameter (0, 1)
output: Point close to z
repeat
Duality measure: i = si ui /m
Search direction: compute [zi , vi , ui , si ](i 1) by solving (4.59) at the
current iterate
Choose step length hi such that si+1 > 0 and ui+1 > 0 Update: (zi+1 , vi+1 , ui+1 , si+1 ) =
(zi , vi , ui , si ) + hi [zi , vi , ui , si ](i i 1)
until stopping criterion
Predictor-Corrector Methods
Modern methods use the full Newton step [z, v, u, s](0) merely as a predictor to
estimate the error made by using a linear model of the central path. If a full step
along the affine-scaling direction were taken, the complementarity condition would
evaluate to
aff
aff
aff
aff
(S + S aff )(u + uaff ) = Su
| + S {zu + Su } +S u
(4.62)
=0
where capital letters denote diagonal matrices constructed from the corresponding vectors. The first three terms sum up to zero by the last equality of (4.59).
Hence the full Newton step produces an error of S aff uaff in the complementarity condition. In order to compensate for this error, a corrector direction
[z, v, u, s](S aff uaff ) can be added to the Newton direction. This compensation is not perfect since usually > 0, that is the predictor direction is only an
58
4.4
59
4.4
4.4.1
c z
Gz w,
(4.64)
(Gj z wj )uj
u
Gz
c,
(4.65a)
0, j = 1, . . . , m
0,
(4.65b)
(4.65c)
w.
(4.65d)
Let us assume that we start our search for the optimum from a feasible vertex
z k in the solution space. At this vertex a set of constraints, indicated by the
matrices GA(zk ) ,wA(zk ) is active (see Section 3.2.4). Then the KKT conditions can
be rewritten as
0
GA(zk )
c
zk
=
(4.66a)
wA(zk )
uA(zk )
GA(zk )
0
GN A(zk ) z k
uA(zk )
wN A(zk )
(4.66b)
0.
(4.66c)
60
An active set strategy aims to determine iteratively the set of active constraints
GA (z k ) and the associated solution [z k , uA(zk ) ] of (4.66a), for which the primal
and dual feasibility conditions (4.66b)-(4.66c) are satisfied. Different sets of active
constraints imply different vertices in the solution space. Thus changing the set of
active constraints implies moving from one vertex to another.
When one searches for the optimal set of active constraints one generally removes and adds one constraint at a time while satisfying primal feasibility (4.66b).
The violation of the dual feasibility conditions (4.66c) gives an indication how to
proceed. One looks for the dual variable constraint (4.66c), which is violated the
most and removes the associated primal constraint from the set of active constraints. One then adds one of the inactive constraints to the set of active constraints, where the choice is made such that the solution of the resulting new
Lagrangian equations is primal feasible. In this way one moves along an edge to a
neighboring feasible vertex in the solution space.
We assumed that our solution procedure starts with a feasible vertex. If a
feasible point is not obvious, e.g., when z = 0 is not feasible, then in Phase 1 of
the procedure a modified LP is defined and solved to determine a feasible point
inf z,s
subj. to
1 s
Gz s w,
s 0.
(4.67)
4.4.2
1
2 z Hz
+ q z + r
Gz w,
(4.68)
=0
(4.69a)
ui (Gi z wi )
u
=0
0
(4.69b)
(4.69c)
Gz w
0.
(4.69d)
4.4
61
Just like in the LP case in the previous section we want to determine the subset of
constraints, which are active at the optimum. For the QP, however, the number of
constraints which are active is not known a priori, because the solution does not
necessarily lie at a vertex in the solution space (Section 3.3).
Before presenting an active set method for solving the QP (4.68), we will first
consider a subset A of the constraints, which are active and the corresponding QP
with these equality constraints only.
minz
subj. to
2 z Hz + q z
GA z = wA .
+r
(4.70)
For this special case the KKT conditions (3.31a)-(3.31b), can be compactly written
as
H GA
z
q
.
(4.71)
=
GA
0
u
wA
The solution to (4.71) has the form
z = Lq + T wA ,
u = T q + SwA .
(4.72a)
(4.72b)
The symmetric matrix in (4.71) is referred as the Lagrangian matrix. If the Lagrangian matrix is nonsingular then the matrices L, T and S can be extracted from
its inverse
1
L T
H GA
=
.
(4.73)
T S
GA
0
Several alternatives for solving (4.73) can be found in [102, p. 236]. In particular, there are instances when the Lagrangian matrix in (4.71) is not invertible or
stability problems occur.
As a simple alternative method, consider the case of a full row rank matrix GA
with m < s rows. Any feasible point z can be written as z = Y wA + Zy, where
y Rsm , and Y and Z satisfying the properties GA Y = I, GA Z = 0 and [Y Z]
non-singular.
One possibility for computing z and u is to use the QR factorization of the
matrix GA
R
R
GA = Q
= [Q1 Q2 ]
= Q1 R
(4.74)
0
0
where Q Rss is orthogonal, R Rmm is upper triangular and Q1 Rsm and
Q2 Rs(sm) . Then the choice
Y = Q1 R1 ,
Z = Q2
(4.75)
satisfies the desired properties for Y and Z. With such a choice, the matrices in
the solution (4.72) can be computed as (cf. [102, Eq. 10.2.7])
L = Z(Z HZ)1 Z ,
T = Y Z(Z HZ)1 Z HY,
S = Y HZ(Z HZ)1 Z HY + Y HY.
(4.76)
62
min
2 H + (q + Hz)
(4.77)
subj. to GA = 0
where z = z k is a known feasible point for the QP (QP) and A = Ak = {j
I : Gj z k = wj } is the associated active constraint set. We want to determine the
direction k to move from the feasible z k , i.e. z k+1 = z k + k .
Problem (4.77) is an equality constrained QP. Let k Rn denote the optimizer
of (4.77) and let uk R|A| be the corresponding vector of Lagrange multipliers.
We note that k is the best correction to z in the direction where the constraints A
are active. Since (4.77) is just a special case of (4.70) it can be solved as explained
before. In particular, with the QR decomposition of GA as in (4.74), and Z and Y
defined by (4.75) we get
k = Z(Z HZ)1 Z (q + Hz),
(4.78)
(4.79)
4.5
63
Compute
= min 1,
min
jI\A, Gj k >0
wj Gj z
Gj k
(4.80)
z z k + k
if < 1
A A {j }, where j is the arg min of (4.80)
endif
k k+1
4.5
64
More generally, the Branch and Bound algorithm for MILPs involves solving
and generating a set of LP subproblems in accordance with a tree search, where
4.5
the nodes of the tree correspond to LP subproblems. Branching generates childnodes from parent-nodes according to branching rules, which can be based, for
instance, on a-priori specified priorities on integer variables, or on the amount by
which the integer constraints are violated. Nodes are labeled either as pending, if
the corresponding LP problem has not yet been solved, or fathomed, if the node
has already been fully explored. The algorithm stops when all nodes have been
fathomed.
The success of the branch and bound algorithm relies on the fact that whole
subtrees can be excluded from further exploration by fathoming the corresponding
root nodes. This happens if the corresponding LP subproblem is either infeasible
or an integer solution is obtained. In the second case, the corresponding value of
the cost function serves as an upper bound on the optimal solution of the MILP
problem, and is used to further fathom any other nodes having either a greater
optimal value or lower bound.
65
66
Chapter 5
5.1
(5.1)
k k
aff(K) = { x + . . . + x
: x K, i = 1, . . . , k,
k
X
i = 1}.
(5.2)
i=1
The affine hull of K is the smallest affine set that contains K in the following sense:
if S is any affine set with K S, then aff(K) S.
68
The dimension of an affine set, affine combination or affine hull is the dimension of the largest ball of radius > 0 included in the set.
Example 5.1 The set
F = {x R2 : x1 + x2 = 1}
is an affine set in R2 of dimension one. The points x1 = [0, 1] and x2 = [1, 0] belong
to the set F. The point x
= 0.2x1 + 1.2x2 = [1.2, 0.2] is an affine combination of
1
2
points x and x . The affine hull of x1 and x2 , aff({x1 , x2 }), is the set F.
k
X
i = 1}. (5.3)
i=1
The convex hull of K is the smallest convex set that contains K in the following
sense: if S is any convex set with K S, then conv(K) S.
Example 5.2 Consider three points x1 = [1, 1] , x2 = [1, 0] , x3 = [0, 1] in R2 . The
point x
= 1 x1 +2 x2 +3 x3 with 1 = 0.2, 2 = 0.2, 3 = 0.6 is x
= [0.4, 0.8] and it
1
2
3
is a convex combination of the points {x , x , x }. The convex hull of {x1 , x2 , x3 }
is the triangle plotted in Figure 5.1. Note that any set in R2 strictly contained in the
triangle and containing {x1 , x2 , x3 } is non-convex. This illustrates that the convex
hull is the smallest set that contains these three points.
x2
1.5
1
0.5
0
-0.5
-0.5
0.5
x1
1.5
5.2
69
k
X
i=1
i xi , i 0, i = 1, . . . , k}.
(5.4)
x3
4
2
0
4
2
x2
x1
(5.5)
By definition, any point in P Q can be written as the sum of two points, one in
P and one in Q. For instance, the Minkowski sum of two balls (P and Q) centered
at the origin and with radius 1, is a ball (P Q) centered at the origin and with
radius 2.
5.2
a3
x
b3
a1 x
b1
a
2x
b2
a4 x
70
Figure 5.3 H-polytope. The planes (here lines) defining the halfspaces are
ai x bi = 0. The arrows represent the direction of ai into the valid
halfspace.
b3
a1 x
b1
a
2x
b2
a
3x
(5.7)
5.2
71
(5.8)
In Section 5.4.4 we show how to convert any H-polytope into a V-polytope and
vice versa.
The dimension of a polytope (polyhedron) P is the dimension of its affine hull
and is denoted by dim(P). We say that a polytope P Rn , P = {x Rn :
P x x P c }, is full-dimensional if dim(P) = n or, equivalently, if it is possible to
fit a nonempty n-dimensional ball in P,
x Rn , > 0 : B(x, ) P,
(5.9)
x Rn , > 0 : kk2 P x (x + ) P c .
(5.10)
or, equivalently,
(5.12)
x2
x2
72
x1
(a) V-representation. The vertices V1P , . . . , V7P are depicted as
dots.
x1
(b) H-representation. The hyperplanes Pix x = Pic , i = 1, . . . , 7 are
depicted as lines.
5.3
Polytopal Complexes
According to our definition every polytope represents a convex, compact (i.e.,
bounded and closed) set. In this book we will also encounter sets that are dis-
5.3
Polytopal Complexes
73
(5.13)
PC
i=1
Example 5.4 A collection R = {[2, 1], [0, 2], [2, 4]} is a P-collection in R1 with the
underlying set R = [2, 1] [0, 4]. As another example, R = {[2, 0], [1, 1], [0, 2]}
is a P-collection in R1 with underlying set R = [2, 2]. Clearly, polytopes that define
a P-collection can overlap, the underlying set can be disconnected and non-convex.
Usually it is clear from the context if we are talking about the P-collection or
referring to the underlying set of a P-collection. Therefore for simplicity, we use
the same notation for both.
C
Definition 5.5 (Strict Polyhedral Partition) A collection of sets {Ci }N
i=1 is a
SNC
strict partition of a set C if (i) i=1 Ci = C and (ii) Ci Cj = , i 6= j. Moreover
NC
C
{Ci }N
i=1 is a strict polyhedral partition of a polyhedral set C if {Ci }i=1 is a strict
partition of C and Ci is a polyhedron for all i, where Ci denotes the closure of the
set Ci .
C
Definition 5.6 (Polyhedral Partition) A collection of sets {Ci }N
i=1 is a partiSNC
tion of a set C if (i) i=1 Ci = C and (ii) (Ci \Ci ) (Cj \Cj ) = , i 6= j. Moreover
NC
C
{Ci }N
i=1 is a polyhedral partition of a polyhedral set C if {Ci }i=1 is a partition of C
and Ci is a polyhedron for all i. The set Cj is the boundary of the set Cj .
Note that in a strict partition some of the sets Ci must be open or neither open
nor closed. In a partition all the sets may be closed and points on the closure of
a particular set may also belong to one or several other sets. Also, note that a
polyhedral partition is a special class of a P-collection.
5.3.1
74
5.4
5.4.1
Minimal Representation
We say that a polytope P Rn , P = {x Rn : P x x P c } is in a minimal
representation if the removal of any row in P x x P c would change it (i.e., if
there are no redundant constraints). The computation of the minimal representation (henceforth referred to as polytope reduction) of polytopes is discussed in [105].
The redundancy of each constraint is checked, which generally requires the solution
of one LP for each half-space defining the non-minimal representation of P. We
summarize this simple implementation of the polytope reduction in Algorithm 5.1.
An improved algorithm for polytope reduction is discussed in [260] where the authors combine the procedure outlined in Algorithm 5.1 with heuristic methods,
such as bounding-boxes and ray-shooting, to discard redundant constraints more
efficiently.
5.4
75
subj. to
x
c
P(I)
x P(I)
, Pix x Pic + 1
end
x
c
let Qx = P(I)
, Qc = P(I)
5.4.2
Convex Hull
i
n
V
The convex hull of a set of points V = {V i }N
i=1 , with V R , is a polytope defined
as
conv(V ) = {x Rn : x =
NV
X
i=1
i V i , i 0,
NV
X
i = 1}.
(5.15)
i=1
i=1
i=1
5.4.3
Envelope
The envelope of two H-polyhedra P = {x Rn : P x x P c } and Q = {x Rn :
Qx x Qc } is an H-polyhedron
env(P, Q) = {x Rn : Px x Pc , Qx x Qc },
(5.17)
76
R1
R2
(a) R =
x2
x2
x1
x1
i{1,2}
conv(R)
Ri .
R1
R2
(a) R =
x1
i{1,2}
x2
x2
env(R)
x1
Ri .
5.4.4
Vertex Enumeration
The operation of extracting the vertices of a polytope P given in H-representation
is referred to as vertex enumeration. The necessary computational effort is expo-
5.4
77
nential in the number of facets. Two different approaches for vertex enumeration
are commonly used: the double description method [107] and reverse search [10].
An efficient implementation of the double description method is available in [104].
Converting a V-polytope into an H-polytope, the reverse operation, is also done
via vertex enumeration as we will explain next. The procedure is based upon the
polar dual.
Definition 5.11 The polar dual of a set P is
P = {y | y x 1, x P } .
Proof: From the definition of the polar dual, we see that P is exactly the set
of all inequalities that support P:
P = {y | y x 1 x such that Ax 1 }
P = {y | y x 1 x = V , 0, 1 = 1 }
= {y | V y 1 0, 1 = 1 }
= {y | V y 1 }
Lemma 5.3 P = P.
Procedure for converting a V-polytope into an H-polytope
Given a V-polytope P = {V | 0, 1 = 1 }
P = {x | V x 1 }
2. Use vertex enumeration to compute the V-representation of the dual
P = {A | 0, 1 = 1 }
3. Use Lemma 5.2 and Lemma 5.3 to recover the H-representation of P
P = P = {x | Ax 1 }
11
78
5.4.5
Chebychev Ball
The Chebychev Ball of a polytope P = {x Rn : P x x P c }, with P x RnP n ,
P c RnP , corresponds to the largest radius ball B(xc , R) with center xc , such that
B(xc , R) P. The center and radius of the Chebychev ball can be easily found by
solving the following linear optimization problem
max R
(5.18a)
xc ,R
i = 1, . . . , nP ,
(5.18b)
where Pix denotes the i-th row of P x . This can be proven as follows. Any point x
of the ball can be written as x = xc + v where v is a vector of length less or equal to
R. Therefore the center and radius of the Chebychev ball can be found by solving
the following optimization problem
max R
(5.19a)
xc ,R
(5.19b)
(5.20)
5.4
79
x2
polytope, e.g., center of the largest volume ellipsoid, analytic center, etc., but those
computations involve the solution of Semi-Definite Programs (SDPs) and therefore
they are more expensive than the Chebyshev Ball computation [61]. An illustration
of the Chebyshev Ball is given in Figure 5.8.
x1
Figure 5.8 Illustration of the Chebychev ball contained in a polytope P.
Projection
Given a polytope P = {[x y ] Rn+m : P x x+ P y y P c } Rn+m the projection
onto the x-space Rn is defined as
projx (P) = {x Rn : y Rm : P x x + P y y P c }.
(5.21)
x3
5.4.6
x2
x1
methods that can operate in general dimensions can be grouped into four classes:
Fourier elimination [78, 164], block elimination [13], vertex based approaches [106]
and wrapping-based techniques [155]. For a good introduction to projection, we
refer the reader to [155] and the references therein.
80
5.4.7
Set-Difference
The set-difference of two polytopes Y and R0
R = Y \ R0 = {x Rn : x Y, x
/ R0 },
(5.22)
in general, can be
and can be described as a PS set T
Sma nonconvex and disconnected
Sm
Y). The P-collection R =
(R
collection
R
=
R
,
where
Y
=
R
0
i
i
i=1
i=1
Sm
i=1 Ri can be computed by consecutively inverting the half-spaces defining R0
as described in the following Theorem 5.2.
Note that here we use the term P-collection in the dual context of both Pcollection and its underlying set (cf. Definitions 5.3 and 5.4). The precise statement
would say that R = Y \ R0 , where R is the underlying set of the P-collection R =
{Ri }m
i=1 . However, whenever it is clear from context, we will use the former, more
compact form.
Theorem 5.2 [39] Let Y TRn be a polyhedron, R0 , {x Rn : Ax b}, and
0 , {x Y : Ax b} = R0 Y, where b Rm , R0 6= and Ax b is a minimal
R
representation of R0 . Also let
Ai x > bi
, i = 1, . . . , m, j = 1, . . . , m 1.
Ri =
xY :
Aj x bj , j < i
S
Let R , m
i=1 Ri . Then, R is a P-collection and {R0 , R1 , . . . , Rm } is a strict
polyhedral partition of Y.
0
Proof: (i) We want to prove that given an x Y, then either x belongs to R
or to Ri for some i but not both. If x R0 , we are done. Otherwise, there exists
an index i such that Ai x > bi . Let i = min{i : Ai x > bi }. Then x Ri , as
im
+
{x
1 x1 x1 , x2 x2 x2 }, and R0 by the inequalities {g1 0, . . . , g5 0}
where g1 , . . ., g5 are linear in x. The procedure consists of considering one by one
the inequalities which define R0 . Considering, for example, the inequality g1 0,
the first set of the rest of the region Y\R0 is given by R1 = {g1 0, x1 x
1 , x2
+
x2 x2 }, which is obtained by reversing the sign of the inequality g1 0 and
removing redundant constraints in Y (see Figure 5.10(b)). Thus, by considering
the rest of
S5the inequalities we get the partition of the rest of the parameter space
Y\R0 = i=1 Ri , as reported in Figure 5.10(d).
Remark 5.1 The set difference of two intersecting polytopes P and Q (or any closed
sets) is not a closed set. This means that some borders of polytopes Ri from a Pcollection R = P \ Q are open, while other borders are closed. Even though it is
5.4
x2
x+
2
81
x2
x+
2
Y
g1 0 g1 0
R0
R1
x
2
x+
1
x
1
x1
x2
g1 0
R1
R0
x
2
x+
1
x
1
x+
2
R2
R2
g2 0
R1
x
2
x
1
(c) Partition of Y\R0 - Step 2.
x1
x2
x+
2
R0
x+
1
x1
R0
R5
R3
R4
x
2
x
1
x+
1
Figure 5.10 Two dimensional example: partition of the rest of the space
Y\R0 .
possible to keep track of the origin of particular borders of Ri , thus specifying if they
are open or closed, we are not doing so in the algorithms described in this book nor
in MPT [174], cf. Remark ??. In computations, we will henceforth only consider the
closure of the sets Ri .
x1
82
5.4.8
Pontryagin Difference
The Pontryagin difference (also known as Minkowski difference) of two polytopes
P and Q is a polytope
P Q = {x Rn : x + q P, q Q}.
(5.23)
Q = {z Rn : Qz z Qb },
(5.24)
then
W =P Q
= {x R
(5.25a)
n
: P x P H(P , Q)},
(5.25b)
(5.26)
xQ
and Piy is the i-th row of the matrix P y . Note that for special cases (e.g. when Q
is a hypercube), more efficient computational methods exist [167]. An illustration
of the Pontryagin difference is given in Figure 5.11a.
8
6
P Q
-2
-4
-6
P Q
P
x2
x2
6
4
2
-2
-6
-8
-6 -4 -2
x1
2 4 6
-8 -6
-2
x1
6 8
5.4.9
Minkowski Sum
The Minkowski sum of two polytopes P and Q is a polytope
P Q = {y + z Rn : y P, z Q}.
(5.27)
5.4
83
Q = {z Rn : Qz z Qc },
=
=
=
=
=
P Q
o
n
x Rn : x = y + z, P y y P c , Qz z Qc , y, z Rn
o
n
x Rn : y Rn , subj. to P y y P c , Qz (x y) Qc
c o
n
0
Py
x
P
x Rn : y Rn , subj. to
Qz Qz y
Qc
c o!
n
x
0
Py
x
P
n+n
projx
R
:
.
y
Qz Qz y
Qc
Both the projection and vertex enumeration based methods are implemented in
the MPT toolbox [174]. An illustration of the Minkowski sum is given in Figure
5.11b.
Remark 5.2 The Minkowski sum is not the complement of the Pontryagin difference.
For two polytopes P and Q, it holds that (P Q) Q P. This is illustrated in
Figure 5.12.
3
2
3
2
(P Q) Q
1
-1
-2
-3
-3
-3
-1
x1
P Q
x2
P Q
x2
x2
-2
-3
-3 -2
x1
difference P Q.
-3 -2
x1
84
5.4.10
Polyhera Union
Consider the following basic problem in polyhedral computation: given two polyhedra P Rn and Q Rn , decide whether their union is convex, and, if so,
compute it. There are three classes of algorithms for the given problem depending
on their representation: (1) P and Q are given in H-representation, (2) P and Q
are given in V-representation, (3) both H- and V-representations are available for
P and Q. Next we present an algorithm for case (1). Case (2), case (3) and the
computational complexity of all three cases are discussed in [33].
Recall the definition of envelope in Section 5.4.3. By definition, it is easy to see
that env(P, Q) is convex and that
P Q env(P, Q).
(5.28)
max(x,)
subj. to
i x
A
i +
j x j +
B
Cx ;
/* = if the LP is infeasible,
= if the LP is unbounded */
if > 0, stop, return nonconvex
end
return env(P, Q).
/* P Q is convex. */
Note that if = 0 for each i, j, then the union is convex and equals env(P, Q).
On the other hand, > 0 indicates the existence of a point x env(P, Q) outside
P Q.
For recognizing convexity and computing the union of k polyhedra, the test
can be modified by checking each k-tuple of removed constraints. Let m
1 , . . ., m
k
be the number of removed constrains from the polyhedra
P
,
.
.
.
,
P
,
respectively.
1
k
Qk
Then similarly to the for loop in Algorithm 5.2, i=1 m
i linear programs need to
be solved in the general case.
5.5
5.4.11
Operations on P-collections
85
(5.30)
(5.31)
(5.33)
Remark 5.3 Often in the literature the symbol is omitted for linear maps f = Az.
Therefore, AP refers to the operation A P and PA refers to the operation P A.
5.5
Operations on P-collections
This section covers some results and algorithms which are specific to operations
with P-collections. P-collections are unions of polytopes (see Definition 5.3) and
therefore the set of points contained in a P-collection can be represented in an
infinite number of ways, i.e. the P-collection representation is not unique. For
example, one can subdivide any polytope P into a number of smaller polytopes
whose union is a P-collection which covers P. Note that the complexity of all
subsequent computations depends strongly on the number of polytopes representing
a P-collection. The smaller the cardinality of a P-collection, the more efficient the
computations. The reader is referred to [234, 233] for proofs and comments on
computational efficiency.
86
5.5.1
Set-Difference
The first two results given here show how the set difference of a P-collection and a
P-collection (or polyhedron) may be computed.
S
Lemma 5.4 Let C , j{1,...,J} Cj be a P-collection, where all the Cj , j {1, . . . , J},
S
are nonempty polyhedra. If S is a nonempty polyhedron, then C\S = j{1,...,J} (Cj \
S) is a P-collection.
S
S
Lemma 5.5 Let the sets C , j{1,...,J} Cj and D , y=1,...,Y Dy be P-collections,
where all the Cj , j {1, . . . , J}, and Dy , y {1, . . . , Y }, are nonempty polyhedra.
If E0 , C and Ey , Ey1 \ Dy , y {1, . . . , Y } then C \ D = EY is a P-collection.
The condition C D can be easily verified since C D C \ D = . Similarly
C = D is also easily verified since
C = D (C \ D = and D \ C = ).
Next, an algorithm for computing the Pontryagin difference of a P-collection and
c
a polytope is presented. If S and B are two subsets of Rn then S B = [S c (B)]
c
where () denotes the set complement. The following algorithm implements the
computation of the Pontryagin difference of a P-collection C , j{1,...,J} Cj , where
Cj , j {1, . . . , J} are polytopes in Rn , and a polytope B Rn .
Algorithm 5.3 Pontryagin Difference for P-collections, C B
input: P-collection C, polytope B;
output: P-collection G , C B.
H , env(C) (or H , conv(C))
D ,HB
E ,H\C
F , E (B)
G ,D\F
Remark 5.4 Note that H in Algorithm 5.3 can be any convex set containing the
P-collection C. The computation of H is generally more efficient if the envelope
operation is used instead of convex hull.
S
S
Remark 5.5 It is important to note that ( j{1,...,J } Cj ) B =
6
j{1,...,J } (Cj B),
where B and Cj are polyhedra; hence, the relatively high computational effort of
computing the Pontryagin difference of a P-collection and a polytope.
Operations on P-collections
C2
C1
x2
x1
x1
j{1,...,J } Cj and B.
x2
(b) H = conv(C).
E
D
x2
(a)
87
x2
5.5
x1
x1
(d) E = H \ C.
x2
x2
(c) D = H B.
x1
(e) F = E (B).
x1
(f) G = D \ F .
88
= {x D : w B s.t. x + w E}
= {x D : x + w
/ E, w B}.
Algorithm 5.3 is illustrated on a sample P-collection in Figures 5.13(a) to 5.13(f).
Remark 5.6 It should be noted that Algorithm 5.3 for computation of the Pontryagin
difference is conceptually similar to the algorithm proposed in [254, 165]. However, in
general the envelope operation H , env(C) employed in Algorithm 5.3 significantly
reduces the number of sets obtained when computing H \ C, which in turn results in
fewer Minkowski set additions. Since the computation of a Minkowski set addition is
expensive, a runtime improvement can be expected.
5.5.2
Polytope Covering
The problem of checking if some P polytope is covered with the union of other
polytopes, i.e. a P-collection Q = i Qi is discussed in this section. We consider
two related problems:
polycover: Check if P Q, and
regiondiff: Compute P-collection R = P \ Q.
Clearly, polycover is just a special case of regiondiff, where the resulting Pcollection R = . Also, it is straightforward to extend the above problems to the
case where both P and Q are both P-collections.
5.5
Operations on P-collections
One idea of solving the polycover problem is inspired by the following observation
P Q P = i (P Qi ).
Therefore, we could create Ri = P Qi , for i = 1, . . . , NQ and then compute the
union of the collection of polytopes {Ri } by using the polyunion algorithm for
computing the convex union of H-polyhedra reported discussed in Section 5.4.10.
If polyunion succeeds (i.e., the union is a convex set) and the resulting polytope is
equal to P then P is covered by Q, otherwise it is not. However, this approach is
computationally very expensive.
More details can be found in [116].
5.5.3
Union of P-collections
Consider a P-collection P = {Pi }pi=1 . We study the problem of finding a minimal
representation of P by merging one or more polyhedra belonging to the P-collection.
Clearly one could use the polyunion algorithm presented in Section 5.4.10 for all
possible subsets of the P-collection and solve the problem by comparing all solutions. However this approach is not computationally efficient.
Our interest in this problem will be clear later in this book (Section 12.1) when
computing the PWA state-feedback control law to optimal control problems. Once
the PWA state-feedback control law has been derived, the memory requirement and
the on-line computation time are linear in the number of polyhedra of the feedback
law when using standard brute force search. Therefore, we will be interested in the
problem of finding a minimal representation of piecewise affine (PWA) systems,
or more specifically, for a given PWA system, we solve the problem of deriving a
PWA system, that is both equivalent to the former and minimal in the number of
regions. This is done by associating a different color to a different feedback law,
and then collecting the polyhedra with the same color. Then, for a given color, we
try to merge the corresponding P-collection. If the number of polyhedra with the
same affine dynamic is large, the number of possible polyhedral combinations for
merging explodes. As most of these unions are not convex or even not connected
and thus cannot be merged, trying all combinations using standard techniques
based on linear programming (LP) is prohibitive. Furthermore, our objective here
is not only to reduce the number of polyhedra but rather to find the minimal and
thus optimal number of disjoint polyhedra. This problem is known to be N P-hard.
In [116] details on the solutions of these problems can be found.
89
90
Part II
Multiparametric Programming
Chapter 6
Multiparametric Nonlinear
Programming
The operations research community has addressed parameter variations in mathematical programs at two levels: sensitivity analysis, which characterizes the change
of the solution with respect to small perturbations of the parameters, and parametric programming, where the characterization of the solution for a full range of
parameter values is studied. In this chapter we introduce the concept of multiparametric programming and recall the main results of nonlinear multiparametric
programming. Later in this book we will use multiparametric programming to
characterize and compute the state-feedback solution of optimal control problems.
6.1
subj. to
J(z, x)
g(z, x) 0
94
of various outcomes.
Our interest in multiparametric programming arises from the field of system
theory and optimal control. For example, for discrete-time dynamical systems,
finite time constrained optimal control problems can be formulated as mathematical
programs where the cost function and the constraints are functions of the initial
state of the dynamical system. In particular, Zadeh and Whalen [284] appear to
have been the first ones to express the optimal control problem for constrained
discrete-time linear systems as a linear program. We can interpret the initial state
as a parameter. By using multiparametric programming we can characterize and
compute the solution of the optimal control problem explicitly as a function of the
initial state.
We are further motivated by the model predictive control (MPC) technique.
MPC is very popular in the process industry for the automatic regulation of process
units under operating constraints, and has attracted a considerable research effort
in the last two decades. MPC requires an optimal control problem to be solved
on-line in order to compute the next command action. This mathematical program
depends on the current sensor measurements. The computation effort can be moved
off-line by solving multiparametric programs, where the control inputs are the
optimization variables and the measurements are the parameters. The solution of
the parametric program problem is a control law describing the control inputs as
function of the measurements. Model Predictive Control and its multiparametric
solution are discussed in Chapter 13.
In the following we will present several examples that illustrate the parametric
programming problem and hint at some of the issues that need to be addressed by
the solvers.
Example 6.1 Consider the parametric quadratic program
J (x) = min
z
subj. to
1 2
z + 2xz + 2x2 + u(z x 1)
2
and the KKT conditions are (see Section 3.3.3 for KKT conditions for quadratic
programs)
z + 2x + u
=0
(6.1a)
u(z x 1)
=0
(6.1b)
zx1
0.
(6.1c)
(6.1d)
6.1
95
x < 13
u>0
1
2
z + 2x + u = 0
z = 2x
J = 0
B. z x 1 0
x 13
u=0
(6.2)
J (x)
1
0
-1
-2
-2
z (x)
-1
0
x
Figure 6.1 Example 6.1: Optimizer z (x) and value function J (x) as a
function of the parameter x.
The above simple procedure, which required nothing but the solution of the KKT
conditions, yielded the optimizer z (x) and the value function J (x) for all values
of the parameter x. The set of admissible parameter values was divided into two
critical regions, defined by x < 13 and x 31 . In the region x < 13 the
inequality constraint is active and the Lagrange multiplier is greater than zero, in
the other region x 31 the Lagrange multiplier is equal to zero and the inequality
constraint is not active (with the exception of the boundary point 31 ).
In general, when there are more than one inequality constraints a critical region
is defined by the set of inequalities that are active in the region. Throughout a
critical region the conditions for optimality derived from the KKT conditions do
not change. For our example, in each critical region the optimizer z (x) is affine and
the value function J (x) is quadratic. Thus, considering all x, z (x) is piecewise
affine and J (x) is piecewise quadratic. Both z (x) and J (x) are continuous, but
z (x) is not continuously differentiable.
In much of this book we will be interested in two questions: how to find the value
function J (x) and the optimizer z (x) and what are their structural properties,
e.g., continuity, differentiability and convexity. Such questions have been addressed
for general nonlinear multiparametric programming by several authors in the past
(see [16] and references therein), by making use of quite involved mathematical
theory based on the continuity of point-to-set maps. The concept of point-to-set
maps will not be much used in this book. However, it represents a key element for a
96
6.2
subj. to
J(z, x)
g(z, x) 0,
(6.3)
K = {x X : R(x) 6= },
(6.5)
by J (x) the real-valued function that expresses the dependence of the minimum
value of the objective function over K on x
J (x) = inf {J(z, x) : z R(x)},
z
(6.6)
and by Z (x) the point-to-set map which assigns the (possibly empty) set of optimizers z 2Z to a parameter x X
Z (x) = {z R(x) : J(z, x) = J (x)}.
(6.7)
J (x) will be referred to as optimal value function or simply value function, Z (x)
will be referred to as the optimal set. If Z (x) is a singleton for all x, then z (x) ,
Z (x) will be called optimizer function. We remark that R and Z are set-valued
functions. As discussed in the notation section, with abuse of notation J (x) and
Z (x) will denote both the functions and the value of the functions at the point x.
The context will make clear which notation is being used.
The book by Bank and coauthors [16] and Chapter 2 of [101] describe conditions
under which the solution of the nonlinear multiparametric program (6.3) is locally
well behaved and establish properties of the optimal value function and of the
optimal set. The description of such conditions requires the definition of continuity
6.2
97
of point-to-set maps. Before introducing this concept we will show through two
simple examples that continuity of the constraints gi (z, x) with respect to z and
x is not enough to imply any regularity of the value function and the optimizer
function.
Example 6.2 [ [16, p. 12]]
Consider the following problem:
J (x) = minz
subj. to
x2 z 2 2x(1 x)z
z0
0 x 1.
(6.8)
Cost and constraints are continuous and continuously differentiable. For 0 < x 1
the optimizer function is z = (1 x)/x and the value function is J (x) = (1 x)2 .
For x = 0, the value function is J (x) = 0 while the optimal set is Z = {z R : z
0}. Thus, the value function is discontinuous at 0 and the optimal set is single-valued
for all 0 < x 1 and set-valued for x = 0.
Example 6.3 Consider the following problem:
J (x) = inf
z
subj. to
z
zx 0
10 z 10
10 x 10,
(6.9)
where z R and x R. For each fixed x the set of feasible z is a segment. The
point-to-set map R(x) is plotted in Figure 6.2a. The function g1 : (z, x) 7 zx is
continuous. Nevertheless, the value function J (x) = z (x) has a discontinuity at the
origin as can be seen in Figure 6.2b.
5
J (x)
10
R(x)
10
0
-5
0
-5
-10
-10
-10
-5
0
x
10
-10
-5
0
x
10
z1 ,z2
subj. to
z1
g1 (z1 , z2 ) + x 0
g2 (z1 , z2 ) + x 0,
(6.10)
98
where examples of the functions g1 (z1 , z2 ) and g2 (z1 , z2 ) are plotted in Figures 6.3(a)
6.3(c). Figures 6.3(a)6.3(c) also depict the point-to-set map R(x) = {[z1 , z2 ]
R2 |g1 (z1 , z2 ) + x 0, g2 (z1 , z2 ) + x 0} for three fixed x. Starting from x = x
1 ,
as x increases, the domain of feasibility in the space z1 , z2 shrinks; at the beginning
it is connected (Figure 6.3(a)), then it becomes disconnected (Figure 6.3(b)) and
eventually connected again (Figure 6.3(c)). No matter how smooth one chooses the
functions g1 and g2 , the value function J (x) = z1 (x) will have a discontinuity at
x=x
3 .
z2
z2
g1 (z1 , z2 ) = x
1
g1 (z1 , z2 ) = x
2
z1
z1
g2 (z1 , z2 ) = x
2
g2 (z1 , z2 ) = x
1
(a) Set R(
x1 ) shaded in gray.
(b) Set R(
x2 ) shaded in gray.
J (x)
z2
g1 (z1 , z2 ) = x
3
z1
x
1
x
3
g2 (z1 , z2 ) = x3
(c) Set R(
x3 ) shaded in gray.
Figure 6.3 Example 6.4: problem (6.10). (a)-(c) Projections of the point-toset map R(x) for three values of the parameter x: x
1 < x
2 < x
3 ; (d) Value
function J (x).
6.2
Examples 6.2, 6.3, 6.4 show the case of simple and smooth constraints which
lead to a discontinuous behavior of value function and in Examples 6.3, 6.4 we also
observe discontinuity of the optimizer function. The main causes are:
in Example 6.2 the feasible vector space Z is unbounded (z 0),
in Examples 6.3 and 6.4 the feasible point-to-set map R(x) (defined in (6.4))
is discontinuous, as defined precisely below.
In the next sections we discuss both cases in detail.
k
k
pair of sequences {x } K , and z R(x ) with the properties
xk x
, z k z.
it follows that z R(
x).
We define the continuity of a point-to-set map according to Hogan [146] as follows:
99
100
z
1
{zk }
0.5
{xk }
x
Figure 6.4 Example 6.5: Open and not closed point-to-set map R(x).
The point-to-set map R(x) is plotted in Figure 6.5. It is easy to verify that R(x) is
closed but not open. Choose z = 0.75 R(
x). Then, for any sequence {xk } that
converges to x
= 1 from the right, one is not able to construct a sequence {z k } Z
k
such that z R(xk ) for and z k z. In fact, such sequence z k will always be
bounded between 0 and 0.5.
z
1
0.5
{zk }
{xk }
1
Figure 6.5 Example 6.6: Closed and not open point-to-set map R(x).
6.2
101
Remark 6.2 Note that convexity in z for each x is not enough to imply the continuity
of R(x) everywhere in K . In [16, Example 3.3.1 on p. 53] an example illustrating
this is presented. We remark that in Example 6.3 the origin does not satisfy the last
hypothesis of Theorem 6.1.
1
}.
2
(6.12)
Remark 6.5 Based on [16, Theorem 3.2.1-(I) and Theorem 3.3.3], the hypotheses of
Theorem 6.1 can be relaxed for affine gi (z, x). In fact, affine functions are weakly
analytic functions according to [16, p. 47]. Therefore, we can state that if Z is convex,
if each component gi (z, x) of g(z, x) is an affine function, then R(x) is continuous at
x
for all x
K .
S
if there exist a neighborhood N of x
such that the closure of the set xN R(x) is
compact.
Now we are ready to state the two main theorems on the continuity of the value
function and of the optimizer function.
102
Theorem 6.2 [146, Theorem 7] Consider problem (6.3)(6.4). If R(x) is a continuous point-to-set map at x
and uniformly compact near x and if J is continuous
on x
R(
x), then J is continuous at x
.
Theorem 6.3 [146, Corollary 8.1] Consider problem (6.3)(6.4). If R(x) is a
continuous point-to-set map at x
, J is continuous on x
R(
x), Z is nonempty
Example 6.7 [Example 6.2 revisited] Consider Example 6.2. The feasible map R(x)
is unbounded and therefore it does not satisfy the assumptions of Theorem 6.2 (since
it is not uniformly compact). Modify Example 6.2 as follows:
J (x) = minz
subj. to
x2 z 2 2x(1 x)z
0zM
0x1
(6.13)
No matter how large we choose M , the value function and the optimal set are continuous for all x [0, 1].
Example 6.8 [Example 6.3 revisited] Consider Example 6.3. The feasible map R(x)
is not continuous at x = 0 and therefore it does not satisfy the assumptions of
Theorem 6.2. Modify Example 6.3 as follows:
J (x) = inf
z
subj. to
z
zx
10 z 10
10 x 10,
(6.14)
where > 0. The value function and the optimal set are depicted in Figure 6.6 for
= 1. No matter how small we choose , the value function and the optimal set are
continuous for all x [10, 10]
subj. to Gz w + Sx,
(6.15)
6.2
5
J (x)
10
R(x)
10
0
-5
103
0
-5
-10
-10
-10
-5
0
x
10
-10
-5
0
x
10
Corollary 6.2 (mp-QP) Consider the special case of the multiparametric program (6.3). where the objective is quadratic and the constraints are linear
J (x) =
min
z
1
2 z Hz
+ z F
subj. to Gz w + Sx,
(6.16)
Remark 6.7 We remark that Corollary 6.1 requires the existence of optimizer z (
x)
with a bounded cost. This is implicitly guaranteed in the mp-QP case since in
Corollary 6.2 the matrix H is assumed to be strictly positive definite. Moreover, the
existence of an optimizer z (
x) with a bounded cost guarantees that J (x) is bounded
for all x in K . This has been proven in [108, p. 178, Theorem 1] for the mp-LP case
and it is immediate to prove for the mp-QP case.
Remark 6.8 Both Corollary 6.1 (mp-LP) and Corollary 6.2 (mp-QP) could be formulated stronger: J and Z are even Lipschitz-continuous. J is also piecewise affine
(mp-LP) or piecewise quadratic (mp-QP), and for the mp-QP z (x) is piecewise
affine. For the linear case, Lipschitz continuity is known from Walkup-Wets [274] as
a consequence of Hoffmans theorem. For the quadratic case, Lipschitz continuity follows from Robinson [242], as e.g. shown by Klatte and Thiere [170]. The piecewise
104
Chapter 7
Multiparametric Programming:
a Geometric Approach
In this chapter we will concentrate on multiparametric linear programs (mp-LP),
multiparametric quadratic programs (mp-QP) and multiparametric mixed-integer
linear programs (mp-MILP).
The main idea of the multiparametric algorithms presented in this chapter is to
construct a critical region in a neighborhood of a given parameter, by using necessary and sufficient conditions for optimality, and then to recursively explore the
parameter space outside such a region. For this reason the methods are classified as
geometric. All the algorithms are easy to implement once standard solvers are
available: linear programming, quadratic programming and mixed-integer linear
programming for solving mp-LP, mp-QP and mp-MILP, respectively. A literature
review is presented in Section 7.6.
7.1
7.1.1
subj. to
Gz w + Sx,
(7.1)
(7.2)
K = {x K : z satisfying Gz w + Sx}.
(7.3)
106
7.1.2
It is clear that A(x) and N A(x) are disjoint and their union is I.
Definition 7.2 Consider a set A I. The critical region associated with the set
of active constraints A is defined as
CRA , {x K : A(x) = A}.
(7.4)
The set CRA is the set of all parameters x such that the constraints indexed by A
are active at the optimum of problem (7.1). Our first objective is to work with fulldimensional critical regions. For this reason, next we discuss how the dimension of
the parameter space can be reduced in case it is not full-dimensional.
7.1
7.1.3
107
subj. to
J(z,
x)
z1 + z2
z1 z2
z1 + z2
z1 z2
z1
z1
9 x1 x2
1 x1 x2
7 + x1 + x2
1 + x1 + x2
4
4
20 x2 ,
(7.5)
(7.6)
where we have identified many of the inequalities to be hidden equalities. This can
be simplified to
x1 + x2 = 1
(7.7)
x2 16.
Thus
x1 + x2 = 1
100 x2 16
(7.8)
The example shows that the polytope K is contained in a lower dimensional subspace
of K, namely a line segment in R2 .
Our goal is to identify the hidden equality constraints (as in (7.6)) and use
them to reduce the dimension of the parameter space (as in (7.7)) for which the
multiparametric program needs to be solved.
Definition 7.3 A hidden equality of the polyhedron C = { Rs : B v} is an
inequality Bi vi such that Bi = vi C.
To find hidden equalities we need to solve
vi , min Bi
subj. to B v,
for all constraints i = 1, . . . , m. If vi = vi , then Bi = vi is a hidden equality.
108
wnh + Snh x
= wh + Sh x,
(7.9a)
(7.9b)
7.2
7.2.1
subj. to
z+1
z 1+x
z 0,
7.2
109
z0, z1+x
z + 1,
u1 (z + x + 1)
= 1
(7.12a)
=0
(7.12b)
u2 (z)
=0
(7.12c)
u1
(7.12d)
u2
z + x + 1
z
(7.12e)
0.
(7.12g)
(7.12f)
z + x + 1 = 0
u1 = 1, u2 = 0
A. z < 0
J = 2 + x
u1 > 0
x > 1
u2 = 0
u1 + u2 = 1
z =0
z + x + 1 < 0
u1 = 0, u2 = 1
B. z = 0
J = 1
u1 = 0
x < 1
u2 > 0
(7.13)
u1 + u2 = 1
z =0
z + x + 1 = 0
u1 0, u2 0, u1 + u2 = 1
C. z = 0
J = 1
u1 0
x = 1
u2 0
The above simple procedure, which required nothing but the solution of the KKT
conditions, yielded the optimizer z (x) and the value function J (x) for all values
of the parameter x. The set of admissible parameters values was divided into three
critical regions, defined by x < 1, x > 1 and x = 1. In the region x < 1
the first inequality constraint is active (z = 1 + x) and the Lagrange multiplier u1
is greater than zero, in the second region x > 1 the second inequality constraint
is active (z = 0) and the Lagrange multiplier u2 is greater than zero. In the third
region x = 1 both constraints are active and the Lagrange multipliers belong to
the set u1 0, u2 0, u1 + u2 = 1. Throughout a critical region the conditions
110
1
0
J (x)
z (x)
-1
Figure 7.1 Example 7.2: Optimizer z (x) and value function J (x) as a
function of the parameter x.
for optimality derived from the KKT conditions do not change. For our example, in
each critical region the optimizer z (x) is affine and the value function J (x) is also
affine.
7.2.2
subj. to
G u = c
u 0.
(7.14)
The dual feasibility, complementary slackness and primal feasibility conditions for
problems (7.11), (7.14) are
G u = c, u 0
(Gj z wj Sj x)uj = 0, j I
Gz w + Sx.
(7.15a)
(7.15b)
(7.15c)
Let us assume that we have determined the optimal partition (A, N A) , (A(x ), N A(x ))
for some x K . The primal feasibility condition (7.15c) can be rewritten as
GA z SA x = wA
GN A z SN A x < wN A .
(7.16a)
(7.16b)
7.2
111
(7.17)
z
U1
U2
P 1
q
z2
=
.
(7.18)
0|A|ll 0|A|l|A|l D
r
x
We partition (7.16b) accordingly
E
z1
SN A x < wN A .
z2
(7.19)
112
(7.20)
(7.21)
(7.22)
where:
Px
Pz2 x
{x : Dx = r} ,
(7.23)
z2
1
1
1
: (F EU1 U2 )z2 + (SN A EU1 P )x < wN A EU1 q .
x
(7.24)
In other words:
CRA = Px projx (Pz2 x ).
(7.25)
We can now state some fundamental properties of the critical regions, value
function and optimizer inside a critical region.
Theorem 7.2 Let (A, N A) , (A(x ), N A(x )) for some x K
i) CRA is an open polyhedron of dimension d where d = n rank GA
rank GA . If d = 0 then CRA = {x }.
SA +
ii) If rank GA = s (recall that z Rs ) then the optimizer z (x) is unique and
given by an affine function of the state inside CRA , i.e., z (x) = Fi x + Gi
for all x CRA .
iii) If the optimizer is not unique in CRA then Z (x) is an open polyhedron for
all x CRA .
iv) J (x) is an affine function of the state, i.e., J (x) = ci x+di for all x CRA .
Proof:
i) Polytope Pz2 x (7.24) is open and nonempty, therefore it is full-dimensional
in the (z2 , x) space and dim projx (Pz2 x ) = n. Also,
dim Px = n rank D = n (rank GA SA rank GA ).
Since the intersection of Px and projx (Pz2 x ) is nonempty (it contains at least
the point x ) we can conclude that
dim CRA = n rank GA SA + rank GA .
Since we assumed that the set K in (7.2) is bounded, CRA is bounded. This
implies that CRA is an open polytope since it is the intersection of an open
polytope and the subspace Dx = r. In general, if we allow K in (7.2) to be
unbounded, then the critical region CRA can be unbounded.
7.2
113
ii) Consider (7.20) and recall that l , rank GA . If l = s, then the primal
optimizer is unique, U2 is an empty matrix and
z (x) = z1 (x) = U11 (P x + q).
(7.26)
iii) If the primal optimizer is not unique in CRA then Z (x) in CRA is the
following point-to-set map: Z (x) = {[z1 , z2 ] : z2 Pz2 x , U1 z1 +U2 z2 +P x =
q)}. Z (x) is an open polyhedron since Pz2 x is open.
iv) Consider the dual problem (7.14) and one of its optimizer u0 for x = x .
By definition of a critical region u0 remains optimal for all x CRA(x ) .
Therefore the value function in CRA(x ) is
J (x) = (w + Sx) u0 ,
(7.27)
7.2.3
subj. to
z1 + z2
z1
z3
1
z
1
z
+
z
1
3
z2 z3
z2 z3 z4
z2 + z3
z2 + z3 + z4
z3
z
3
z4
z4
x1 + x2
x2
x1 x2
x2
x1 + 2x2
x2
x1 2x2
x2
1
1
1
1,
(7.29)
114
where K is given by
2.5 x1 2.5
2.5 x2 2.5.
(7.30)
A solution to the mp-LP problem is shown in Figure 7.2 and the constraints which
are active in each associated critical region are reported in Table 7.1. Clearly, as
z R4 , CR1 = CR{1,4,5,6,7,8} and CR2 = CR{2,3,5,6,7,8} are primal degenerate
full-dimensional critical regions.
Critical Region
CR1=CR{1,4,5,6,7,8}
CR2=CR{2,3,5,6,7,8}
CR3=CR{1,5,6,9}
CR4=CR{4,6,7,10}
CR5=CR{1,5,8,12}
CR6=CR{2,5,6,9}
CR7=CR{3,6,7,10}
CR8=CR{3,6,7,11}
CR9=CR{1,5,9,12}
CR10=CR{4,7,10,12}
CR11=CR{2,5,9,12}
CR12=CR{3,7,10,12}
Value function
x1 x2
x1 + x2
2x1 3x2 1
x1 + 3x2 2
1.5x1 1.5x2 + 0.5
x1 3x2 2
2x1 + 3x2 1
1.5x1 + 1.5x2 0.5
2x1 3x2 1
x1 + 3x2 2
x1 3x2 2
2x1 + 3x2 1
Table 7.1 Example 7.3: Critical regions and corresponding value function.
By observing Figure 7.2 and Table 7.1 we notice the following. Under no primal
and dual degeneracy, (i) full critical regions are described by a set of active constraints of dimension n, (ii) two neighboring full-dimensional critical regions CRAi
and CRAj have Ai and Aj differing only in one constraint, (iii) CRAi and CRAj
will share a facet which is a primal degenerate critical region CRAp of dimension
n 1 with Ap = Ai Aj . In Example 7.3, CR7 ad CR12 are two full-dimensional
and neighboring critical regions and the corresponding active set differs only in one
constraint (constraint 6 in CR7 and constraint 12 in CR12). They share a facet
which is a one dimensional critical region (an open line), CR13=CR{3,6,7,10,12}.
(The solution depicted in Figure 7.2 and detailed in Table 7.1 contains only fulldimensional critical regions.)
If primal and/or dual degeneracy occur, then the situation becomes more complex. In particular, in the case of primal degeneracy, it might happen that fulldimensional critical regions are described by more than s active constraints (CR1
or CR2 in Example 7.3). In case of dual degeneracy, it might happen that fulldimensional critical regions are described by fewer than s active constraints. Details
on this case are provided in the next section.
7.2
115
2.5
10
12
4
5
7
x2
6
11
-2.5
-2.5
2.5
x1
7.2.4
Nonunique Optimizer
If rank[GA ] < s then Z (x) is not a singleton and the projection of the set Pz2 x
in (7.24) is required in order to compute the critical region CRA(x ) (see Section 7.1.1 for a discussion of polyhedra projection).
If one needs to determine one possible optimizer z () in the dual degenerate
region CRA(x ) the following simple method can be used. Choose a particular
b ) A(x )
optimizer which lies on a vertex of the feasible set, i.e., determine set A(x
d b of
of active constraints for which rank(GA(x
b ) ) = s, and compute a subset CRA(x
)
the dual degenerate critical region (namely, the subset of parameters x such that
b ) are active at the optimizer, which is not a critical region in
the constraints A(x
d b , the piecewise linear expression of an
the sense of Definition 7.2). Within CR
A(x )
d b until
The algorithm proceeds by exploring the space surrounding CR
A(x )
CRA(x ) is covered. The arbitrariness in choosing an optimizer leads to different ways of partitioning CR{A(x )} , where the partitions, in general, may overlap.
Nevertheless, in each region a unique optimizer is defined. The storing of overlapping regions can be avoided by intersecting each new region (inside the dual
degenerate region) with the current partition computed so far. This procedure is
illustrated in the following example.
116
CR{1,5}
CR{1,3}
x1
Figure 7.3 Example 7.4: Polyhedral partition of the parameter space corresponding to the solution.
subj. to
2z1 z2
z1 + 3z2
2z1 + z2
z1
z1
z2
9 2x1 + x2
8 + x1 2x2
4 + x1 + x2
0
0,
10 x1 10
10 x2 10.
(7.31)
(7.32)
The solution is represented in Figure 7.3 and the critical regions are listed in Table 7.2.
Region
CR{2}
CR{1,5}
CR{1,3}
Optimizer
not single valued
z1 = 2x1 + x2 + 9, z2 = 0
z1 = x1 + x2 + 4, z2 = x1 + 5/3
Value Function
x1 + 2x2 8
4x1 2x2 18
x1 2x2 29/3
Table 7.2 Example 7.4: Critical regions and corresponding optimal value.
The critical region CR{2} is related to a dual degenerate solution with nonunique optimizers. The analytical expression of CR{2} is obtained by projecting
the H-polyhedron
z1 + 3z2 + 2x1 x2 < 9
2z1 + z2 x1 + 2x2 = 8
z1 x1 x2
< 4
(7.33)
z1
< 0
z2
< 0
7.2
CR{2} = [x1 , x2 ] :
2.5x1 2x2
0.5x1 + x2
12x2
x1 x2
117
5
4
5
4
(7.34)
which is effectively the result of (7.25). For all x CR{2} , only one constraint is
active at the optimum, which makes the optimizer not unique.
Figures 7.4 and 7.5 show two possible ways of covering CR{2} without using
projection. The generation of overlapping regions is avoided by intersecting each
new region with the current partition computed so far, as shown in Figure 7.6
g {2,4} and CR
g {2,1} represent the intersected critical regions. In Figure 7.4
where CR
the regions are overlapping, and in Figure 7.6 artificial cuts are introduced at
the boundaries inside the degenerate critical region CR{2} . No artificial cuts are
g {2,3} and CR
g {2,5} happen to be nonintroduced in Figure 7.5 because the CR
overlapping.
7.2.5
solution z (
x). Then, the dual problem (7.14) for x = x
is infeasible. This implies
that the dual problem will be infeasible for all real vectors x since x enters only in
the cost function. This contradicts the hypothesis since the dual problem (7.14)
for x = x0 has a finite optimal solution.
Theorem 7.4 The set of all parameters x such that the LP (7.11) has a finite
optimal solution z (x) equals K .
Proof: It follows directly from from Theorem 7.1 and Theorem 7.3.
Note that from Definition 7.3 K is the set of feasible parameters. However the
LP (7.11) might be unbounded from some x K . Theorem 7.4 excludes this case.
The following Theorem 7.5 summarizes the properties enjoyed by the multiparametric solution.
Theorem 7.5 The function J () is convex and piecewise affine over K . If the
optimizer z (x) is unique for all x K , then the optimizer function z : K Rs
is continuous and piecewise affine. Otherwise it is always possible to define a
continuous and piecewise affine optimizer function z such that z (x) Z (x) for
all x K .
118
x2
z2
d {2,5}
CR
x
1
c x = const
x1
1
4
z1
d {2,5} CR{2} , and below
(a) First region CR
the feasible set in the z-space corresponding
d {2,5} .
to x
1 CR
x2
d {2,4}
CR
x
2
z2
c x = const
x1
2
4
3
5
z1
7.2
119
x2
x
3
z2
c x = const
d {2,1}
CR
x1
4
5
3
z1
x2
d {2,4}
CR
d {2,5}
CR
d {2,1}
CR
x1
Remark 7.2 In Theorem 7.5, the piecewise affine property of optimizer and value
function follows immediately from Theorem 7.2 and from the enumeration of all
possible combinations of active constraint sets. Convexity of J () and continuity of
Z (x) follows from standard results on multiparametric programs (see Theorem 6.2
and Corollary 6.1). In the presence of multiple optimizers, the proof of existence of
120
x2
d {2,5}
CR
x1
d {2,5} CR{2} .
(a) First region CR
x2
d {2,3}
CR
x1
d {2,3} CR{2} .
(b) Second region CR
d {2,5}
Figure 7.5 A possible solution to Example 7.4 where the regions CR
d
and CR{2,3} are non-overlapping
a continuous and piecewise affine optimizer function z such that z (x) Z (x) for
all z K is more involved and we refer the reader to [108, p. 180].
7.2.6
mp-LP Algorithm
The goal of an mp-LP algorithm is to determine the partition of K into critical
regions CRAi , and to find the expression of the functions J () and z () for each
critical region. An mp-LP algorithm has two components: the active set generator and the KKT solver. The active set generator computes the set of active
constraints Ai . The KKT solver computes CRAi and the expression of J () and
7.2
x2
121
d {2,5}
CR
g {2,4}
CR
g {2,1}
CR
x1
g {2,4} is obtained
Figure 7.6 A possible solution for Example 7.4 where CR
d
d
g {2,1} by
by intersecting CR{2,4} with the complement of CR{2,5} , and CR
d
d
d
intersecting CR{2,1} with the complement of CR{2,5} and CR{2,4}
Ti x + kTi k2 Ni , i = 1, . . . , nT
Gz Sx w,
(7.35)
where nT is the number of rows Ti of the matrix T defining the set K in (7.2).
If 0, then the LP problem (7.11) is infeasible for all x in the interior of K.
Otherwise, we solve the primal and dual problems (7.11), (7.14) for x = x0 . Let
z0 and u0 be the optimizers of the primal and the dual problem, respectively. The
value z0 defines the following optimal partition
A(x0 ) , {j I : Gj z0 Sj x0 wj = 0}
NA(x0 ) , {j I : Gj z0 Sj x0 wj < 0}
(7.36)
and consequently the critical region CRA(x0 ) . Once the critical region CRA(x0 )
has been defined, the rest of the space Rrest = K\CRA(x0 ) has to be explored and
new critical regions generated. An approach for generating a polyhedral partition {R1 , . . . , Rnr est } of the rest of the space Rrest is described in Theorem 5.2 in
122
Section 5.4.7. The procedure proposed in Theorem 5.2 for partitioning the set of
parameters allows one to recursively explore the parameter space. Such an iterative
procedure terminates after a finite time, as the number of possible combinations of
active constraints decreases with each iteration. The following two issues need to
be considered:
1. The partitioning in Theorem 5.2 defines new polyhedral regions Rk to be
explored that are not related to the critical regions which still need to be
determined. This may split some of the critical regions, due to the artificial
cuts induced by Theorem 5.2. Post-processing can be used to join cut critical
regions [39]. As an example, in Figure 7.7 the critical region CR{3,7} is
discovered twice, one part during the exploration of R1 and the second part
during the exploration of R2 .
Although algorithms exist for convexity recognition and computation of the
union of polyhedra, the post-processing operation is computationally expensive. Therefore, it is more efficient not to intersect the critical region obtained
by (7.28) with halfspaces generated by Theorem 5.2, which is only used to
drive the exploration of the parameter space. Then, no post processing is
needed to join subpartitioned critical regions.
On the other hand, some critical regions may appear more than once. Duplicates can be uniquely identified by the set of active constraints A(x) and can
be easily eliminated. To this aim, in the implementation of the algorithm we
keep a list of all the critical regions which have already been generated in order to avoid duplicates. In Figure 7.7 the critical region CR{3,7} is discovered
twice but stored only once.
x2
CR{3,7}
x+
2
R2
CR{6,7}
R1
x
2
x
1
x+
1
x1
7.2
Based on the above discussion, the mp-LP solver can be summarized in the
following recursive Algorithm 7.1. Note that the algorithm generates a partition
of the state space which is not strict. The algorithm could be modified to store
the critical regions as defined in (7.4) which are open sets, instead of storing their
closure. In this case the algorithm would have to explore and store all the critical regions that are not full-dimensional in order to generate a strict polyhedral
partition of the set of feasible parameters. From a practical point of view such a
procedure is not necessary since the value function and the optimizer are continuous
functions of x.
Algorithm 7.1 mp-LP Algorithm
input: Matrices c, G, w, S of the mp-LP (7.11) and set K in (7.2)
output: Multiparametric solution to the mp-LP (7.11)
execute partition(K);
end.
procedure partition(Y )
let x0 Y and be the solution to the LP (7.35);
if 0 then exit; (no full-dimensional CR is in Y )
solve the LP (7.11), (7.14) for x = x0 ;
if the optimizer is not unique, then choose z0 with rank(A(x0 )) = s, where
A(x0 ) is the set of active constraints defined in (7.36)
let A(x0 ) be the set of active constraints as in (7.36);
let U1 , U2 , P, D matrices as in (7.18).
determine z (x) from (7.26) and CRA(x0 ) from (7.28);
choose z0 as one of the possible optimizers;
let J (x) as in (7.27) for x = x0 ;
define and partition the rest of the region as in Theorem 5.2;
for each new sub-region Ri , partition(Ri );
end procedure
Remark 7.3 As remarked in Section 7.2.2, if rank(D) > 0 in Algorithm 7.1, the
region CRA(x0 ) is not full-dimensional. To avoid further recursion in the algorithm
which does not produce any full-dimensional critical region, after computing U, P, D
if D 6= 0 one should compute a random vector Rn smaller than the Chebychev
radius of Y and such that the LP (7.11) is feasible for x0 + and then repeat step
where A(x0 ) is computed with x0 x0 + .
Remark 7.4 Note that in Algorithm 7.1 in order to choose z0 with rank(A(x0 )) = s
one can use an active set method for solving the LP (7.11) for x = x0 .
Remark 7.5 The algorithm determines the partition of K recursively. After the first
critical region is found, the rest of the region in K is partitioned into polyhedral sets
123
124
{Ri } as in Theorem 5.2. By using the same method, each set Ri is further partitioned,
and so on.
7.3
7.3
7.3.1
125
subj. to
J(z, x) = 12 z 2
z 1 + 3x,
z1+x
J(z, x),
1 2
z + u(z 3x 1)
2
and the KKT conditions are (see Section 3.3.3 for KKT conditions for quadratic
programs)
z+u
=0
(7.38a)
u(z 3x 1)
=0
(7.38b)
z 3x 1
(7.38c)
(7.38d)
126
z+u=0
u = 3x 1
A. z 3x 1 = 0
J
= 29 x2 + 3x +
u>0
x < 31
1
2
z =0
z+u=0
u =0
B. z 3x 1 0
J = 0
u=0
x 31
(7.39)
This solution is depicted in Figure 7.8. The above simple procedure, which required
nothing but the solution of the KKT conditions, yielded the optimizer z (x) and
the value function J (x) for all values of the parameter x. The set of admissible
parameters values was divided into two critical regions, defined by x < 31 and
x 31 . In the region x < 31 the inequality constraint is active and the Lagrange
multiplier is greater than zero, in the other region x 31 the Lagrange multiplier
is equal to zero and the inequality constraint is not active (with the exception of the
boundary point 31 ). Note that for x = 13 the inequality constraint z 3x 1 0
is active at z and the Lagrange multiplier is equal to zero.
Throughout a critical region the conditions for optimality derived from the KKT
conditions do not change. In each critical region the optimizer z (x) is affine and the
value function J (x) is quadratic. Both z (x) and J (x) are continuous.
J (x)
1
0
-1
-2
-2
z (x)
-1
0
x
Figure 7.8 Example 7.5: Optimizer z (x) and value function J (x) as a
function of the parameter x.
7.3.2
7.3
127
(7.40a)
(7.40b)
(7.40c)
(7.40d)
(7.41)
and substitute the result into (7.40b) to obtain the complementary slackness conditions
ui (Gi H 1 G u wi Si x) = 0, i = 1, . . . , m.
(7.42)
(7.44)
(7.45)
(7.46)
and note that z is also an affine function of x. J (x) = 12 z (x) Hz (x) and
therefore is a quadratic function of x. The critical region CRA is computed by
substituting z from (7.46) in the primal feasibility conditions (7.40d)
Pp , {x : GH 1 GA (GA H 1 GA )1 (wA + SA x) < w + Sx},
(7.47)
128
and the Lagrange multipliers from (7.45) in the dual feasibility conditions (7.40c)
Pd , {x : (GA H 1 GA )1 (wA + SA x) 0}.
(7.48)
(7.49)
u
U1
U2
P A,1
q
uA,2 =
.
(7.50)
0|A|ll 0|A|l|A|l D
r
x
We compute uA,1 from (7.50)
Finally, we can substitute uA = uA,1 uA,2 from (7.51) into (7.41) to obtain
1
U1 (U2 uA,2 + P x q)
z = H 1 GA
.
(7.52)
uA,2
The optimizer z is unique and therefore independent of the choice of uA,2 0.
Setting uA,2 = 0 we obtain
z = H 1 GA,1 U11 (P x q),
(7.53)
7.3
129
where we have partitioned GA as GA = GA,1 GA,2 and GA,1 Rls .
Note that z is an affine function of x. J (x) = 12 z (x) Hz (x) and therefore
is a quadratic function of x.
The critical region CRA is computed by substituting z from (7.53) in the
primal feasibility conditions (7.40d)
Pp , {x : G(H 1 GA,1 U11 (P x q)) < w + Sx},
(7.54)
and the Lagrange multipliers from (7.51) in the dual feasibility conditions (7.40c)
PuA,2 ,x , {[ uA,2 , x] : U11 (U2 uA,2 P x + q) 0, uA,2 0}.
(7.55)
The critical region CRA is the intersection of the sets Dx = r, Pp and projx (PuA,2 ,x ):
CRA = {x : Dx = r, x Pp , x projx (PuA,2 ,x )}.
(7.56)
Remark 7.6 In general, the critical region polyhedron (7.47)-(7.49) is open on facets
arising from primal feasibility and closed on facets arising from dual feasibility.
Remark 7.7 If D in (7.50) is nonzero, then from (7.56), CRA is a lower dimensional
region, which, in general, corresponds to a common boundary between two or more
full-dimensional regions. Typically, the following definition is introduced
7.3.3
min
1
z Hz
2
subj. to
Gz w + Sx,
with
H = [ 10 01 ] , F =
and
G=
where K is given by
1
1
00
1
1
0
0
1
,
1
1
1
E=
0
0
00
1
2
1
+ x F z
1
1 1
0
0
0
,
0
1
1
(7.57)
(7.58)
1
1
w = 11 ,
(7.59)
0
0
1 x1 1
(7.60)
1 x2 1.
A solution to the mp-QP problem is shown in Figure 7.9 and the constraints which
are active in each associated full-dimensional critical region are reported in Table 7.3.
130
Active Constraints
{5}
{3,5}
{}
{3}
{6}
{3,6}
{4,6}
Table 7.3 Example 7.6: Critical regions and corresponding set of active
constraints.
6
4
5
x2
3
1
2
-1
1
-1
x1
Since A3 is empty in CR3 we can conclude from (7.44) that the facets of CR3
are facets of primal feasibility and therefore do not belong to CR3. In general,
as discussed in Remark 7.6 critical regions are open on facets arising from primal
feasibility and closed on facets arising from dual feasibility. Next we focus on the
closure of the critical regions.
By observing Figure 7.9 and Table 7.3 we notice that as we move away from
region CR3 (corresponding to no active constraints), the number of active constraints increases. In particular, for any two neighboring full-dimensional critical
regions CRAi and CRAj we have Ai Aj and |Ai | = |Aj | 1 or Aj Ai and
|Ai | = |Aj | + 1. This means that as one moves from one full-dimensional region to
7.3
7.3.4
J (x )
1
z Hz .
2
Hence
1
1
1
J (x ) [z1 Hz1 + (1 )z2 Hz2 ] z Hz [z1 Hz1 + (1 )z2 Hz2 ].
2
2
2
We can upperbound the latter term as follows:
1
2 z Hz
131
132
Since H 0, the optimum is unique, and hence the solution must be continuous
across the boundary. Therefore z (x) : K Rs is continuous and piecewise affine
on polyhedra. The fact that J (x) is continuous and piecewise quadratic follows
trivially.
We can easily derive a result equivalent to Theorem 7.7 for the dual function
(x) (see (7.37) and (3.28))
1
1
(7.61)
Theorem 7.9 Assume that the mp-QP problem (7.37) is not degenerate, then the
value function J (x) in (7.37) is C (1) .
Proof: The dual of (7.37) is
1
(x) = max u (GH 1 G )u u (w + Sx).
u0
2
(7.62)
(7.63)
For the moment let us restrict our attention to the interior of a critical region.
From (7.45) we know that u (x) is continuously differentiable. Therefore
u
u
(x) GH 1 G u (x)
(x) (W + Sx) S u (x) =
x
x
u
(x) (GH 1 G u (x) W Sx) S u (x).
=
x
(7.64)
Using (7.41)
x J(x)
z = H 1 G u
(7.65)
7.3
133
we get
x J(x)
=
=
u
(x) (Gz W Sx) =
x
P
ui
(x) (Gi z Wi Si x) +
S u (x) +
|
{z
}
iA x
0
P ui
(x) (Gi z Wi Si x) =
+
iN A x
| {z }
S u (x) +
S u (x).
(7.66)
3
3 1
11 23
H= 3
1 23 75
1
1
1
1
3
5
1 1 1
G=
1 3 5
1
0
0
0
1
0
0
0
1
w=
1
0
1
0
0
0
0
S=
0
1
0
1
0
0
0
(7.67)
The problem was solved by using Algorithm 7.2 described next. The solution comprises 5 critical regions. The critical regions and the expression of the value function
are reported in Table 7.4. The reader can verify that the value function is not continuously differentiable at x = 3. Indeed, at x = 3 the LICQ condition does not hold
and therefore, the hypothesis of Theorem 7.9 is not fulfilled.
7.3.5
mp-QP Algorithm
The goal of an mp-QP algorithm is to determine the partition of K into critical
regions CRi , and find the expression of the functions J () and z () for each critical
region. An mp-QP algorithm has two components: the active set generator and
the KKT solver. The active set generator computes the set of active constraints
Ai . The KKT solver computes CRAi and the expression of J () and z () in
CRAi as explained in Theorem 7.6. The active set generator is the critical part.
134
Region
CR{1,2,3,4,6} = {x : 1 x < 1.5}
CR{1,2,3,4} = {x : 1.5 x 2}
CR{1,2,3,4,7} = {x : 2 < x < 3}
CR{1,2,3,4,5,7} = {x : x = 3}
CR{1,2,3,4,5} = {x : 3 < x 5}
Optimal value
2.5x2 6x + 5
0.5x2 + 0.5
x2 2x + 2.5
x2 2x + 2.5
5x2 24x + 32.5
Table 7.4 Critical regions and value function corresponding to the solution
of Example 7.7
In principle, one could simply generate all the possible combinations of active
sets. However, in many problems only a few active constraints sets generate fulldimensional critical regions inside the region of interest K. Therefore, the goal is
to design an active set generator algorithm which computes only the active sets Ai
with the associated full-dimensional critical regions covering only K .
Next an implementation of an mp-QP algorithm is described. See Section 7.6
for a literature review on alternative approaches to the solution of an mp-QP.
In order to start solving the mp-QP problem, we need an initial vector x0
inside the polyhedral set K of feasible parameters. A possible choice for x0 is the
Chebychev center (see Section 5.4.5) of K obtained as a solution of
maxx,z,
subj. to
Ti x + kTi k2 Ni , i = 1, . . . , nT
G
z Sx w
(7.68)
where nT is the number of rows Ti of the matrix T defining the set K in (7.2).
If 0, then the QP problem (7.37) is infeasible for all x in the interior of K.
Otherwise, we set x = x0 and solve the QP problem (7.37), in order to obtain the
corresponding optimal solution z0 . Such a solution is unique, because H 0. The
value of z0 defines the following optimal partition
A(x0 ) , {j J : Gj z0 Sj x0 wj = 0}
NA(x0 ) , {j J : Gj z0 Sj x0 wj < 0}
(7.69)
and consequently the critical region CRA(x0 ) . Once the critical region CRA(x0 )
has been defined, the rest of the space Rrest = K\CRA(x0 ) has to be explored and
new critical regions generated. An approach for generating a polyhedral partition
{R1 , . . . , Rnrest } of the rest of the space Rrest is described in Theorem 5.2. Theorem 5.2 provides a way of partitioning the non-convex set K \ CR0 into polyhedral
subsets Ri . For each Ri , a new vector xi is determined by solving the LP (7.68),
and, correspondingly, an optimum zi , a set of active constraints Ai , and a critical
region CRi . The procedure proposed in Theorem 5.2 for partitioning the set of
parameters allows one to recursively explore the parameter space. Such an iterative
procedure terminates after a finite time, as the number of possible combinations
of active constraints decreases with each iteration. Two main elements need to be
considered:
7.3
1. As for the mp-LP algorithm, the partitioning in Theorem 5.2 defines new
polyhedral regions Rk to be explored that are not related to the critical
regions which still need to be determined. This may split some of the critical
regions, due to the artificial cuts induced by Theorem 5.2. Post-processing
can be used to join cut critical regions [39]. As an example, in Figure 7.7 the
critical region CR{3,7} is discovered twice, one part during the exploration of
R1 and the second part during the exploration of R2 .
Although algorithms exist for convexity recognition and computation of the
union of polyhedra, the post-processing operation is computationally expensive. Therefore, it is more efficient not to intersect the critical region obtained
by (7.28) with halfspaces generated by Theorem 5.2, which is only used to
drive the exploration of the parameter space. Then, no post processing is
needed to join subpartitioned critical regions. On the other hand, some critical regions may appear more than once. Duplicates can be uniquely identified
by the set of active constraints A(x) and can be easily eliminated. To this
aim, in the implementation of the algorithm we keep a list of all the critical
regions which have already been generated in order to avoid duplicates. In
Figure 7.7 the critical region CR{3,7} is discovered twice but stored only once.
2. If Case 2 occurs in Section 7.3.2 and D is nonzero, CRA is a lower dimensional critical region (see Remark 7.7). Therefore we do not need to explore
the actual combination GA , SA , wA . On the other hand, if D = 0 the
KKT conditions do not lead directly to (7.47)(7.48). In this case, a fulldimensional critical region can be obtained from (7.56) by projecting the set
Px,uA,2 in (7.55).
Based on the above discussion and results, the main steps of the mp-QP solver
are outlined in the following algorithm.
Algorithm 7.2 mp-QP Algorithm
input: Matrices H, G, w, S of Problem (7.37) and set K in (7.2)
output: Multiparametric solution to Problem (7.37)
execute partition(K);
end.
procedure partition(Y )
let x0 Y and be the solution to the LP (7.68);
if 0 then exit; (no full-dimensional CR is in Y )
solve the QP (7.37) for x = x0 to obtain (z0 , u0 );
Determine the set of active constraints A when z = z0 , x = x0 , and build GA ,
wA , S A ;
if r = rank GA is equal to the number of rows of GA
Determine uA (x), z (x) from (7.45) and (7.46);
Characterize the CR from (7.47) and (7.48);
elseif r = rank GA is less than the number of rows of GA
Determine z (x) from (7.53);
Characterize the CR from (7.56);
end
135
136
Remark 7.8 If rank(D) > 0 in Algorithm 7.2, the region CRA(x0 ) is not fulldimensional. To avoid further recursion in the algorithm which does not produce
any full-dimensional critical region, one should compute a random vector Rn
smaller than the Chebychev radius of Y and such that the QP (7.37) is feasible for
x0 + and then repeat step where A(x0 ) is computed with x0 x0 + .
Remark 7.9 The algorithm solves the mp-QP problem by partitioning the given
parameter set K into Nr closed polyhedral regions. Note that the algorithm generates
a partition of the state space which is not strict. The algorithm could be modified
to store the critical regions as defined in Section 7.1.2. (which are neither closed nor
open as proven in Theorem 7.6) instead of storing their closure. This can be done by
keeping track of which facet belongs to a certain critical region and which not. From
a practical point of view, such a procedure is not necessary since the value function
and the optimizer are continuous functions of x.
Remark 7.10 The proposed algorithm does not apply to the case when H 0 and
when the optimizer may not be unique. In this case one may resort to regularization, i.e. adding an appropriate small quadratic term, or follow the ideas of [222].
7.4
7.4
7.4.1
137
J (x) = min
z
subj. to
{J(z, x) = c z}
Gz w + Sx,
(7.70)
7.4.2
138
c z
subj. to
Gz Sx w
xK
{z,x}
(7.72)
min
z
subj. to
c z
Gz w + Sx
zd = zdNi bi +1
x CRi
(7.73)
By Theorem 7.5, the solution of mp-LP (7.73) provides a partition of CRi into
polyhedral regions Rik , k = 1, . . . , NRi and a PWA value function
k (x) , ck x + pk ) if x Rk , k = 1, . . . , NRi
Ji (x) = (JR
i
i
i
i
(7.74)
7.4
139
{z,x}
subj. to
c z
(7.75)
Gz Sx w
c z Ji (z)
(7.76)
x CRi
(7.79)
zd 6=
zdki ,
k = 1, . . . , N bi
(7.77)
(7.78)
where constraints (7.78) prohibit integer solutions that have been already analyzed
in CRi from appearing again and constraint (7.77) excludes integer solutions with
higher values than the current upper bound. If problem (7.79) is infeasible then
the region CRi is excluded from further recursion and the current upper bound
represents the final solution. If problem (7.79) is feasible, then the discrete optimal
component zdi is stored and represents a feasible integer variable that is optimal
at least in one point of CRi .
Recursion
At step j we have stored
1. A list of Nj polyhedral regions CRi and for each of them an associated
parametric affine upper bound Ji (x) (Ji (x) = + if no integer solution has
been found yet in CRi ).
2. For each CRi a set of integer variables Zi = zd0i , . . . , zdNi bi , that have already
been explored in the region CRi .
/ Zi such that there exists zc
3. For each CRi an integer feasible variable zdNi bi +1
and x
CRi for which Gz w + S x
and c z < Ji (
x) where z = {zc , zdNi bi +1 }.
That is, zdNi bi +1 is an integer variable that improves the current bound for at
least one point of the current polyhedron.
For all the regions CRi not excluded from the MILPs subproblem (7.75)-(7.79)
the algorithm continues to iterate between the mp-LP (7.73) with zdNi bi +1 = zdi
140
and the MILP (7.75)-(7.79). The algorithm terminates when all the MILPs (7.75)(7.79) are infeasible.
Note that the algorithm generates a partition of the state space. Some parameter x could belong to the boundary of several regions. Differently from the LP
and QP case, the value function may be discontinuous and therefore such a case
has to be treated carefully. If a point x belongs to different critical regions, the
expressions of the value function associated with such regions have to be compared
in order to assign to x the right optimizer. Such a procedure can be avoided by
keeping track of which facet belongs to a certain critical region and which not.
Moreover, if the value functions associated with the regions containing the same
parameter x coincide this may imply the presence of multiple optimizers.
7.4.3
Solution Properties
The following properties of J (x) and Z (x) easily follow from the algorithm described above.
Theorem 7.10 Consider the mp-MILP (7.70). The set K is the union of a finite
number of (possibly open) polyhedra and the value function J is piecewise affine
on polyhedra. If the optimizer z (x) is unique for all x K , then the optimizer
functions zc : K Rsc and zd : K {0, 1}sd are piecewise affine and piecewise
constant, respectively, on polyhedra. Otherwise, it is always possible to define a
piecewise affine optimizer function z (x) Z (x) for all x K .
Note that, differently from the mp-LP case, the set K can be non-convex and
even disconnected.
7.5
7.5
7.5.1
141
min
J(z, x) = z H1 z + c1 z
subj. to
Gz w + Sx,
(7.80)
minz1 ,z2 ,
subj. to
1
0
1
0
10
1
0
1
0
10
0
1
0
1
10
0 1
0 1
10
0
0
1
0 10
z1
0
0
1
0 10
z2
0
0
0
1 10
0
0
0 1 10
1
0
0
0
0
1
0
0
0
0
0
1
0
0
0
0 1
0
0
0
x1
x2 +
(7.82)
can be simply solved by noting that for = 1 z1 = x1 and z2 = x2 while for = 0
z1 = z2 = 0. By comparing the value functions associated with = 0 and = 1
we obtain two critical regions
CR1 = {x1 , x2 R : x21 + x22 25}
CR2 = {x1 , x2 R : 10 x1 10, 10 x2 10, x21 + x22 > 25}
(7.83)
10
10
10
10
0
0
0
0
10
10
10
10
142
x1
0
x2
z2 (x1 , x2 ) =
0
z1 (x1 , x2 ) =
if
if
if
if
[x1 , x2 ] CR1
[x1 , x2 ] CR2
[x1 , x2 ] CR1
[x1 , x2 ] CR2
J (x1 , x2 ) =
100
(7.84)
if [x1 , x2 ] CR1
if [x1 , x2 ] CR2
(7.85)
x2
CR1
J (x)
The two critical regions and the value function are depicted in Figure 7.10.
CR2
x1
(a) Critical Regions.
x2
x1
7.6
Literature Review
Many of the theoretical results on parametric programming can be found in[16,
101, 44, 109].
The first method for solving parametric linear programs was proposed by Gass
and Saaty [113], and since then extensive research has been devoted to sensitivity
and (multi)-parametric linear analysis, as attested by the hundreds of references
in [108] (see also [109] for recent advances in the field). One of the first methods
for solving multiparametric linear programs (mp-LPs) was formulated by Gal and
Nedoma [110]. The method constructs the critical regions iteratively, by visiting
the graph of bases associated with the LP tableau of the original problem. Many
of the results on mp-LP presented in this book can be found in [108, p. 178-180].
7.6
Literature Review
Note that in [110, 108] a critical region is defined as a subset of the parameter space on which a certain basis of the linear program is optimal. The algorithm proposed in [110] for solving multiparametric linear programs generates
non-overlapping critical regions by generating and exploring the graph of bases. In
the graph of bases the nodes represent optimal bases of the given multiparametric
problem and two nodes are connected by an edge if it is possible to pass from one
basis to another by one pivot step (in this case the bases are called neighbors).
In this book we use the definition (7.4) of critical regions which is not associated
with the bases but with the set of active constraints and it is directly related to
the definition given in [3, 200, 109].
The solution to multiparametric quadratic programs has been studied in detail
in [16, Chapter 5]. In [39] Bemporad and coauthors presented a simple method
for solving mp-QPs. The method constructs a critical region in a neighborhood
of a given parameter, by using the Karush-Kuhn-Tucker conditions for optimality,
and then recursively explores the parameter space outside such a region. Other
algorithms for solving mp-QPs have been proposed by Seron, DeDon
a and Goodwin in [253, 93] in parallel with the study of Bemporad and coauthors in [39], by
Tondel, Johansen and Bemporad in [265] and by Baotic in [17]. All these algorithms are based on an iterative procedure that builds up the parametric solution
by generating new polyhedral regions of the parameter space at each step. The
methods differ in the way they explore the parameter space, that is, the way they
identify active constraints corresponding to the critical regions neighboring to a
given critical region, i.e., in the active set generator component.
In [253, 93] the authors construct the unconstrained critical region and then
generate neighboring critical regions by enumerating all possible combinations of
active constraints.
In [265] the authors explore the parameter space outside a given region CRi
by examining its set of active constraints Ai . The critical regions neighboring to
CRi are constructed by elementary operations on the active constraints set Ai
that can be seen as an equivalent pivot for the quadratic program. For this
reason the method can be considered as an extension of the method of Gal [108]
to multiparametric quadratic programming.
In [17] the author uses a direct exploration of the parameter space as in [39]
but he avoids the partition of the state space described in Theorem 5.2. Given a
polyhedral critical region CRi , the procedure goes through all its facets and generates the Chebychev center of each facet. For each facet Fi a new parameter xi
is generated, by moving from the center of the facet in the direction of the normal
to the facet by a small step. If such parameter xi is infeasible or is contained in
a critical region already stored, then the exploration in the direction of Fi stops.
Otherwise, the set of active constraints corresponding to the critical region sharing
the facet Fi with the region CRi is found by solving a QP for the new parameter
xi .
In [2, 95] two approaches were proposed for solving mp-MILP problems. In both
methods the authors use an mp-LP algorithm and a branch and bound strategy
that avoids the complete enumeration of combinations of 0-1 integer variables by
143
144
Part III
Optimal Control
Chapter 8
8.1
Problem Formulation
We consider the nonlinear time-invariant system
x(t + 1) = g(x(t), u(t)),
(8.1)
(8.2)
m
at all time instants t 0. In (8.1)(8.2), x(t) R and u(t) R are the state and
input vector, respectively. Inequality (8.2) with h : Rn Rm Rnc expresses the
nc constraints imposed on the input and the states. These may be simple upper
and lower bounds or more complicated expressions. We assume that the origin is
an equilibrium point (g(0, 0) = 0) in the interior of the feasible set, i.e., h(0, 0) < 0.
We assumed the system to be specified in discrete time. One reason is that we
are looking for solutions to engineering problems. In practice, the controller will
almost always be implemented through a digital computer by sampling the variables
of the system and transmitting the control action to the system at discrete time
points. Another reason is that for the solution of the optimal control problems for
discrete-time systems we will be able to make ready use of powerful mathematical
programming software.
We want to caution the reader, however, that in many instances the discrete
time model is an approximation of the continuous time model. It is generally
148
difficult to derive good discrete time models from nonlinear continuous time
models, and especially so when the nonlinear system has discontinuities as would
be the case for switched systems. We also note that continuous time switched
systems can exhibit behavioral characteristics not found in discrete-time systems,
for example, an ever increasing number of switches in an ever decreasing time
interval (Zeno behavior [121]).
We define the following performance objective or cost function from time instant
0 to time instant N
J0N (x0 , U0N ) , p(xN ) +
N
1
X
q(xk , uk ),
(8.3)
k=0
where N is the time horizon and xk denotes the state vector at time k obtained by
starting from the measured state x0 = x(0) and applying to the system model
xk+1 = g(xk , uk ),
(8.4)
the input sequence u0 , . . . , uk1 . From this sequence we define the vector of future
inputs U0N , [u0 , . . . , uN 1 ] Rs , s , mN . The terms q(xk , uk ) and p(xN )
are referred to as stage cost and terminal cost, respectively, and are assumed to be
positive definite (q 0, p 0):
p(x, u) > 0 x 6= 0, u 6= 0, p(0, 0) = 0
q(x, u) > 0 x =
6 0, u =
6 0, q(0, 0) = 0.
The form of the cost function (8.3) is very general. If a practical control objective
can be expressed as a scalar function then this function usually takes the indicated
form. Specifically, we consider the following constrained finite time optimal control
(CFTOC) problem.
J0N
(x0 ) =
minU0N
subj. to
(8.5)
Here Xf Rn is a terminal region that we want the system states to reach at the
end of the horizon. The terminal region could be the origin, for example. We define
X0N Rn to be the set of initial conditions x(0) for which there exists an input
vector U0N so that the inputs u0 , . . . , uN 1 and the states x0 , . . . , xN satisfy the
model xk+1 = g(xk , uk ) and the constraints h(xk , uk ) 0 and that the state xN
lies in the terminal set Xf .
We can determine this set of feasible initial conditions in a recursive manner.
Let us denote with XjN the set of states xj at time j which can be steered into
Xf at time N , i.e., for which the model xk+1 = g(xk , uk ) and the constraints
h(xk , uk ) 0 are feasible for k = j, . . . , N 1 and xN Xf . This set can be
defined recursively by
XjN
XN N
Xf .
(8.7)
8.2
149
The set X0N is the final result of these iterations starting with Xf .
and q are continuous. Also, there might be several input vectors U0N
which yield
the minimum (J0N (x0 ) = J0N (x0 , U0N )). In this case we will define one of
8.2
(8.8)
xN = g(xN 1 , uN 1 ),
then the optimal control problem (8.3)(8.5), rewritten below
J0N
(x0 ) = minU0N
subj. to
PN 1
p(xN ) + k=0 q(xk , uk )
x1 = g(x0 , u0 )
x2 = g(x1 , u1 )
..
.
xN = g(xN 1 , uN 1 )
h(xk , uk ) 0, k = 0, . . . , N 1
xN Xf
x0 = x(0)
(8.9)
150
is recognized more easily as a general nonlinear programming problem with variables u0 , . . . , uN 1 and x1 , . . . , xN .
As an alternative we may try to eliminate the state variables and equality
constraints (8.8) by successive substitution so that we are left with u0 , . . . , uN 1
as the only decision variables. For example, we can express x2 as a function of
x(0), u0 and u1 only, by eliminating the intermediate state x1
x2 = g(x1 , u1 )
x2 = g(g(x(0), u0 ), u1 ).
(8.10)
Except when the state equations are linear this successive substitution may become
complex. Even when they are linear it may be bad from a numerical point of view.
Either with or without successive substitution the solution of the nonlinear pro
8.3
N
1
X
q(xk , uk ),
(8.11)
k=j
JjN
(xj ) ,
(8.12)
8.3
151
Jj1N
(xj1 ) =
min
uj1
subj. to
xj = g(xj1 , uj1 )
h(xj1 , uj1 ) 0
xj XjN .
(8.13)
Here the only decision variable left for the optimization is uj1 , the input at time
j 1. All the other inputs uj , . . . , uN 1 have already been selected optimally to
Jj1N
(xj1 ) =
min
uj1
subj. to
h(xj1 , uj1 ) 0
g(xj1 , uj1 ) XjN ,
(8.14)
JN
N (xN ) = p(xN )
XN N = Xf ,
(8.15)
(8.16)
JN
1N (xN 1 )
min
uN 1
subj. to
..
.
J0N
(x0 )
min
u0
subj. to
q(xN 1 , uN 1 ) + JN
N (g(xN 1 , uN 1 ))
h(xN 1 , uN 1 ) 0,
g(xN 1 , uN 1 ) XN N
q(x0 , u0 ) + J1N
(g(x0 , u0 ))
h(x0 , u0 ) 0,
g(x0 , u0 ) X1N
x0 = x(0).
(8.17)
This algorithm, popularized by Bellman, is referred to as dynamic programming.
The dynamic programming problem is appealing because it can be stated compactly
and because at each step the optimization takes place over one element uj of the
optimization vector only. This optimization is rather complex, however. It is not a
standard nonlinear programming problem, since we have to construct the optimal
cost-to-go JjN
(xj ), a function defined over the subset XjN of the state space.
In a few special cases we know the type of function and we can find it efficiently.
For example, in the next chapter we will cover the case when the system is linear
and the cost is quadratic. Then the optimal cost-to-go is also quadratic and can be
constructed rather easily. Later in the book we will show that, when constraints
are added to this problem, the optimal cost-to-go becomes piecewise quadratic and
efficient algorithms for its construction are also available.
152
. With JjN
known, for a fixed xj1 the optimization
an approximation of Jj1N
problem (8.14) becomes a standard nonlinear programming problem. Thus, we
can define a grid in the set Xj1N of the state space and compute the optimal
cost-to-go function on each grid point. We can then define an approximate value
function Jj1N
(xj1 ) at intermediate points via interpolation. The complexity
of constructing the cost-to-go function in this manner increases rapidly with the
dimension of the state space (curse of dimensionality).
The extra benefit of solving the optimal control problem via dynamic pro
gramming is that we do not only obtain the vector of optimal inputs U0N
for a
particular initial state x(0) as with the batch approach. At each time j the optimal
cost-to-go function defines implicitly a nonlinear feedback control law.
q(xj , uj ) + Jj+1N
(g(xj , uj ))
h(xj , uj ) 0,
g(xj , uj ) Xj+1N .
subj. to
(8.18)
For a fixed xj this nonlinear programming problem can be solved quite easily in
8.4
J0
(x0 ) = minu0 ,u1 ,...
q(xk , uk )
k=0
subj. to
xk+1 = g(xk , uk ), k = 0, . . . ,
h(xk , uk ) 0, k = 0, . . . ,
x0 = x(0).
(8.19)
We define the set of initial conditions for which this problem has a solution.
lim q(xk , uk ) = 0,
8.4
153
and
lim uk = 0.
Thus the sequence of control actions generated by the solution of the infinite horizon
problem drives the system to the origin. For this solution to exists the system must
be, loosely speaking, stabilizable.
Using the recursive dynamic programming approach we can seek the solution
of the infinite horizon optimal control problem by increasing N until we observe
convergence. If the dynamic programming algorithm converges as N then
(8.14) becomes the Bellman equation
J (x) = minu
subj. to
(8.21)
8.4.1
(8.22)
It is time invariant and guarantees convergence to the origin for all states in X0 .
For a given x X0 , u (x) can be found from (8.21) by solving a standard
nonlinear programming problem.
In order to find the value function J (x) we do not need to compute the sequence
of cost-to-go functions implied by the dynamic program (8.14) but we can solve
(8.21) directly. We can start with some initial guess J0 (x) for the value function
and an initial guess X0 for the region in the state space where we expect the infinite
horizon problem to converge and iterate
Ji+1
(x) = minu
subj. to
(8.23)
(8.24)
Note that here i is the iteration index and does not denote time. This iterative
procedure is called value function iteration. It can be executed as follows. Let us
154
assume that at iteration step i we gridded the set Xi and that Ji (x) is known at each
grind point from the previous iteration. We can approximate Ji (x) at intermediate
points via interpolation. For a fixed point x the optimization problem (8.23) is a
nonlinear programming problem yielding Ji (
x). In this manner the approximate
value function Ji (x) can be constructed at all grid points and we can proceed to
the next iteration step i + 1.
8.4.2
J0
(x0 ) as N then the effect of increasing N on the value of u0 should
diminish as N . Thus, intuitively, instead of making the horizon infinite we
can get a similar behavior when we use a long, but finite horizon N, and repeat
this optimization at each time step, in effect moving the horizon forward (moving
horizon or receding horizon control).
In the batch approach we would solve an optimal control problem with horizon
N yielding a sequence of optimal inputs u0 , . . . , uN 1 , but we would implement
only the first one of these inputs u0 . At the next time step we would measure the
current state and then again solve the N -step problem with the current state as
new initial condition x0 . If the horizon N is long enough then we expect that this
approximation of the infinite horizon problem should not matter and the implemented sequence should drive the states to the origin.
In the dynamic programming approach we would always implement the control
u0 obtained from the optimization problem
J0N
(x0 ) =
min
u0
subj. to
q(x0 , u0 ) + J1N
(g(x0 , u0 ))
h(x0 , u0 ) 0,
g(x0 , u0 ) X1N ,
x0 = x(0)
(8.25)
where J1N
(g(x0 , u0 )) is the optimal cost-to-go from the state x1 = g(x0 , u0 ) at
time 1 to the end of the horizon N .
If the dynamic programming iterations converge as N , then for a long,
but finite horizon N we expect that this receding horizon approximation of the
infinite horizon problem should not matter and the resulting controller will drive
the system asymptotically to the origin.
In both the batch and the recursive approach, however, it is not obvious how
long N must be for the receding horizon controller to inherit these desirable convergence characteristics. Indeed, for computational simplicity we would like to
keep N small. We will argue next that the proposed control scheme guarantees
convergence just like the infinite horizon variety if we impose a specific terminal
constraint, for example, if we require the terminal region to be the origin Xf = 0.
8.4
155
(x1 ).
J0N
(x0 ) = min q(x0 , u0 ) + J1N
u0
(8.26)
Assume that we are at x(0) at time 0 and implement the optimal u0 that takes us
to the next state x1 = g(x(0), u0 ). At this state at time 1 we postulate to use over
the next N steps the sequence of optimal moves determined at the previous step
followed by zero: u1 , . . . , uN 1 , 0. This sequence is not optimal but the associated
cost over the shifted horizon from 1 to N + 1 can be easily determined. It consists
J0N
(x0 ) q(x0 , u0 ).
Because this postulated sequence of inputs is not optimal at time 1
J1N
+1 (x1 ) J0N (x0 ) q(x0 , u0 ).
Because the system and the objective are time invariant J1N
+1 (x1 ) = J0N (x1 )
so that
J0N
(x1 ) J0N
(x0 ) q(x0 , u0 ).
As q 0 for all (x, u) 6= (0, 0), the sequence of optimal costs J0N
(x0 ), J0N
(x1 ), . . .
is strictly decreasing for all (x, u) 6= (0, 0). Because the cost J0N 0 the sequence
J0N
(x0 ), J0N
(x1 ), . . . (and thus the sequence x0 , x1 ,. . .) is converging. Thus we
have established the following important theorem.
j+N
X
k=j
q(xk , uk ), q 0
(8.27)
Jjj+N
(xj ) ,
Thus we have established that a receding horizon controller with terminal constraint xN = 0 has the same desirable convergence characteristics as the infinite
156
horizon controller. At first sight the theorem appears very general and powerful. It is based on the implicit assumption, however, that at every time step the
CFTOC problem has a solution. Infeasibility would occur, for example, if the underlying system is not stabilizable. It could also happen that the constraints on
the inputs which restrict the control action prevent the system from reaching the
terminal state in N steps. In Chapter 13 we will present special formulations of
problem (8.28) such that feasibility at the initial time guarantees feasibility for
all future times. Furthermore in addition to asymptotic convergence to the origin we will establish stability for the closed-loop system with the receding horizon
controller.
Remark 8.1 For the sake of simplicity in the rest of the book we will use the following
shorter notation
Jj (xj ) , JjN
(xj ), j = 0, . . . , N
J (x0 ) , J0
(x0 )
(8.29)
Xj , XjN , j = 0, . . . , N
X , X0
U0 , U0N
and use the original notation only if needed.
8.5
Lyapunov Stability
While asymptotic convergence limk xk = 0 is a desirable property, it is generally not sufficient in practice. We would also like a system to stay in a small
neighborhood of the origin when it is disturbed slightly. Formally this is expressed
as Lyapunov stability.
8.5.1
(8.30)
with f (0) = 0.
Definition 8.1 (Lyapunov Stability) The equilibrium point x = 0 of system (8.30)
is
- stable (in the sense of Lyapunov) if, for each > 0, there is > 0 such that
kx0 k < kxk k < , k 0
- unstable if not stable
(8.31)
8.5
Lyapunov Stability
157
(8.32)
(8.33)
The following example shows that Lyapunov stability and convergence are, in
general, different properties.
Example 8.1 Consider the following system with one state x R:
xk+1 = xk (xk 1 |xk 1|).
(8.35)
The state x = 0 is an equilibrium for the system. For any state x [1, 1] we have
(x 1) 0 and the system dynamics (8.35) become
xk+1 = 2x2k 2xk .
(8.36)
System (8.36) generates oscillating and diverging trajectories for any x0 (1, 1)\{0}.
Any such trajectory will enter in finite time T the region with x 1. In this region
the system dynamics (8.35) become
xk+1 = 0, k T.
(8.37)
Therefore the origin is not Lyapunov stable, however the system converges to the
origin for all x0 (, +).
Usually to show Lyapunov stability of the origin for a particular system one
constructs a so called Lyapunov function, i.e., a function satisfying the conditions
of the following theorem.
158
Theorem 8.2 [177] Consider the equilibrium point x = 0 of system (8.30). Let
Rn be a closed and bounded set containing the origin. Let V : Rn R be a
function, continuous at the origin, such that
V (0) = 0 and V (x) > 0, x \ {0}
(8.38a)
(8.38b)
(8.39)
Condition (8.39) along with condition (8.38a) are sufficient to guarantee stability
of the origin as long as the set {xk : V (f (xk )) V (xk ) = 0} contains no trajectory
of the system xk+1 = f (xk ) except for xk = 0 for all k 0. This relaxation
of Theorem 8.2 is the so called Barbashin-Krasovski-LaSalle principle [176]. It
basically means that V (xk ) may stay constant and non zero at one or more time
instants as long as it does not do so at an equilibrium point or periodic orbit of
the system.
A similar result as Theorem 8.2 can be derived for global asymptotic stability,
i.e., = Rn .
Theorem 8.3 [168] Consider the equilibrium point x = 0 of system (8.30). Let
V : Rn R be a function, continuous at the origin, such that
kxk V (x)
(8.40a)
(8.40b)
(8.40c)
8.5
Lyapunov Stability
159
Definition 8.4 A radially unbounded Lyapunov function is called a Global Lyapunov Function.
Note that it was not enough just to restate Theorem 8.2 with = Rn but we
also have to require V (x) to be radially unbounded to guarantee global asymptotic
stability. To motivate this condition consider the candidate Lyapunov function for
a system in R2 [168]
x21
V (x) =
+ x22 ,
(8.41)
1 + x21
x2
V (x)
x2
-1-4
x1
-2
0
x1
which is depicted in Fig. 8.1, where x1 and x2 denote the first and second components of the state vector x, respectively. V (x) in (8.41) is not radially unbounded
as for x2 = 0
lim V (x) = 1.
x1
For this Lyapunov function even if condition (8.40c) is satisfied, the state x may
escape to infinity. Condition (8.40c) of Theorem 8.3 guarantees that the level sets
c of V (x) (c = {x Rn : V (x) c}) are closed.
The construction of suitable Lyapunov functions is a challenge except for linear
systems. First of all one can quite easily show that for linear systems Lyapunov
stability agrees with the notion of stability based on eigenvalue location.
Theorem 8.4 A linear system xk+1 = Axk is globally asymptotically stable in the
sense of Lyapunov if and only if all its eigenvalues are strictly inside the unit circle.
We also note that stability is always global for linear systems.
8.5.2
(8.42)
160
which satisfies conditions (8.40a)-(8.40b) of Theorem 8.3. In order to test condition (8.40c) we compute
V (xk+1 ) V (xk ) = xk+1 P xk+1 xk P xk = xk A P Axk xk P xk = xk (A P A P )xk
(8.43)
Therefore condition (8.40c) is satisfied if P 0 can be found such that
A P A P = Q, Q 0
(8.44)
(8.45)
which implies that after at most (n 1) steps the rate of decrease will become
nonzero. This is a special case of the the Barbashin-Krasovski-LaSalle principle.
Note that for C square and nonsingular Theorem 8.6 reduces to Theorem 8.5.
From (8.44) it follows that for stable systems and for a chosen Q 0 one can
always find P 0 solving
A P A P + Q 0
(8.46)
This Lyapunov inequality shows that for a stable system we can always find a P
such that V (x) = x P x decreases at a desired rate indicated by Q. We will need
this result later to prove stability of receding horizon control schemes.
8.5
8.5.3
Lyapunov Stability
161
V (x) = kP xkp
with P R
of full column rank satisfies the requirements (8.40a), (8.40b)
of a Lyapunov function. It can be shown that a matrix P can be found such
that condition (8.40c) is satisfied for the system xk+1 = Axk if and only if the
eigenvalues of A are inside the unit circle. The number of rows l necessary in P
depends on the system. The techniques to construct P are based on the following
theorem [169, 228].
Theorem 8.7 Let P Rln with rank(P ) = n and p {1, }. The function
V (x) = kP xkp
(8.47)
k 0,
(8.48)
(8.49a)
kHkp < 1.
(8.49b)
(8.50)
Note that the inequality (8.50) is equivalent to the Lyapunov inequality (8.46)
when the squared two-norm is replaced by the 1 or norm. Once we have
constructed a P and H to fulfill the conditions of Theorem 8.7 we can easily find
P to satisfy (8.50) according to the following lemma.
Lemma 8.1 Let P and H be matrices satisfying conditions (8.49), with P full
column rank. Let , 1 kHk , , kQP # k , where P # , (P T P )1 P T is the
left pseudoinverse of P . Then, the square matrix
P = P
(8.51)
162
Chapter 9
9.1
Problem Formulation
We consider a special case of the problem stated in the last chapter, where the
system is linear and time-invariant
x(t + 1) = Ax(t) + Bu(t).
(9.1)
Again, x(t) Rn and u(t) Rm are the state and input vectors respectively.
We define the following quadratic cost function over a finite horizon of N steps
J0 (x0 , U0 ) , xN P xN +
N
1
X
xk Qxk + uk Ruk ,
(9.2)
k=0
where xk denotes the state vector at time k obtained by starting from the state
x0 = x(0) and applying to the system model
xk+1 = Axk + Buk
(9.3)
the input sequence U0 , [u0 , . . . , uN 1 ] . Consider the finite time optimal control
problem
J0 (x(0)) =
J0 (x(0), U0 )
minU0
subj. to xk+1 = Axk + Buk , k = 0, 1, . . . , N 1
x0 = x(0).
(9.4)
164
9.2
x(0)
x1
..
.
..
.
xN
{z
X
=
}
I
A
..
.
..
.
AN
{z
Sx
x(0) +
0
B
AB
..
.
AN 1 B
... ... 0
u0
0 ... 0
..
.
..
.
.
. . . ..
..
.
..
.
.
. . . ..
uN 1
... ... B
{z
}
(9.5)
Su
Here all future states are explicit functions of the present state x(0) and the future
inputs u0 , u1 , u2 , . . . only. By defining the appropriate quantities we can rewrite
this expression compactly as
X = S x x(0) + S u .U0
(9.6)
(9.7)
= blockdiag{Q, , Q, P }, Q
0, and R
= blockdiag{R, , R}, R
0.
where Q
Substituting (9.6) into the objective function (9.7) yields
J0 (x(0), U0 )
0
(S x x(0) + S u U0 ) + U0 RU
= (S x x(0) + S u U0 ) Q
u u
x u
x ) x(0)
= U0 (S QS + R) U0 + 2x (0) (S QS ) U0 + x (0) (S x QS
|
| {z }
| {z }
{z
}
H
(9.8)
0, also H 0. Thus J0 (x(0), U0 ) is a positive definite quadratic
Because R
function of U0 . Therefore, its minimum can be found by computing its gradient
and setting it to zero. This yields the optimal vector of future inputs
U0 (x(0))
=
=
H 1 F x(0)
x x(0).
u+R
1 S u QS
S u QS
(9.9)
9.3
165
(9.10)
Note that the optimal vector of future inputs U0 (x(0)) is a linear function (9.9) of
the initial state x(0) and the optimal cost J0 (x(0)) is a quadratic function (9.10)
of the initial state x(0).
9.3
Jj (xj ) =
min
uj , ,uN 1
xN P xN +
N
1
X
xk Qxk + uk Ruk .
k=j
According to the principle of optimality the optimal one step cost-to-go can be
obtained from
JN
1 (xN 1 ) = min xN PN xN + xN 1 QxN 1 + uN 1 RuN 1
uN 1
xN
PN
= AxN 1 + BuN 1
= P.
JN
1 (xN 1 ) = minuN 1 xN 1 (A PN A + Q)xN 1
+2xN 1 A PN BuN 1
+uN 1 (B PN B + R)uN 1 .
(9.11)
(9.12)
(9.13)
(9.14)
JN
1 (xN 1 ) = xN 1 PN 1 xN 1 ,
(9.15)
PN 1 = A PN A + Q A PN B(B PN B + R)1 B PN A.
(9.16)
FN 1
166
At the next stage, consider the two-step problem from time N 2 forward:
JN
2 (xN 2 ) = min xN 1 PN 1 xN 1 + xN 2 QxN 2 + uN 2 RuN 2
uN 2
xN 1
AxN 2 + BuN 2 .
(9.17)
(9.18)
We recognize that (9.17), (9.18) has the same form as (9.11), (9.12). Therefore we
can state the optimal solution directly.
uN 2 = (B PN 1 B + R)1 B PN 1 A xN 2 .
|
{z
}
(9.19)
JN
2 (xN 2 ) = xN 2 PN 2 xN 2 ,
(9.20)
FN 2
where we defined
PN 2 = A PN 1 A + Q A PN 1 B(B PN 1 B + R)1 B PN 1 A.
(9.21)
Continuing in this manner, at some arbitrary time k the optimal control action is
u (k) =
=
(9.22)
where
Pk = A Pk+1 A + Q A Pk+1 B(B Pk+1 B + R)1 B Pk+1 A
(9.23)
and the optimal cost-to-go starting from the measured state x(k) is
Jk (x(k)) = x (k)Pk x(k).
(9.24)
Equation (9.23) (called Discrete Time Riccati Equation or Riccati Difference Equation - RDE) is initialized with PN = P and is solved backwards, i.e., starting with
PN and solving for PN 1 , etc. Note from (9.22) that the optimal control action
u (k) is obtained in the form of a feedback law as a linear function of the measured
state x(k) at time k. The optimal cost-to-go (9.24) is found to be a quadratic
function of the state at time k.
Remark 9.1 According to Section 8.4.2, the receding horizon control policy is obtained by solving problem (9.4) at each time step t with x0 = x(t). Consider the
state-feedback solution u (k) in (9.22) to problem (9.4). Then, the receding horizon
control policy is:
u (t) = F0 x(t), t 0
(9.25)
9.4
9.4
167
U0 = S u QS
1
x x(0).
S u QS
(9.26)
The recursive dynamic programming approach yields a feedback policy, i.e., a sequence of feedback laws expressing at each time step the control action as a function
of the state at that time.
u (k) = Fk x(k), for k = 0, . . . , N 1.
(9.27)
minuj , ,uN 1
subj. to
xN P xN +
xj = x(j),
PN 1
k=j
xk Qxk + uk Ruk
(9.28)
where we note that the horizon length is shrinking at each time step.
As seen from (9.26) the solution to (9.28) relates the sequence of inputs uj , uj+1 , . . .
to the state x(j) through a linear expression. The first part of this expression yields
again the optimal feedback law (9.27) at time j, u (j) = Fj x(j).
Here the dynamic programming approach is clearly a more efficient way to generate the feedback policy because it only uses a simple matrix recursion (9.23).
Repeated application of the batch approach, on the other hand, requires the repeated inversion of a potentially large matrix in (9.26). For such inversion, however,
one can take advantage of the fact that only a small part of the matrix H changes
at every time step.
What makes dynamic programming so effective here is that in this special case,
where the system is linear and the objective is quadratic, the optimal cost-to-go,
the value function Jj (x(j)) has a very simple form: it is quadratic. If we make the
168
problem only slightly more complicated, e.g., if we add constraints on the inputs or
states, the value function can still be constructed, but it is much more complex. In
general, the value function can only be approximated as discussed in the previous
chapter. Then a repeated application of the batch policy, where we resolve the
optimization problem at each time step is an attractive alternative.
9.5
(x(0)) = min
J
u0 ,u1 ,...
xk Qxk + uk Ruk .
(9.29)
k=0
Since the prediction must be carried out to infinity, application of the batch method
becomes impossible. On the other hand, derivation of the optimal feedback law via
dynamic programming remains viable. We can initialize the RDE (9.23)
Pk = A Pk+1 A + Q A Pk+1 B(B Pk+1 B + R)1 B Pk+1 A
(9.30)
with the terminal cost matrix P0 = Q and solve it backwards for k . Let
us assume for the moment that the iterations converge to a solution P . Such P
would then satisfy the Algebraic Riccati Equation (ARE)
P = A P A + Q A P B(B P B + R)1 B P A.
(9.31)
k = 0, ,
(9.32)
J
(x(0)) = x(0) P x(0).
(9.33)
9.6
169
9.6
(9.34)
J
(x) = x P x
(9.35)
is a Lyapunov function (and thus proving the last part of Theorem 9.1).
Let Q = C C and R = D D with det(D) 6= 0. The proof
of two main
consists
C
, [A BF ]
steps: (i) prove that if (C, A) is an observable pair then
DF
is also observable, (ii) prove that the ARE equation (9.31) can be rewritten as
P = (A BF ) P (A BF ) +
C
DF
C
DF
(9.36)
Once (i) and (ii) are proven, the final result is immediate.
In fact, equation
(9.36)
is
C
, [A BF ]
a Lyapunov equation for (A BF ). Since P 0 and
DF
is observable we can use Theorem 8.6 and claim the asymptotic stability of the
closed-loop system A BF for any F .
Proof of (i). Since det(D) 6= 0 then B = M D for some M . We have
zI A
I 0 M
zI A
zI (A BF )
= rank 0 I 0
= rank
.
C
C
C
rank
DF
0 0 I
DF
DF
(9.37)
Since (C, A) is observable, by the Hautus condition
zI A
rank
= n for every z
(9.38)
C
and therefore by (9.37),
C
DF
, [A BF ]
170
= A P A F
B P A A P BF + F
B P BF + Q + F RF
= A P A F
B P A A P BF + F
(B P B + R)F + Q.
(9.39)
From the definition of F in (9.32), and the previous equation we have
P = A P A A P B(B P B + R)1 B P A + Q,
which is the ARE equation (9.31).
(9.41)
Chapter 10
(10.1)
Again, x(t) Rn and u(t) Rm are the state and input vector respectively.
We define the following piecewise linear cost function over a finite horizon of N
steps
N
1
X
J0 (x0 , U0 ) , kP xN kp +
kQxk kp + kRuk kp
(10.2)
k=0
(10.3)
minU0
subj. to
J0 (x(0), U0 )
xk+1 = Axk + Buk , k = 0, 1, . . . , N 1
x0 = x(0).
(10.4)
172
10.1
10
so that all future states are explicit functions of the present state x(0) and the
future inputs u0 , u1 , u2 , . . . only.
The optimal control problem (10.4) with p = can be rewritten as a linear
program by using the following standard approach (see e.g. [75]). The sum of
components of any vector {x0 , . . . , xN , u0 , . . . , uN 1 } that satisfies
1n xk Qxk , k = 0, 1, . . . , N 1
1n xk Qxk , k = 0, 1, . . . , N 1
1r xN P xN ,
1r xN P xN ,
1m uk Ruk , k = 0, 1, . . . , N 1
1m uk Ruk , k = 0, 1, . . . , N 1
(10.6)
subj. to
x0 + . . . + xN + u0 + . . . + uN 1
1n xk Q Ak x0 +
k1
X
j=0
1r xN P AN x0 +
1m uk Ruk
k = 0, . . . , N 1
x0 = x(0),
(10.7a)
Aj Buk1j
N
1
X
j=0
(10.7b)
Aj BuN 1j
(10.7c)
(10.7d)
(10.7e)
where constraints (10.7b)(10.7d) are componentwise, and means that the constraint appears once with each sign, as in (10.6).
Remark 10.1 The cost function (10.2) with p = can be interpreted as a special
case of a cost function with 1-norm over time and -norm over space. For instance,
10.1
173
the dual choice (-norm over time and 1-norm over space) leads to the following cost
function
J0 (x(0), U0 ) , max {kQxk k1 + kRuk k1 }.
(10.8)
k=0,...,N
c0 z0
subj. to
G z0 w + S x(0),
z0
(10.9)
c0
2nN+2r
w
S
=
=
mN
z }| { z }| { z }|
[1 . . . 1 1 . . . 1 0 . . .
N+1
}|
z
0
...
1n
1n
0
...
0
1n
...
0
1n
...
...
...
...
0
. . . 1n
0
. . . 1n
0
.
..
0
0
.
.
.
0
0
0
...
0
0
...
...
.
.
.
...
0
0
...
0
0
...
0
0
0
0
...
0
0
1r
1r
0
0
...
0
0
0
0
0
0
...
0
0
0
0
1m
1m
...
0
0
2mN
z }| { z }| {
[0 . . . 0 0 . . . 0]
z
[ Q
{
0]
(QA)
}|
...
...
...
...
...
...
...
...
...
...
...
...
...
...
0
0
0
0
...
0
0
0
0
0
0
...
1m
1m
mN
0
0
QB
QB
...
QAN2 B
QAN2 B
P AN1 B
P AN1 B
R
R
...
0
0
(QA)
...
...
...
...
...
...
...
...
...
...
...
...
...
...
2r
2nN
}|
(QA2 )
}|
0
0
0
0
...
QAN3 B
QAN3 B
P AN2 B
P AN2 B
0
0
...
0
0
...
(QAN1 )
0
0
0
0
...
0
0
PB
P B
0
0
...
R
R
2mN
}|
{ z
{ z
}|
{
(QAN1 ) ) (P AN ) (P AN ) 0m . . . 0m ] .
(10.10)
Note that in (10.10) we include the zero vector w to make the notation consistent
with the one used in Section 7.2.
By treating x(0) as a vector of parameters, the problem (10.9) becomes a multiparametric linear program (mp-LP) that can be solved as described in Section 7.2.
174
10
Once the multiparametric problem (10.7) has been solved, the explicit solution
z0 (x(0)) of (10.9) is available as a piecewise affine function of x(0), and the optimal control law U0 is also available explicitly, as the optimal input U0 consists
simply of the last part of z0 (x(0))
U0 (x(0)) = [0 . . . 0 Im Im . . . Im ]z0 (x(0)).
(10.11)
Theorem 7.5 states that there exists a continuous and PPWA solution z0 (x)
of the mp-LP problem (10.9). Clearly the same properties are inherited by the
controller. The following Corollaries of Theorem 7.5 summarize the analytical
properties of the optimal control law and the value function.
Corollary 10.1 There exists a control law U0 = f0 (x(0)), f0 : Rn Rm , obtained
as a solution of the optimal control problem (10.2)-(10.4) with p = 1 or p = ,
which is continuous and PPWA
f0 (x) = F0i x
if
x CR0i , i = 1, . . . , N0r ,
(10.12)
where the polyhedral sets CR0i , {H0i x 0}, i = 1, . . . , N0r , are a partition of Rn .
Note that in Corollary 10.1 the control law is linear (not affine) and the critical
regions have a conic shape (CR0i , {H0i x 0}). This can be proven immediately
from the results in Section 7.2 by observing that the constant term w at the
right-hand side on the mp-LP problem (10.9) is zero.
Corollary 10.2 The value function J (x) obtained as a solution of the optimal
control problem (10.2)-(10.4) is convex and PPWA.
Remark 10.2 Note that if the optimizer of problem (10.4) is unique for all x(0) Rn ,
then Corollary 10.1 reads: The control law U (0) = f0 (x(0)), f0 : Rn Rm ,
obtained as a solution of the optimal control problem (10.2)-(10.4) with p = 1 or
p = , is continuous and PPWA,. . .. From the results of Section 7.2 we know that
in case of multiple optimizers for some x(0) Rn , a control law of the form (10.12)
can always be computed.
10.2
min
uj , ,uN 1
kP xN k +
N
1
X
k=j
kQxk k + kRuk k .
According to the principle of optimality the optimal one step cost-to-go can be
obtained from
JN
1 (xN 1 ) = min kPN xN k + kQxN 1 k + kRuN 1 k
uN 1
(10.13)
10.2
= AxN 1 + BuN 1
= P.
175
(10.14)
JN
1 (xN 1 ) = min kPN (AxN 1 + BuN 1 )k + kQxN 1 k + kRuN 1 k .
uN 1
(10.15)
We find the optimum by solving the mp-LP
min
u
x
x
N 1 ,N ,N 1 ,uN 1
subj. to
xN 1 + xN + uN 1
(10.16a)
1n xN 1 QxN 1
1rN xN PN [AxN 1 + BuN 1 ]
(10.16b)
(10.16c)
1m uN 1 RuN 1,
(10.16d)
JN
1 (xN 1 ) = kPN 1 xN 1 k
(10.17)
with PN 1 defined appropriately. At the next stage, consider the two-step problem
from time N 2 forward:
JN
2 (xN 2 ) = min kPN 1 xN 1 k + kQxN 2 k + kRuN 2 k
uN 2
xN 1
= AxN 2 + BuN 2 .
(10.18)
(10.19)
We recognize that (10.18), (10.19) has the same form as (10.13), (10.14). Therefore we can compute the optimal solution again by solving the mp-LP
min
x
u
x
N 2 ,N 1 ,N 2 ,uN 2
subj. to
xN 2 + xN 1 + uN 2
(10.20a)
1n xN 2 QxN 2
(10.20b)
x
1rN 1 N 1 PN 1 [AxN 2 + BuN 2 ] (10.20c)
1m uN 2 RuN 2,
(10.20d)
JN
(10.21)
2 (xN 2 ) = kPN 2 xN 2 k ,
Continuing in this manner, at some arbitrary time k the optimal control action is
u (k) = fk (x(k)),
(10.22)
176
10
Hki x 0, i = 1, . . . , Nkr ,
(10.23)
where the polyhedral sets {Hki x 0}, i = 1, . . . , Nkr , are a partition of Rn . The
optimal cost-to-go starting from the measured state x(k) is
Jk (x(k)) = kPk x(k)k .
(10.24)
Here we have introduced the notation Pk to express the optimal cost-to-go Jk (x(k)) =
kPk x(k)k from time k to the end of the horizon N . We also remark that the rows
of Pk correspond to the different affine functions constituting Jk and thus their
number varies with the time index k. Clearly, we do not have a closed form as
for the 2-norm with the Riccati Difference Equation (9.23) linking cost and control
law at time k given their value at time k 1.
10.3
0r .
0i x(0) 0, i = 1, . . . , N
H
(10.25)
The recursive dynamic programming approach yields a feedback policy, i.e., a sequence of feedback laws expressing at each time step the control action as a function
of the state at that time.
u (k) = Fki x(k)
(10.26)
10.4
177
optimization problem
Jj (x(j)) =
minuj , ,uN 1
subj. to
kP xN k +
xj = x(j),
PN 1
k=j
kQxk k + kRuk k
(10.27)
where we note that the horizon length is shrinking at each time step.
As seen from (10.25) the solution to (10.27) relates the sequence of inputs
uj , uj+1 , . . . to the state x(j) through a linear expression. The first part of this
expression yields again the optimal feedback law (10.26) at time j, u (j) = fj (x(j)).
Here the dynamic programming approach is clearly a more efficient way to
generate the feedback policy because it requires the solution of a small mp-LP
problem (10.7) for each time step. Repeated application of the batch approach, on
the other hand, requires repeatedly the solution of a larger mp-LP for each time
step.
10.4
J
(x(0)) =
min
u(0),u(1),...
k=0
kQxk k + kRuk k .
(10.28)
Since the prediction must be carried out to infinity, application of the batch method
becomes impossible. On the other hand, derivation of the optimal feedback law
via dynamic programming remains viable. We can use the dynamic programming
formulation
kPj xj k = min kPj+1 xj+1 k + kQxj k + kRuj k
uj
xj+1
= Axj + Buj
(10.29)
(10.30)
with the terminal cost matrix P0 = Q and solve it backwards for k . Let
us assume for the moment that the iterations converge to a solution P in a finite
number of iterations. Then the optimal feedback control law is time-invariant and
piecewise affine
u (k) = F i x(k) if
H i x 0, i = 1, . . . , N r
(10.31)
J
(x(0)) = kP x(0)k .
(10.32)
178
10
(10.33)
0
[ 9.44 29.44 ] x
if
0.71 0.71 x [ 0 ]
0.10 1.00
[ 9.00 25.00 ] x
if
0.11 0.99 x [ 0 ]
0.89
0
[ 1.00 19.00 ] x
if [ 0.45
0.71 0.71 ] x [ 0 ]
0.11 0.99
[ 8.00 16.00 ] x
if
0.12 0.99 x [ 0 ]
0.95
0
[ 2.00 17.00 ] x
if [ 0.32
0.45 0.89 ] x [ 0 ]
0.12 0.99
if
0.14 0.99 x [ 0 ]
[ 7.00 8.00 ] x
0.32 0.95
[ 3.00 14.00 ] x
if
0.24 0.97 x [ 0 ]
0.14 0.99
[ 6.00 1.00 ] x
if
0.16 0.99 x [ 0 ]
0.24 0.97
[ 4.00 10.00 ] x
if
0.20 0.98 x [ 0 ]
0.16 0.99
[ 5.00 5.00 ] x
if
0.20 0.98 x [ 0 ]
0.20 0.98
u=
0
[ 5.00 5.00 ] x
if
0.16 0.99 x [ 0 ]
0.20 0.98
[ 4.00 10.00 ] x
if
0.24 0.97 x [ 0 ]
0.16 0.99
[ 6.00 1.00 ] x
if
0.14 0.99 x [ 0 ]
0.24 0.97
[ 3.00 14.00 ] x
if
0.32 0.95 x [ 0 ]
0.14 0.99
[ 7.00 8.00 ] x
if
0.12 0.99 x [ 0 ]
0.32 0.95
[ 2.00 17.00 ] x
if
0.45 0.89 x [ 0 ]
0.12 0.99
[ 8.00 16.00 ] x if
0.11 0.99 x [ 0 ]
0.45 0.89
[ 1.00 19.00 ] x
if
0.71 0.71 x [ 0 ]
0.11 0.99
[ 9.00 25.00 ] x if
0.10 1.00 x [ 0 ]
0
0.10 1.00
0.71 ] x [ 0 ]
10.4
with P equal to
179
9.44
9.00
1.00
8.00
2.00
7.00
3.00
6.00
4.00
5.00
=
5.00
4.00
6.00
3.00
7.00
2.00
8.00
1.00
9.00
9.44
29.44
25.00
19.00
16.00
17.00
8.00
14.00
1.00
10.00
5.00
5.00
10.00
1.00
14.00
8.00
17.00
16.00
19.00
25.00
(10.34)
29.44
J (x)
Note that P in (10.34) has 20 rows corresponding to the 20 linear terms (or pieces)
J (x) in region 1 is J (x) = 9.44x1 + 29.44x2 , where x1 and x2 denote the first
and second component of the state vector x, respectively. Note that each linear term
x2
x1
180
10
Part IV
Chapter 11
Controllability, Reachability
and Invariance
This chapter is a self-contained introduction to reachability and invariant set theory.
11.1
(11.1)
(11.2)
(11.3)
(11.4)
Pre(S) is the set of states which evolve into the target set S in one time step.
Definition 11.2 For the system (11.2) we denote the one-step controllable set to
the set S as
Pre(S) , {x Rn : u U s.t. f (x, u) S}.
(11.5)
184
11
For a system with inputs, Pre(S) is the set of states which can be driven into the
target set S in one time step while satisfying input and state constraints.
Definition 11.3 For the autonomous system (11.1) we denote the one-step reachable set from the set S as
Reach(S) , {x Rn : x(0) S s.t. x = fa (x(0))}.
Definition 11.4 For the system (11.2) with inputs we will denote the one-step
reachable set from the set S as
Reach(S) , {x Rn : x(0) S, u(0) U s.t. x = f (x(0), u(0))}.
Therefore, all the states contained in S are mapped into the set Reach(S) under
the map fa or under the map f for some input u U.
Remark 11.1 Note that in our notation we use the word controllable for the Pre(S)
set even if the system is autonomous.
The sets Pre(S) and Reach(S) are also denoted as one-step backward-reachable set
and one-step forward-reachable set, respectively, in the literature.
N -step controllable and reachable sets are defined by iterating Pre() and Reach()
computations, respectively.
Definition 11.5 (N -Step Controllable Set KN (S)) For a given target set S
X , the N -step controllable set KN (S) of the system (11.1) or (11.2) subject to the
constraints (11.3) is defined recursively as:
Kj (S) , Pre(Kj1 (S)) X , K0 (S) = S,
j {1, . . . , N }
(11.6)
From Definition 11.5, all states x0 of the system (11.1) belonging to the N -Step
Controllable Set KN (S) will evolve to the target set S in N steps, while satisfying
state constraints.
Also, all states x0 of the system (11.2) belonging to the N -Step Controllable
Set KN (S) can be driven, by a suitable control sequence, to the target set S in N
steps, while satisfying input and state constraints.
Definition 11.6 (N -Step Reachable Set RN (X0 )) For a given initial set X0
X , the N -step reachable set RN (X0 ) of the system (11.1) or (11.2) subject to the
constraints (11.3) is defined as:
Ri+1 (X0 ) , Reach(Ri (X0 )), R0 (X0 ) = X0 ,
i = 0, . . . , N 1
(11.7)
From Definition 11.6, all states x0 belonging to X0 will evolve to the N -step reachable set RN (X0 ) in N steps.
11.1
11.1.1
185
(11.8)
(11.9)
The set Pre(X ) can be obtained as follows: Since the set X is a polytope, it can be
represented as a H-polytope (Section 5.2)
X = {x : Hx h},
where
1
0
H=
1
0
(11.10)
10
0
10
1
and h = .
10
0
10
1
By using this H-presentation and the system equation (11.8), the set Pre(X ) can be
derived:
Pre(X )
=
=
{x : Hfa (x) h}
{x : HAx h} .
(11.11)
(11.12)
The set (11.12) may contain redundant inequalities which can be removed by using
Algorithm 5.1 in Section 5.4.1 to obtain its minimal representation. Note that by
using the notation in Section 5.4.11, the set Pre(X ) in (11.12) is simply X A.
The set Pre(X ) is
Pre(X ) = x :
1
20
0
0.5
x 10 .
20
0
10
0.5
The set Pre(X ) X , the significance of which we will discuss in Section 11.2, is
10
1
0
10
0
1
10
1
0
Pre(X ) X = x :
x
1
10
10
1
0.5
10
1
0.5
and it is depicted in Fig. 11.1.
186
11
10
x2
5
0
Pre(X )
-5
X
-10
-10
0
x1
-5
10
Figure 11.1 Example 11.1: one-step controllable set Pre(X ) X for system
(11.8) under constraints (11.9).
The set Reach(X ) is obtained by applying the map A to the set X . Let us write X
in V-representation (see Section 5.1)
X = conv(V ),
(11.13)
and let us map the set of vertices V through the transformation A. Because the
transformation is linear, the reach set is simply the convex hull of the transformed
vertices
Reach(X ) = A X = conv(AV ).
(11.14)
We refer the reader to Section 5.4.11 for a detailed discussion on linear transformations of polyhedra.
10
5
0
-5
-10
1
1
1
1
Re
a
x2
is
5
0
5
0
x
2.5
0.5
2.5
0.5
ch
(X
)
Reach(X ) = x :
-10 -5
0
x1
X
5 10
Figure 11.2 Example 11.1: one-step reachable set for system (11.8).
(11.15)
11.1
187
(11.16a)
(11.16b)
For the non-autonomous system (11.15), the set Pre(X ) can be computed using the
H-representation of X and U,
X = {x | Hx h},
U = {u | Hu u hu },
(11.17)
to obtain
Pre(X )
=
=
x R2 | u U s.t. f (x, u) X ,
h
x
HA HB
.
x R2 | u R s.t.
hu
u
0
Hu
(11.18)
(11.19)
The half-spaces in (11.19) define a polytope in the state-input space, and a projection
operation (see Section 5.4.6) is used to derive the half-spaces which define Pre(X ) in
the state space. The set Pre(X ) X is depicted in Fig. 11.3 and reported below:
10
1
0
10
0
1
1
0
x 10
10
0
1
10
1
1.5
10
1
1.5
10
x2
5
0
Pre(X ) X
-5
X
-10
-10
-5
0
x1
10
Figure 11.3 Example 11.2: one-step controllable set Pre(X ) X for system
(11.15) under constraints (11.16).
Note that by using the definition of the Minkowski sum given in Section 5.4.9 and
the affine operation on polyhedra in Section 5.4.11 we can write the operations in
(11.19) compactly as follows:
Pre(X ) =
{x
{x
{x
{x
{x
{x
:
:
:
:
:
:
u U s.t. Ax + Bu X }
y = Ax + Bu, y X , u U}
Ax = y + (Bu), y X , u U}
Ax C, C = X (B) U}
x C A, C = X (B) U}
x (X (B) U) A} .
(11.20)
188
11
Reach(X ) = {y + Bu : y A X , u U}.
We can use the definition of the Minkowski sum given in Section 5.4.9 and rewrite
the set Reach(X ) as
Reach(X ) = (A X ) (B U).
We can compute the Minkowski sum via projection or vertex enumeration as explained in Section 5.4.9 and obtain the set Reach(X ) in H-representation
20
1
0
20
1
25
0
1
,
x
Reach(X ) = x :
25
0
1
27.5
1
1.5
27.5
1
1.5
40
x2
20
Reach(X )
0
-20
-40
-20
-10
0
x1
10
20
Figure 11.4 Example 11.2: one-step reachable sets for system (11.15) under
constraints (11.16).
In summary, the sets Pre(X ) and Reach(X ) are the results of linear operations
on the polyhedra X and U and therefore are polyhedra. By using the definition
of the Minkowski sum given in Section 5.4.9 and of affine operation on polyhedra
in Section 5.4.11 we can compactly summarize the Pre and Reach operations on
linear systems in Table 11.1.
The N -step controllable set KN (S) and the N -step reachable set RN (X0 )
can be computed by using their recursive formulas and computing the
Pre and Reach operations as in Table 11.1.
Example 11.3 Consider the second order unstable system (11.15) subject to the input
and state constraints (11.16). Consider the target set
1
1
.
x
S = x|
1
1
11.1
Pre(X )
Reach(X )
189
x(t + 1) = Ax(t)
X A
AX
Table 11.1 Pre and Reach operations for linear systems subject to polyhedral
state and input constraints x(t) X , u(t) U
The N -step controllable set KN (S) of the system (11.15) subject to the constraints (11.16)
can be computed by using the recursively formula (11.6)
Kj (S) , Pre(Kj1 (S)) X , K0 (S) = S,
j {1, . . . , N }
and the steps described in Example 11.2 to compute the Pre() set.
The sets KN (S) for N = 1, 2, 3, 4 are depicted in Fig. 11.5. The sets are shifted along
the x-axis for a clearer visualization.
K4 (S)
K3 (S)
K2 (S)
K1 (S)
Figure 11.5 Example 11.3: one-step controllable sets Kj (S) for system
(11.15) under constraints (11.16) for N=1,2,3,4. Note that the sets are
shifted along the x-axis for a clearer visualization.
Example 11.4 Consider the second order unstable system (11.15) subject to the input
and state constraints (11.16). Consider the initial set
1
1
X0 = x |
x
.
1
1
The N -step reachable set RN (X0 ) of the system (11.15) subject to the constraints (11.16)
can be computed by using the recursively formula (11.7)
Ri+1 (X0 ) , Reach(Ri (X0 )), R0 (X0 ) = X0 ,
i = 0, . . . , N 1
and the steps described in Example 11.2 to compute the Reach() set.
The sets KN (X0 )) for N = 1, 2, 3, 4 are depicted in Fig. 11.6. The sets are shifted
along the x-axis for a clearer visualization.
190
11
X0
R1 (X0 )
R2 (X0 )
R3 (X0 )
R4 (X0 )
Figure 11.6 Example 11.3: N-step reachable sets for system (11.15) under
constraints (11.16).
11.2
Invariant Sets
Consider the constrained autonomous system (11.1) and the constrained system
subject to external inputs (11.2) defined in Section 11.1.
Two different types of sets are considered in this section: invariant sets and
control invariant sets. We will first discuss invariant sets.
Remark 11.3 Note that, in general, the nonlinear system (11.1) may have multiple equilibrium points, and thus O might be the union of disconnected sets each
11.2
Invariant Sets
191
/ Pre(O)
and thus O * Pre(O)
It is immediate to prove that condition (11.22) of Theorem 11.1 is equivalent
to the following condition
Pre(O) O = O.
(11.23)
Based on condition (11.23), the following algorithm provides a procedure for computing the maximal positive invariant subset O for system (11.1),(11.3) [9, 47,
165, 118].
Algorithm 11.1 Computation of O
input fa , X
output O
let 0 X
repeat
k =k+1
k+1 Pre(k ) k
until k+1 = k
O k
Conditions for finite time termination of Algorithm 11.1 can be found in [117].
A simple sufficient condition for finite time termination of Algorithm 11.1 requires
the system fa (x) to be linear and stable, and the constraint set X to be bounded
and to contain the origin.
192
11
Example 11.5 Consider the second order stable system in Example 11.1. The maximal positive invariant set of system (11.8) subject to constraints (11.9) is depicted
in Fig. 11.7 and reported below:
10
1
0
10
0
1
1
0
x 10 .
10
0
1
10
1
0.5
10
1
0.5
Note from the previous discussion of the example and from Figure 11.1 that
here the maximal positive invariant set O is obtained after a single step of Algorithm 11.1, i.e.
O = 1 = Pre(X ) X .
10
x2
5
0
-5
-10
X
-10
-5
0
x1
10
Figure 11.7 Maximal Positive Invariant Set of system (11.8) under constraints (11.9).
t N+ .
11.2
Invariant Sets
193
The following algorithm provides a procedure for computing the maximal control invariant set C for system (11.2),(11.3) [9, 47, 165, 118].
Algorithm 11.2 Computation of C
input: f , X and U
output: C
let 0 X
repeat
k =k+1
k+1 Pre(k ) k
until k+1 = k
C k+1
Algorithm 11.2 generates the set sequence {k } satisfying k+1 k , k N. Algorithm 11.2 terminates when k+1 = k . If it terminates, then k is the maximal
control invariant set C for the system (11.2)-(11.3). In general, Algorithm 11.2
may never terminate [9, 47, 165, 158]. If the algorithm does not terminate in a finite number of iterations, in general, convergence to the maximal control invariant
set is not guaranteed
C 6= lim k .
(11.24)
k+
The work in [48] reports examples of nonlinear systems where (11.24) can be observed. A sufficient condition for the convergence of k to C as k + requires
the polyhedral sets X and U to be bounded and the system f (x, u) to be continuous [48].
Example 11.6 Consider the second order unstable system in example 11.2. Algorithm 11.2 is used to compute the maximal control invariant set of system (11.15)
subject to constraints (11.16). Algorithm 11.2 terminates after 45 iterations and the
maximal control invariant set C is:
4
0
1
4
0
1
0.55
0.83
x 2.22 .
2.22
0.55
0.83
10
1
0
10
1
0
194
11
10
x2
5
0
-5
-10
X
-10
-5
0
x1
10
Definition 11.11 (Finitely determined set) Consider Algorithm 11.1 (Algorithm 11.2). The set O (C ) is finitely determined if and only if i N such
that i+1 = i . The smallest element i N such that i+1 = i is called the
determinedness index.
Remark 11.5 From the results in Section 11.1.1, for linear system with linear constraints the sets O and C are polyhedra if they are finitely determined.
For all states contained in the maximal control invariant set C there exists
a control law such that the system constraints are never violated. This does not
imply that there exists a control law which can drive the state into a user-specified
target set. This issue is addressed in the following by introducing the concepts of
maximal controllable sets and stabilizable sets.
Definition 11.12 (Maximal Controllable Set K (O)) For a given target set
O X , the maximal controllable set K (O) for system (11.2) subject to the constraints in (11.3) is the union of all N -step controllable sets KN (O) contained in
X (N N).
We will often deal with controllable sets KN (O) where the target O is a control
invariant set. They are special sets, since in addition to guaranteeing that from
KN (O) we reach O in N steps, one can ensure that once it has reached O, the
system can stay there at all future time instants.
Definition 11.13 (N -step (Maximal) Stabilizable Set) For a given control
invariant set O X , the N -step (maximal) stabilizable set of the system (11.2)
subject to the constraints (11.3) is the N -step (maximal) controllable set KN (O)
(K (O)).
The set K (O) contains all states which can be steered into the control invariant
set O and hence K (O) C . The set K (O) C can be computed as follows
(cf. [55, 48]):
11.2
Invariant Sets
195
until Kc+1 = Kc ,
K (O) Kc ,
(11.25a)
(11.25b)
1
2x(t)
u(t) =
if
if
if
x(t) 1, 21
1
1
x(t) 2 , 2
x(t) 12 , 1 .
(11.26)
The closed-loop system has three equilibria at 1, 0, and 1 and system (11.25) is
always feasible for all initial states in [1, 1] and therefore C = [1, 1]. It is,
however, asymptotically stable only for the open set (1, 1). In fact, u(t) and any
other feasible control law cannot stabilize the system from x = 1 and from x = 1
and therefore when O = then K (O) = (1, 1) C . We note in this example
that the maximal stabilizable set is open. One can easily argue that, in general, if the
maximal stabilizable set is closed then it is equal to the maximal control invariant
set.
196
11.3
11
(11.27)
(11.28)
Both systems are subject to the disturbance w(k) and to the constraints
x(k) X , u(k) U, w(k) W k 0.
(11.29)
(11.30)
Pre(S, W) defines the set of system (11.27) states which evolve into the target set
S in one time step for all possible disturbance w W.
Definition 11.15 For the system (11.28) we will denote the one-step robust controllable set to the set S as
Pre(S, W) , {x Rn : u U s.t. f (x, u, w) S, w W}.
(11.31)
For a system with inputs, Pre(S, W) is the set of states which can be robustly
driven into the target set S in one time step for all admissible disturbances.
Definition 11.16 For the autonomous system (11.27) we will denote the one-step
robust reachable set from the set S as
Reach(S, W) , {x Rn : x(0) S, w W such that x = fa (x(0), w)}.
Definition 11.17 For the system (11.28) with inputs we will denote the one-step
robust reachable set from the set S as
Reach(S, W) , {x Rn : x(0) S, u U, w W, such that x = f (x(0), u, w)}.
Thus, all the states contained in S are mapped into the set Reach(S, W) under the
map fa for all disturbances w W, and under the map f for all inputs u U and
for all disturbances w W.
Remark 11.7 Note that we in our notation we use the word controllable for the
Pre(S, W)) set even if the system is autonomous.
The sets Pre(S, W)) and Reach(S, W) are also denoted as one-step robust backwardreachable set and one-step robust forward-reachable set, respectively, in the literature.
11.3
197
N -step robust controllable and robust reachable sets are defined by iterating
Pre(, ) and Reach(, ) computations, respectively.
Definition 11.18 (N -Step Robust Controllable Set KN (O, W)) For a given
target set O X , the N -step robust controllable set KN (O, W) of the system
(11.27) or (11.28) subject to the constraints (11.29) is defined recursively as:
Kj (O, W) , Pre(Kj1 (O, W), W) X , K0 (O, W) = O,
j {1, . . . , N }.
From Definition 11.18, all states x0 belonging to the N -Step Robust Controllable
Set KN (O, W) can be robustly driven, through a time-varying control law, to the
target set O in N steps, while satisfying input and state constraints for all possible
disturbances.
N -step robust reachable sets are defined analogously to N -step robust controllable set.
Definition 11.19 (N -Step Robust Reachable Set RN (X0 , W)) For a given initial set X0 X , the N -step robust reachable set RN (X0 , W) of the system (11.27)
or (11.28) subject to the constraints (11.29) is defined recursively as:
Ri+1 (X0 , W) , Reach(Ri (X0 , W), W), R0 (X0 , W) = X0 ,
i = 0, . . . , N 1.
From Definition 11.19, all states x0 belonging to X0 will evolve to the N -step robust
reachable set RN (X0 , W) in N steps.
Next we will show through simple examples the main steps involved in the
computation of robust controllable and robust reachable sets for certain classes of
uncertain constrained linear systems.
11.3.1
(11.32)
(11.33)
(11.34)
The set Pre(X , W) can be obtained as described next. Since the set X is a polytope,
it can be represented as an H-polytope (Section 5.2)
X = {x : Hx h},
(11.35)
198
11
where
1
0
H=
1
0
10
0
10
1
and h = .
10
0
10
1
By using this H-presentation and the system equation (11.32), the set Pre(X , W) can
be rewritten as
Pre(X , W)
=
=
{x : Hfa (x, w) h, w W}
{x : HAx h Hw, w W} .
(11.36a)
(11.36b)
Pre(X , W) = {x Rn : HAx h}
(11.37)
(11.38)
with
wW
Pre(X , W) = x :
1
18
0
0.5
x 9 .
18
0
9
0.5
10
1
0
10
0
1
10
1
0
x
Pre(X , W) X = x :
10
0
1
1
0.5
9
1
0.5
Note that by using the definition of Pontryagin difference given in Section 5.4.8 and
affine operations on polyhedra in Section 5.4.11 we can compactly summarize the
operations in (11.36) and write the set Pre in (11.30) as
Pre(X , W)
= {x Rn : Ax + w X , w W} = {x Rn : Ax X W} =
= (X W) A.
The set
Reach(X , W) = {y : x X , w W such that y = Ax + w}
(11.39)
11.3
199
10
x2
5
0
Pre(X , W)
-5
X
-10
-10
-5
0
x1
10
is obtained by applying the map A to the set X and then considering the effect of
the disturbance w W. Let us write X in V-representation (see section 5.1)
X = conv(V ),
(11.40)
and let us map the set of vertices V through the transformation A. Because the
transformation is linear, the composition of the map A with the set X , denoted as
A X , is simply the convex hull of the transformed vertices
A X = conv(AV ).
(11.41)
We refer the reader to Section 5.4.11 for a detailed discussion on linear transformations of polyhedra. Rewrite (11.39) as
Reach(X , W) = {y Rn : z A X , w W such that y = z + w}.
We can use the definition of Minkowski sum given in Section 5.4.9 and rewrite the
Reach set as
Reach(X , W) = (A X ) W.
We can compute the Minkowski sum via projection or vertex enumeration as explained in Section 5.4.9. The set Reach(X , W) in H-representation is
4
1
0.5
16
0
1
0
x ,
Reach(X , W) = x :
0.5
16
0
1
6
1
0
11.3.2
(11.42)
200
11
)
W
ch
(X
,
Re
a
x2
10
-10
-10
0
x1
X
10
Figure 11.10 Example 11.8: one-step robust reachable set for system (11.32)
under constraints (11.34).
(11.43a)
(11.43b)
where
w(t) W = {w : 1 w 1} , t 0.
(11.44)
For the non-autonomous system (11.42), the set Pre(X , W) can be computed using
the H-presentation of X and U,
X = {x : Hx h},
U = {u : Hu u hu },
(11.45)
to obtain
Pre(X , W)
=
=
x R2 : u U s.t. Ax + Bu + w X , w W
(11.46a)
h Hw
x
HA HB
2
, wW .
x R : u R s.t.
hu
u
0
Hu
(11.46b)
x
HA HB
h
2
, (11.47)
Pre(X , W) =
x R : u R s.t.
u
0
Hu
hu
where
(11.48)
The halfspaces in (11.47) define a polytope in the state-input space, and a projection
operation (see Section 5.4.6) is used to derive the halfspaces which define Pre(X , W)
11.3
201
in the state space. The set Pre(X , W) X is depicted in Fig. 11.11 and reported
below:
9.3
1
0
9.3
1
0
1
1.5
x 9 .
1
1.5
10
0
1
10
0
1
10
x2
5
0
Pre(X , W)
-5
X
-10
-10
-5
0
x1
10
Note that by using the definition of a Minkowski sum given in Section 5.4.9
and the affine operation on polyhedra in Section 5.4.11 we can compactly write the
operations in (11.46) as follows:
Pre(X , W) =
{x :
= {x
= {x
= {x
= {x
= {x
u U s.t. Ax + Bu + w X , w W}
: y X , u U s.t. y = Ax + Bu + w, w W}
: y X , u U s.t. Ax = y + (Bu) w, w W}
: Ax C and C = X (B) U W}
: x C A, C = X (B) U W}
: x ((X W) (B U)) A} .
(11.49)
Remark 11.8 Note that in (11.49) we have used the fact that if a set S is described
as S = {v : z Z, s.t. v = z w, w W}, then S = {v : z Z, s.t. z =
v + w, w W} or S = {v : v + w Z, w W} = Z W. Also, to derive
the last equation of (11.49) we have used the associative property of the Pontryagin
difference.
202
11
40
x2
20
0
Reach(X , W)
-20
-40
-40
-20
0
x1
20
40
Figure 11.12 Example 11.9: one-step robust reachable set for system (11.42)
under constraints (11.43)-(11.44).
Table 11.2 Pre and Reach operations for uncertain linear systems subject
to polyhedral input and state constraints x(t) X , u(t) U with additive
polyhedral disturbances w(t) W
The N -step robust controllable set KN (S, W) and the N -step robust
reachable set RN (X0 , W) can be computed by using their recursive formulas and computing the Pre and Reach operations as described in Table 11.2.
11.3.3
11.3
203
W
. Then, the constraint
{w
i }ni=1
g(z, x, w) 0 w W
(11.50)
g(z, x, w
i ) 0, i = 1, . . . , nW .
(11.51)
Proof: Easily follows by the fact that the maximum of a convex function over
a compact convex set is attained at an extreme point of the set.
Lemma 11.2 shows how to reduce the number of constraints in (11.51) for a
specific class of constraint functions.
Lemma 11.2 Assume g(z, x, w) = g 1 (z, x) + g 2 (w). Then the constraint (11.50)
can be replaced by g 1 (z, x)
g, where g , g1 , . . . , gng is a vector whose i-th
component is
gi = max gi2 (w),
(11.52)
wW
and
gi2 (w)
1
0
H=
1
0
(11.55)
10
0
1
and h = 10 .
10
0
10
1
By using this H-presentation and the system equation (11.53), the set Pre(X , W) can
be rewritten as
Pre(X , W) =
x : Hfa (x, w) h, w W
(11.56)
=
{x : HA(wp )x h Hwa , wa W a , wp W p } .
(11.57)
204
11
By using Lemmas 11.1 and 11.2, the set (11.57) can be rewritten as a the polytope
h
HA(0)
x }
(11.58)
x Pre(X , W) = {x Rn :
HA(0.5)
h
with
i = min (hi Hi wa ).
h
a
a
(11.59)
w W
9
1
0
8.049
0.89
0.44
1
0
x 9 .
8.049
0.89
0.44
10
0
1
10
0
1
10
x2
5
0
Pre(X , W) X
-5
X
-10
-10
-5
0
x1
10
Figure 11.13 Example 11.10: one-step robust controllable set Pre(X , W)X
for system (11.53) under constraints (11.54).
(11.60)
x(t) X =
(11.61)
Let w = [wp ] and W = W p . The set Reach(X , W) can be written as infinite union
of reachable sets
[
Reach(X , w)
(11.62)
Reach(X , W) =
wW
where Reach(X , w)
is computed by as described in Example 11.1 for the deterministic
system A(w).
In general, the union in equation (11.62) generates a non-convex set,
as can be seen in Fig. 11.14. Non convexity of one-step reachable sets for parametric
linear systems is also discussed in [56][Sec. 6.1.2].
11.4
205
)
W
ch
(X
,
Re
a
x2
10
-10
-10
0
x1
10
11.4
x(t) O,
w(t) W, t N+ .
(11.63)
The proof of Theorem 11.2 follows the same lines of the proof of Theorem 11.1.
206
11
(11.64)
Based on condition (11.64), the following algorithm provides a procedure for computing the maximal robust positive invariant subset O for system (11.27)-(11.29)
(for reference to proofs and literature see Chapter 11.1).
Algorithm 11.4 Computation of O
input fa , X , W
output O
let 0 X
repeat
k =k+1
k+1 Pre(k , W) k
until k+1 = k
O k
Conditions for finite time termination of Algorithm 11.4 can be found in [117]. A
simple sufficient condition for finite time termination of Algorithm 11.1 requires the
system fa (x, w) to be linear and stable, and the constraint set X and disturbance
set W to be bounded and to contain the origin.
Example 11.12 Consider the second order stable system in Example 11.8
x(t + 1)
Ax(t) + w(t) =
0.5
1
0
0.5
x(t) + w(t)
(11.65)
x(t) X =
10
, t 0
10
1
, t 0.
1
(11.66)
11.4
207
The maximal robust positive invariant set of system (11.65) subject to constraints (11.66)
is depicted in Fig. 11.15 and reported below:
8.04
0.89
0.44
8.04
0.89
0.44
1
0
x 10 .
10
0
1
10
1
0
10
0
1
10
x2
5
0
-5
X
-10
-10
-5
0
x1
10
Figure 11.15 Maximal Robust Positive Invariant Set of system (11.65) subject to constraints (11.66).
208
11
The following algorithm provides a procedure for computing the maximal robust
control invariant set C for system (11.28)-(11.29).
Algorithm 11.5 Computation of C
input: f , X , U and W
output: C
let 0 X
repeat
k =k+1
k+1 Pre(k , W) k
until k+1 = k
C k+1
(11.67)
k+
The work in [48] reports examples of nonlinear systems where (11.67) can be observed. A sufficient condition for the convergence of k to C as k + requires
the polyhedral sets X , U and W to be bounded and the system f (x, u, w) to be
continuous [48].
Example 11.13 Consider the second order unstable system in example 11.9. The
maximal robust control invariant set of system (11.42) subject to constraints (11.43)(11.44) is an empty set. If the uncertain set (11.44) is replaced with
w(t) W = {w : 0.1 w 0.1} , t 0
the maximal robust control invariant set is
0
1
0
1
x
0.55
0.83
0.55
0.83
which is depicted in Fig. 11.16.
3.72
3.72
2.0
2.0
11.4
209
10
x2
5
0
-5
X
-10
-10
-5
0
x1
10
Figure 11.16 Maximal Robust Control Invariant Set of system (11.42) subject to constraints (11.43).
For all states contained in the maximal robust control invariant set C there
exists a control law, such that the system constraints are never violated for all
feasible disturbances. This does not imply that there exists a control law which
can drive the state into a user-specified target set. This issue is addressed in the
following by introducing the concept of robust controllable and stabilizable sets.
Definition 11.25 (Maximal Robust Controllable Set K (O, W)) For a given
target set O X , the maximal robust controllable set K (O, W) for the system
(11.28) subject to the constraints (11.29) is the union of all N -step robust controllable sets contained in X (N N).
Robust controllable sets KN (O, W) where the target O is a robust control invariant
set are special sets, since in addition to guaranteeing that from KN (O, W) we
robustly reach O in N steps, one can ensure that once reached O, the system can
stay there at all future time instants and for all possible disturbance realizations.
Definition 11.26 (N -step (Maximal) Robust Stabilizable Set) For a given
robust control invariant set O X , the N -step (maximal) robust stabilizable set
of the system (11.28) subject to the constraints (11.29) is the N -step (maximal)
robust controllable set KN (O, W) (K (O, W)).
The set K (O, W) contains all states which can be robustly steered into the
robust control invariant set O and hence K (O, W) C . The set K (O, W)
C can be computed as follows:
Algorithm 11.6 The Maximal Robust Stabilizable Set K (O, W)
let K0 O, where O is a robust control invariant set
repeat
c=c+1
T
Kc+1 Pre(Kc , W) X
until Kc+1 = Kc ,
K (O, W) = Kc ,
210
11
Chapter 12
12.1
Problem Formulation
Consider the linear time-invariant system
x(t + 1) = Ax(t) + Bu(t),
(12.1)
where x(t) Rn , u(t) Rm are the state and input vectors, respectively, subject
to the constraints
x(t) X , u(t) U, t 0.
(12.2)
The sets X Rn and U Rm are polyhedra.
Remark 12.1 The results of this chapter also hold for more general forms of linear
constraints such as mixed input and state constraints
[x(t) , u(t) ] Px,u ,
(12.3)
212
12
where Px,u is a polyhedron in Rn+m or mixed input and state constraints over a finite
time
[x(0) , . . . , x(N 1) , u(0) , . . . , u(N 1) ] Px,u,N ,
(12.4)
where Px,u,N is a polyhedron in RN(n+m) . Note that constraints of the type (12.4)
can arise, for example, from constraints on the input rate u(t) , u(t) u(t 1). In
this chapter, for the sake of simplicity, we will use the less general form (12.2).
N
1
X
q(xk , uk ),
(12.5)
k=0
where xk denotes the state vector at time k obtained by starting from the state
x0 = x(0) and applying to the system model
xk+1 = Axk + Buk
(12.6)
N
1
X
k=0
kQxk kp + kRuk kp .
(12.7)
If the squared Euclidian norm is used in the cost function (12.5), then we set
p(xN ) = xN P xN and q(xk , uk ) = xk Qxk + uk Ruk with P 0, Q 0 and R 0.
Cost (12.5) is rewritten as
J0 (x(0), U0 ) , xN P xN +
N
1
X
xk Qxk + uk Ruk .
(12.8)
k=0
J0 (x(0), U0 )
xk+1 = Axk + Buk , k = 0, . . . , N 1
xk X , uk U, k = 0, . . . , N 1
xN Xf
x0 = x(0),
(12.9)
12.1
Problem Formulation
213
Remark 12.2 Note that we distinguish between the current state x(k) of system (12.1)
at time k and the variable xk in the optimization problem (12.9), that is the predicted
state of system (12.1) at time k obtained by starting from the state x0 = x(0) and
applying to system (12.6) the input sequence u0 , . . . , uk1 . Analogously, u(k) is the
input applied to system (12.1) at time k while uk is the k-th optimization variable of
the optimization problem (12.9).
If we use cost (12.8) with the squared Euclidian norm and set
{(x, u) Rn+m : x X , u U} = Rn+m , Xf = Rn ,
(12.11)
problem (12.9) becomes the standard unconstrained finite time optimal control
problem (Chapter 9) whose solution (under standard assumptions on A, B, P , Q
and R) can be expressed through the time varying state feedback control law (9.27)
u (k) = Fk x(k)
k = 0, . . . , N 1.
(12.12)
(12.13)
(12.14)
In the following chapters we will show that the solution to Problem (12.9) can
again be expressed in feedback form where now u (k) is a continuous piecewise
affine function on polyhedra of the state x(k), i.e., u (k) = fk (x(k)) where
fk (x) = Fkj x + gkj
if
(12.15)
Matrices Hkj and Kkj in equation (12.15) describe the j-th polyhedron CRkj = {x
Rn |Hkj x Kkj } inside which the feedback optimal control law u (k) at time k has
the affine form Fkj x + gkj . The set of polyhedra CRkj , j = 1, . . . , Nkr is a polyhedral
partition of the set of feasible states Xk of Problem (12.9) at time k. The sets
Xk are discussed in detail in the next section. Since the functions fk (x(k)) are
continuous, the use of polyhedral partitions rather than strict polyhedral partitions
(Definition 5.5) will not cause any problem, indeed it will simplify the exposition.
In the rest of this chapter we will characterize the structure of the value function
and describe how the optimal control law can be efficiently computed by means of
multiparametric linear and quadratic programming. We will distinguish the cases
1- or -norm and squared 2-norm.
214
12.2
12
Feasible Solutions
We denote with Xi the set of states xi at time i for which (12.5)(12.9) is feasible,
for i = 0, . . . , N . The sets Xi for i = 0, . . . , N play an important role in the
the solution of (12.9). They are independent of the cost function (as long as it
guarantees the existence of a minimum) and of the algorithm used to compute the
solution to problem (12.9). There are two ways to rigourously define and compute
the sets Xi : the batch approach and the recursive approach. In the batch approach
Xi =
Au
0
... 0
0
A
... 0
u
..
..
..
..
.
.
.
.
0
0
.
.
.
A
u
0
... 0
Gi =
0
Ax B
0
... 0
Ax AB
A
B
... 0
x
.
..
..
.
..
..
.
.
Af AN i1 B Af AN i2 B . . . Af B
0
0
..
.
(12.17)
bu
bu
..
.
bu
0
, wi = bx
Ax
bx
Ax A
bx
Ax A2
.
..
..
.
N i
bf
Af A
(12.18)
The set Xi is a polyhedron as it is the projection of the polyhedron Pi in (12.17)(12.18) on the xi space.
In the recursive approach,
Xi
XN
, Ei =
i = 0, . . . , N 1
Xf .
(12.19)
The definition of Xi in (12.19) is recursive and requires that for any feasible initial
state xi Xi there exists a feasible input ui which keeps the next state Axi + Bui
in the feasible set Xi+1 . It can be compactly written as
Xi = Pre(Xi+1 ) X .
(12.20)
12.2
Feasible Solutions
215
Initializing XN to Xf and solving (12.19) backward in time yields the same sets
Xi as the batch approach. This recursive method, however, leads to an alternative
approach for computing the sets Xi . Let Xi be the H-polyhedra AXi x bXi . Then
the set Xi1 is the projection of the following polyhedron
Au
0
bu
0 ui + Ax xi bx
(12.21)
AXi B
AXi A
bXi
on the xi space.
Consider problem (12.9). The set X0 is the set of all initial states x0 for
which (12.9) is feasible. The sets Xi with i = 1, . . . , N 1 are hidden. A
0 = [
given U
u0 , . . . , uN 1 ] is feasible for problem (12.9) if and only if at all time
instants i, the state xi obtained by applying u
0 , . . . , ui1 to the system model
xk+1 = Axk + Buk with initial state x0 X0 belongs to Xi . Also, Xi is the set of
feasible initial states for problem
PN 1
p(xN ) + k=i q(xk , uk )
Ji (x(0)) = minUi
subj. to xk+1 = Axk + Buk , k = i, . . . , N 1
(12.22)
xk X , uk U, k = i, . . . , N 1
xN Xf .
Next, we provide more insights into the set Xi by using the invariant set theory
of Section 11.2. We consider two cases: (1) Xf = X which corresponds to effectively
removing the terminal constraint set and (2) Xf chosen to be a control invariant
set.
Theorem 12.1 [165, Theorem 5.3]. Let the terminal constraint set Xf be equal
to X . Then,
1. The feasible set Xi , i = 0, . . . , N 1 is equal to the (N i)-step controllable
set:
Xi = KN i (X ).
2. The feasible set Xi , i = 0, . . . , N 1 contains the maximal control invariant
set:
C Xi .
3. The feasible set Xi is control invariant if and only if the maximal control
invariant set is finitely determined and N i is equal to or greater than its
, i.e.
determinedness index N
.
Xi P re(Xi ) C = KN i (X ) for all i N N
4. Xi Xj if i < j for i = 0, . . . , N 1. The size of the feasible set Xi stops
decreasing (with decreasing i) if and only if the maximal control invariant set
is finitely determined and N i is larger than its determinedness index, i.e.
< i < j < N.
Xi Xj if N N
Furthermore,
.
Xi = C if i N N
216
12
Theorem 12.2 [165, Theorem 5.4]. Let the terminal constraint set Xf be a control
invariant subset of X . Then,
1. The feasible set Xi , i = 0, . . . , N 1 is equal to the (N i)-step stabilizable
set:
Xi = KN i (Xf ).
2. The feasible set Xi , i = 0, . . . , N 1 is control invariant and contained within
the maximal control invariant set:
Xi C .
3. Xi Xj if i < j, i = 0, . . . , N 1. The size of the feasible Xi set stops
increasing (with decreasing i) if and only if the maximal stabilizable set is
finitely determined and N i is larger than its determinedness index, i.e.
< i < j < N.
Xi Xj if N N
Furthermore,
.
Xi = K (Xf ) if i N N
Remark 12.3 Theorems 12.1 and 12.2 help us understand how the feasible sets Xi
propagate backward in time as a function of the terminal set Xf . In particular,
when Xf = X the set Xi shrinks as i becomes smaller and stops shrinking when it
becomes the maximal control invariant set. Also, depending on i, either it is not a
control invariant set or it is the maximal control invariant set. We have the opposite
if a control invariant set is chosen as terminal constraint Xf . The set Xi grows as
i becomes smaller and stops growing when it becomes the maximal stabilizable set.
Both cases are shown in the Example 12.1 below.
0
1 1
u(t)
x(t) +
x(t + 1) =
1
0 1
y(t) =
1 0 x(t)
(12.23)
(12.24)
5
5
for all k 0.
x(k)
5
5
(12.25)
We compute the feasible sets Xi and plot them in Figure 12.1 in two cases.
12.2
Feasible Solutions
217
0.32132
0.32132
1
1
0.3806
0.94697
0.94697
x 0.3806 .
2.5
0
2.5
0
(12.26)
0.44721
0.24254
0.31623
0.24254
0.31623
0.44721
1
1
0.70711
0.70711
0.89443
0.97014
0.94868
0.97014
0.94868
0.89443
0
0
0.70711
0.70711
2.6833
2.6679
2.5298
2.6679
2.5298
2.6833
5
5
3.5355
3.5355
(12.27)
Note that in this case C = K (Xf ) and the determinedness index is six.
2
x2
x2
-2
-2
-4
-4
-6-6
-4
-2
0
x1
-6-6
-4
-2
0
x1
(b) Case 2: Xf = X .
218
12.3
12
N
1
X
minU0
J0 (x(0), U0 ) = xN P xN +
xk Qxk + uk Ruk
subj. to
k=0
12.3.1
(12.28)
min
min
U0
U0
subj. to
G0 U0 w0 + E0 x(0),
(12.29)
with G0 , w0 and E0 defined in (12.18) for i = 0 and H, F , Y defined in (9.8). As
F 0. Note that the optimizer
J0 (x(0), U0 ) 0 by definition it follows that H
F Y
U0 is independent of the term involving Y in (12.29).
We view x(0) as a vector of parameters and our goal is to solve (12.29) for all
values of x(0) X0 and to make this dependence explicit. The computation of
the set X0 of initial states for which problem (12.29) is feasible was discussed in
Section 12.2.
Before proceeding further, it is convenient to define
z , U0 + H 1 F x(0),
(12.30)
12.3
219
Theorem 7.7 states that the solution z (x(0)) of the mp-QP problem (12.31) is
a continuous and piecewise affine function on polyhedra of x(0). Clearly the same
properties are inherited by the controller. The following corollaries of Theorem 7.7
establish the analytical properties of the optimal control law and of the value
function.
Corollary 12.1 The control law u (0) = f0 (x(0)), f0 : Rn Rm , obtained as a
solution of the CFTOC (12.28) is continuous and piecewise affine on polyhedra
f0 (x) = F0j x + g0j
if
x CR0j , j = 1, . . . , N0r ,
(12.32)
where the polyhedral sets CR0j = {x Rn |H0j x K0j }, j = 1, . . . , N0r are a partition
of the feasible polyhedron X0 .
Proof: From (12.30) U0 (x(0)) = z (x(0)) H 1 F x(0). From Theorem 7.7
we know that z (x(0)), solution of (12.31), is PPWA and continuous. As U0 (x(0))
is a linear combination of a linear function and a PPWA function, it is PPWA. As
U0 (x(0)) is a linear combination of two continuous functions it is continuous. In
particular, these properties hold for the first component u (0) of U0 .
Remark 12.5 Note that, as discussed in Remark 7.9, the critical regions defined
in (7.4) are in general sets that are neither closed nor open. In Corollary 12.1 the
polyhedron CR0i describes the closure of a critical region. The function f0 (x) is
continuous and therefore it is simpler to use a polyhedral partition rather than a
strict polyhedral partition.
Corollary 12.2 The value function J0 (x(0)) obtained as solution of the CFTOC (12.28)
is convex and piecewise quadratic on polyhedra. Moreover, if the mp-QP problem (12.31) is not degenerate, then the value function J0 (x(0)) is C (1) .
F 0,
Proof: By Theorem 7.7 J (x(0)) is a a convex function of x(0). As H
F Y
its Schur complement1 Y F H 1 F 0, and therefore J0 (x(0)) = J (x(0)) +
x(0) (Y F H 1 F )x(0) is a convex function, because it is the sum of convex
functions. If the mp-QP problem (12.31) is not degenerate, then Theorem 7.9
implies that J (x(0)) is a C (1) function of x(0) and therefore J0 (x(0)) is a C (1)
function of x(0). The results of Corollary 12.1 imply that J0 (x(0)) is piecewise
quadratic.
Remark 12.6 The relation between the design parameters of the optimal control
problem (12.28) and the degeneracy of the mp-QP problem (12.31) is complex, in
general.
The solution of the multiparametric problem (12.31) provides the state feedback
solution u (k) = fk (x(k)) of CFTOC (12.28) for k = 0 and it also provides the
A
B
C BA1 B 0.
1 Let
X =
B
C
220
12
open-loop optimal control laws u (k) as function of the initial state, i.e., u (k) =
u (k, x(0)). The state feedback PPWA optimal controllers u (k) = fk (x(k)) with
fk : Xk 7 U for k = 1, . . . , N are computed in the following way. Consider the
same CFTOC (12.28) over the shortened time-horizon [i, N ]
N
1
X
minUi
xN P xN
subj. to
xk Qxk + uk Ruk
k=i
(12.33)
min
(12.34)
subj. to
Hi
Gi Ui wi + Ei x(i),
where Hi =
0, Fi , Yi are appropriately defined for each i and Gi , wi , Ei are
defined in (12.18). The first component of the multiparametric solution of (12.34)
has the form
ui (x(i)) = fi (x(i)), x(i) Xi ,
(12.35)
where the control law fi : Rn Rm , is continuous and PPWA
fi (x) = Fij x + gij
if
CRij
x CRij , j = 1, . . . , Nir ,
n
|Hij x
Kij },
(12.36)
1, . . . , Nir
j =
are a
partition of the feasible polyhedron Xi . Therefore the feedback solution u (k) =
fk (x(k)), k = 0, . . . , N 1 of the CFTOC (12.28) is obtained by solving N mp-QP
problems of decreasing size. The following corollary summarizes the final result.
Corollary 12.3 The state-feedback control law u (k) = fk (x(k)), fk : Xk Rn
U Rm , obtained as a solution of the CFTOC (12.28) and k = 0, . . . , N 1 is
time-varying, continuous and piecewise affine on polyhedra
fk (x) = Fkj x + gkj
CRkj
if
x CRkj , j = 1, . . . , Nkr ,
12.3.2
Hkj x
Kkj },
j =
(12.37)
1, . . . , Nkr
are a
subj. to
xj X , uj U,
Axj + Buj Xj+1
(12.38)
12.3
221
JN
(xN ) = xN P xN
(12.39)
XN = Xf ,
(12.40)
where Xj denotes the set of states x for which the CFTOC (12.28) is feasible at time
12.3.3
J
(x(0)) = minu0 ,u1 ,...
xk Qxk + uk Ruk
k=0
subj. to
(12.41)
(12.42)
J
(x(0)) would be infinite). For this reason, the set X in (12.42) is the maximal
stabilizable set K (O) of system (12.1) subject to the constraints (12.2) with O
being the origin (Definition 11.13).
If the initial state x0 = x(0) is sufficiently close to the origin, then the constraints will never become active and the solution of Problem (12.41) will yield the
unconstrained LQR (9.32). More formally we can define a corresponding invariant
set around the origin.
LQR
Definition 12.1 (Maximal LQR Invariant Set O
) Consider the system x(k+
LQR
n
1) = Ax(k) + Bu(k). O R denotes the maximal positively invariant set for
the autonomous constrained linear system:
222
12
where u(k) = F x(k) is the unconstrained LQR control law (9.32) obtained from
the solution of the ARE (9.31).
(x0 ), deTherefore, from the previous discussion, there is some finite time N
LQR
pending on the initial state x0 , at which the state enters O and after which the
). This consystem evolves in an unconstrained manner (xk X , uk U, k > N
sideration allows us to split Problem (12.41) into two parts by using the dynamic
where the constraints may be active
programming principle, one up to time k = N
and one for longer times k > N where there are no constraints.
J
(x(0))
1
N
X
xk Qxk + uk Ruk + JN
)
(xN
k=0
subj. to
1
xk X , uk U, k = 0, . . . , N
xk+1 = Axk + Buk , k 0
x0 = x(0),
(12.43)
where
JN
) =
(xN
k=N
subj. to
=
xk Qxk + uk Ruk
(12.44)
xN P xN .
This key insight due to Sznaier and Damborg [263] is formulated precisely in the
following.
Theorem 12.3 (Equality of Finite and Infinite Optimal Control, [250]) For
any given initial state x(0), the solution to (12.43, 12.44) is equal to the infinitetime solution of (12.41), if the terminal state xN of (12.43) lies in the positive
LQR
invariant set O
and no terminal set constraint is applied in (12.43), i.e. the
LQR
steps.
state voluntarily enters the set O
after N
Theorem 12.3 suggests that we can obtain the infinite horizon constrained linear
quadratic regulator CLQR by solving the finite horizon problem for a horizon of N
with a terminal weight of P = P and no terminal constraint. The critical question
(x0 ) or at least an upper bound was studied by several
of how to determine N
researchers. Chmielewski and Manousiouthakis [81] presented an approach that
(x0 ) for all x0 belonging
provides a conservative estimate Nest of the finite horizon N
S (x0 ), x0 S).
to a compact set of initial conditions S X = K (0) (Nest N
They solve a single, finite dimensional, convex program to obtain Nest . Their
estimate can be used to compute the PWA solution of (12.43) for a particular
set S. Alternatively, the quadratic program with horizon Nest can be solved to
determine u0 , u1 , . . . , uNest for a particular x(0) S. For a given initial state
x(0), rather then a set S, Scokaert and Rawlings [250] presented an algorithm that
(x(0)) iteratively. In summary we can state the following
attempts to identify N
Theorem.
12.3
223
Theorem 12.4 (Explicit solution of CLQR) Assume that (A, B) is a stabilizable pair and (Q1/2 , A) is an observable pair, R 0. The state-feedback solution to the CLQR problem (12.41) in a compact set of the initial conditions
S X = K (0) is time-invariant, continuous and piecewise affine on polyhedra
u (k) = f (x(k)), f (x) = F j x + g j
if
j
r
x CR
, j = 1, . . . , N
, (12.45)
j
r
where the polyhedral sets CR
= {x Rn : H j x K j }, j = 1, . . . , N
are a
finite partition of the feasible compact polyhedron S X .
12.3.4
CLQR Algorithm
In this section we will sketch an efficient algorithm to compute the PWA solution
to the CLQR problem in (12.41) for a given set S of initial conditions. Details
S ,
are available in [127, 128]. As a side product, the algorithm also computes N
the shortest horizon N for which the problem (12.43), (12.44) is equivalent to the
infinite horizon problem (12.41).
The idea is as follows. For the CFTOC problem (12.28) with a horizon N with
no terminal constraint (Xf = Rn ) and terminal cost P = P , where P is the
solution to the ARE (9.31), we solve an mp-QP and obtain the PWA control law.
From Theorem 12.3 we can conclude that for all states which enter the invariant set
LQR
O
introduced in Definition 12.1 with the computed control law in N steps, the
infinite-horizon problem has been solved. For these states, which we can identify
via a reachability analysis, the computed feedback law is infinite-horizon optimal.
In more detail, we start the procedure by computing the Maximal LQR InvariLQR
LQR
= {x
ant Set O
introduced in Definition 12.1, the polyhedron P0 , O
n
LQR
R |H0 x K0 }. Figure 12.2(a) depicts O . Then, the algorithm finds a point
LQR
x
by stepping over a facet of O
with a small step , as described in [17]. If
(12.28) is feasible for horizon N = 1 (terminal set constraint Xf = Rn , terminal
cost P = P and x(0) = x
), the active constraints will define the neighboring
polyhedron P1 = {x Rn |H1 x K1 } (
x P1 , see Figure 12.2(b)) [39]. By Theorem 12.3, the finite time optimal solution computed above equals the infinite-time
LQR
optimal solution if x1 O
. Therefore we extract from P1 the set of points that
LQR
will enter O in N = 1 time-steps, provided that the optimal control law associated with P1 (i.e., U1 = F1 x(0) + G1 ) is applied. The Infinite-Time Polyhedron
(IT P 11 ) is therefore defined by the intersection of the following two polyhedra:
LQR
x1 O
, x1 = Ax0 + BU1 ,
x0 P1 .
(12.46a)
(12.46b)
Equation (12.46a) is the reachability constraint and (12.46b) defines the set of
states for which the computed feedback law is feasible and optimal over N = 1
224
12
2
Step over facet
LQR
O
-1
P1
1
x2
x2
LQR
O
-1
-2
-3
-3
-2
-2
-1
0
x1
-3
-3
-2
-1
0
x1
LQR
region O
and step over facet
with step-size .
3
2
Reachability subset IT P 11
x2
1
0
LQR
O
-1
-2
-3
-3
-2
-1
0
x1
of P1 to obtain IT P 11 .
Figure 12.2 CLQR Algorithm: Region Exploration.
steps (see [39] for details). The intersection is the set of points for which the
control law is infinite-time optimal.
A general step r of the algorithm involves stepping over a facet to a new point x
and determining the polyhedron Pr and the associated control law (UN
= Fr x(0)+
Gr ) from (12.28) with horizon N. Then we extract from Pr the set of points that
LQR
will enter O
in N time-steps, provided that the optimal control law associated
with Pr is applied. The Infinite-Time Polyhedron (IT P N
r ) is therefore defined by
the intersection of the following two polyhedra:
LQR
xN O
x0 Pr .
(12.47a)
(12.47b)
This intersection is the set of points for which the control law is infinite-time
optimal. Note that, as for x1 in the one-step case, xN in (12.47a) can be described
12.3
225
r=0,...,R
12.3.5
Examples
Example 12.2 Consider the double integrator (12.23). We want to compute the
state feedback optimal controller that solves problem (12.28) with N = 6, Q = [ 10 01 ],
R = 0.1, P is equal to the solution of the Riccati equation (9.31), Xf = R2 . The
input constraints are
1 u(k) 1, k = 0, . . . , 5
(12.48)
10
10
, k = 0, . . . , 5.
x(k)
10
10
(12.49)
This task is addressed as shown in Section (12.3.1). The feedback optimal solution
u (0), . . . , u (5) is computed by solving six mp-QP problems and the corresponding
polyhedral partitions of the state-space are depicted in Fig. 12.3. Only the last two
optimal control moves are reported below:
u (5) =
[ 0.58
1.00
1.00
1.55 ] x(5)
if
0.35
0.94
0.35 0.94
1.00 0.00
1.00 0.00
0.94 #
1.00
0.71 x(5)
0.00
0.00
if
"
if
0.35
0.00
0.71
1.00
1.00
1.00
0.00
1.00
0.71
x(5)
0.00
1.00
0.00
0.71
0.61
0.61
10.00
10.00
(Region #1)
" 0.61 #
(Region #2)
x(5)
" 0.61 #
(Region #3)
10.00
7.07
10.00
10.00
10.00
10.00
10.00
7.07
226
12
[ 0.58
1.00
1.00
u (4) =
1.00
1.00
[ 0.44
[ 0.44
1.55 ] x(4)
if
if
if
if
if
1.43 ] x(4)
0.46
if
1.43 ] x(4)
0.46
if
0.35
0.94
0.35 0.94
0.77 0.64
0.77 0.64
0.29
0.00
0.71
0.45
1.00
1.00
x(4)
0.96
1.00
0.71
x(4)
0.89
0.00
0.00
0.29
0.35
0.00
0.71
0.45
1.00
1.00
0.96
0.94
1.00
0.71 x(4)
0.89
0.00
0.00
1.00
0.00
1.00
0.71
0.45
0.96
0.00
1.00
x(4)
0.00
0.71
0.89
0.29
0.29
0.35
1.00
0.00
1.00
0.71
0.45
0.29
0.96
0.94
0.00
1.00 x(4)
0.00
0.71
0.89
0.96
0.29 0.96
0.77 0.64
1.00 0.00
0.29
0.96
0.29 0.96
0.77 0.64
1.00 0.00
0.61
0.61
2.43
2.43
(Region #1)
0.98
10.00
7.07
4.92
10.00
10.00
0.37
(Region #2)
0.61
10.00
7.07
4.92
10.00
10.00
0.98
(Region #3)
10.00
10.00
10.00
(Region #4)
7.07
4.92
0.37
0.61
10.00
10.00
x(4)
x(4)
10.00
7.07
4.92
0.98
0.37
2.43
10.00
0.37
0.98
2.43
10.00
(Region #5)
(Region #6)
(Region #7)
Note that by increasing the horizon N , the control law changes only far away from the
origin, the larger N the more in the periphery. This must be expected from the results
of Section 12.3.3. The control law does not change anymore with increasing N in the
set where the CFTOC law becomes equal to the constrained infinite-horizon linear
quadratic regulator (CLQR) problem. This set gets larger as N increases [81, 250].
Example 12.3 The infinite-time CLQR (12.41) was determined for Example 12.2
S is 12. The
by using the approach presented in Section 12.3.3. The resulting N
state-space is divided into 117 polyhedral regions and is depicted in Fig. 12.4(a). In
Fig. 12.4(b) the same control law is represented where polyhedra with the same affine
control law were merged.
12.4
12.4
J0 (x(0)) =
minU0
subj. to
J0 (x(0), U0 ) = kP xN kp +
N
1
X
k=0
227
kQxk kp + kRuk kp
12.4.1
Batch Approach
The optimal control problem (12.50) with p = can be rewritten as a linear program by using the approach presented in Section 10.1. Therefore, problem (12.50)
can be reformulated as the following LP problem
228
12
min
z0
subj. to
x0 + . . . + xN + u0 + . . . + uN 1
1n xk
Q A x0 +
k1
X
j=0
1r xN P AN x0 +
1m uk Ruk ,
Ak x0 +
k1
X
j=0
AN x0 +
j=0
A Buk1j ,
N
1
X
j=0
Aj BuN 1j ,
Aj Buk1j X , uk U,
N
1
X
(12.51a)
Aj BuN 1j Xf ,
k = 0, . . . , N 1
x0 = x(0),
(12.51b)
(12.51c)
(12.51d)
(12.51e)
(12.51f)
(12.51g)
where [Gx , Gu , Gc ] is the block partition of the matrix G into the three parts
corresponding to the variables xi , ui and ui , respectively. The vector c0 and the
submatrices G , w , S associated with the constraints (12.51b)-(12.51d) are defined
in (10.10). The matrices G0 , w0 and E0 are defined in (12.18) for i = 0.
As in the 2-norm case, by treating x(0) as a vector of parameters, Problem (12.52)
becomes a multiparametric linear program (mp-LP) that can be solved as described
in Section 7.2. Once the multiparametric problem (12.51) has been solved, the
explicit solution z0 (x(0)) of (12.52) is available as a piecewise affine function of
x(0), and the optimal control law u (0) is also available explicitly, as the optimal
input u (0) consists simply of m components of z0 (x(0))
u (0) = [0 . . . 0 Im 0 . . . 0]z0 (x(0)).
(12.54)
Theorem 7.5 states that there always exists a continuous PPWA solution z0 (x)
of the mp-LP problem (12.52). Clearly the same properties are inherited by the
controller. The following Corollaries of Theorem 7.5 summarize the analytical
properties of the optimal control law and of the value function.
12.4
229
if
x CR0j , j = 1, . . . , N0r ,
(12.55)
where the polyhedral sets CR0j , {H0j x k0j }, j = 1, . . . , N0r , are a partition of the
feasible set X0 .
Corollary 12.5 The value function J (x) obtained as a solution of the CFTOC (12.50)
is convex and PPWA.
Remark 12.7 Note that if the optimizer of problem (12.50) is unique for all x(0) X0 ,
then Corollary 12.4 reads: The control law u (0) = f0 (x(0)), f0 : Rn Rm ,
obtained as a solution of the CFTOC (12.50) with p = 1 or p = , is continuous
and PPWA,. . .. From the results of Section 7.2 we know that in case of multiple
optimizers for some x(0) X0 , a continuous control law of the form (12.55) can
always be computed.
The multiparametric solution of (12.52) provides the open-loop optimal sequence u (0), . . . , u (N 1) as an affine function of the initial state x(0). The
state feedback PPWA optimal controllers u (k) = fk (x(k)) with fk : Xk 7 U for
k = 1, . . . , N are computed in the following way. Consider the same CFTOC (12.50)
over the shortened time horizon [i, N ]
minUi
subj. to
kP xN kp +
N
1
X
k=i
kQxk kp + kRuk kp
(12.56)
min
zi
(12.57)
i , Si , w
i , are approwhere zi , {xi , . . . , xN , ui , . . . , uN 1 , ui , . . . , uN 1 } and ci , G
(12.58)
230
12
if x CRij , j = 1, . . . , Nir
(12.59)
12.4.2
Recursive Approach
Consider the dynamic programming formulation of (12.50) with J0 () defined by (12.7)
with p = 1 or p =
Jj (xj ) ,
(12.61)
JN
(xN ) = kP xN kp
XN = Xf ,
(12.62)
(12.63)
Unlike for the 2-norm case the dynamic program (12.61)-(12.63) can be solved as
explained in the next theorem.
Theorem 12.6 The state feedback piecewise affine solution (12.60) of the CFTOC (12.50)
for p = 1 or p = is obtained by solving the optimization problem (12.61)-(12.63)
via N mp-LPs.
Proof: Consider the first step j = N 1 of dynamic programming (12.61)(12.63)
kQxN 1 kp + kRuN 1 kp + JN
(AxN 1 + BuN 1 )
xN 1 X , uN 1 U
AxN 1 + BuN 1 Xf .
(12.64)
JN
(x
),
u
(x
)
and
X
are
computable
via
the
mp-LP:
N 1
1 N 1
N 1 N 1
JN
1 (xN 1 ) ,
minuN 1
subj. to
12.4
JN
1 (xN 1 ) ,
231
By Theorem 7.5, JN
1 is a convex and piecewise affine function of xN 1 , the
corresponding optimizer uN 1 is piecewise affine and continuous, and the feasible
kQxN 2 kp + kRuN 2 kp + JN
1 (AxN 2 + BuN 2 )
xN 2 X , uN 2 U
AxN 2 + BuN 2 XN 1 .
(12.67)
Since JN
(x)
is
a
convex
and
piecewise
affine
function
of
x,
the
problem
(12.67)
1
can be recast as the following mp-LP (see Section 3.2.5 for details)
JN
2 (xN 2 ) ,
minuN 2
subj. to
JN
2 (xN 2 ) ,
min,uN 2
subj. to
JN
(x
),
u
(x
)
and
X
are
computed
by
solving
the
mp-LP
(12.68).
N 2
2 N 2
N 2 N 2
By Theorem 7.5, JN
2 is a convex and piecewise affine function of xN 2 , the
corresponding optimizer uN 2 is piecewise affine and continuous, and the feasible
set XN 2 is a convex polyhedron.
The convexity and piecewise linearity of Jj and the polyhedra representation
of Xj still hold for j = N 3, . . . , 0 and the procedure can be iterated backwards
in time, proving the theorem.
Consider the state feedback piecewise affine solution (12.60) of the CFTOC (12.50)
for p = 1 or p = and assume we are interested only in the optimal controller at
232
12
time 0. In this case, by using duality arguments we can solve the equations (12.61)(12.63) by using vertex enumerations and one mp-LP. This is proven in the next
theorem.
Theorem 12.7 The state feedback piecewise affine solution (12.60) at time k = 0
of the CFTOC (12.50) for p = 1 or p = is obtained by solving the optimization
problem (12.61)-(12.63) via one mp-LP.
Proof: Consider the first step j = N 1 of dynamic programming (12.61)(12.63)
JN
1 (xN 1 ) ,
kQxN 1 kp + kRuN 1 kp + JN
(AxN 1 + BuN 1 )
xN 1 X , uN 1 U
AxN 1 + BuN 1 Xf
(12.69)
minuN 1
subj. to
JN
1 (xN 1 ) ,
By Theorem 7.5, JN
is
a
convex
and
piecewise
affine
function
of
x
,
and
the
N
1
1
min
zN 1
subj. to
cN 1 zN 1
N 1 zN 1 w
G
N 1 + SN 1 xN 1 ,
(12.71)
subj. to G
(12.72)
N 1 v = cN 1
v 0.
JN
N 1 + SN 1 xN 1 ) Vi }
1 (xN 1 ) = max {(w
i=1,...,k
i.e.,
JN
N
1 (xN 1 ) = max {(Vi SN 1 )xN 1 w
1 Vi }.
i=1,...,k
Recall that if the dual of a mp-LP is unbounded, then the primal is infeasible
(Theorem 7.3). For this reason the feasible set XN 1 is obtained by requiring that
the cost of (12.72) does not increase in the direction of the rays:
XN 1 = {xN 1 : (w
N 1 + SN 1 xN 1 ) yi 0, i = 1, . . . , e}
12.4
233
with JN
1 (xN 1 ) and XN 1 available, we can iterate the procedure backwards in
time for steps N 2, N 3, . . . , 1. At step 0 one mp-LP will be required in order
to compute u0 (x(0)).
12.4.3
Example
Example 12.4 Consider the double integrator system (12.23). We want to compute
the
controller that solves (12.50) with p = , N = 6, P =
optimal
state feedback
1 0
1 0
, R = 0.8, subject to the input constraints
,Q=
0 1
0 1
U = {u R : 1 u 1}
and the state constraints
X = {x R2 :
10
10
10
10
(12.73)
(12.74)
and Xf = X . The optimal feedback solution u (0), . . . , u (5) was computed by solving
six mp-LP problems and the corresponding polyhedral partitions of the state-space
are depicted in Fig. 12.5, where polyhedra with the same control law were merged.
Only the last optimal control move is reported below:
0.45 0.89
0.00
1.00 0.00
0.00
0
(Region #1)
if
x(5)
0.71
0.71
7.07
1.00 0.00
10.00
0.00
0.45 0.89
1.00 0.00
0.00
(Region #2)
x(5)
0
if
7.07
0.71 0.71
10.00
1.00 0.00
0.45 0.89
0.00
0.45 0.89
0.45
(Region #3)
[
if
x(5)
0.71 0.71
0.00
1.00 0.00
10.00
0.45
0.45 0.89
(Region #4)
x(5)
0.71
0.71
0.00
10.00
1.00 0.00
0.00
0.71 0.71
u (5) =
1.00 0.00
1.00
[
if
0.00
1.00 0.00
10.00
0.00
1.00
0.71 0.71
0.00
1.00 0.00
0.00
1.00
0.00
(Region #6)
[
]
x(5)
if
x(5)
1.00 0.00
1.00
0.00 1.00
10.00
0.45
0.45 0.89
1.00 0.00
1.00
1.00
x(5)
(Region #7)
if
0.00
1.00
10.00
1.00 0.00
10.00
0.45
0.45 0.89
1.00 0.00
1.00
1.00
x(5)
(Region #8)
if
1.00 0.00
10.00
0.00
1.00
10.00
(12.75)
Note that the controller (12.75) is piecewise linear around the origin. In fact, the
origin belongs to multiple regions (1 to 6). Note that the number Nir of regions is not
always increasing with decreasing i (N5r = 8, N4r = 12, N3r = 12, N2r = 26, N1r = 28,
234
12
N0r = 26). This is due to the merging procedure, before merging we have N5r = 12,
N4r = 22, N3r = 40, N2r = 72, N1r = 108, N0r = 152.
12.4.4
Infinite-Time Solution
Assume that Q and R have full column rank and that the constraint sets X and U
contain the origin in their interior. Consider the following infinite-horizon problem
with constraints
J
(x(0)) = minu0 ,u1 ,...
kQxk kp + kRuk kp
k=0
subj. to
(12.76)
(12.77)
uk
set then J
(x(0)) would be infinite.) Furthermore, if the initial state x0 = x(0)
is sufficiently close to the origin, then the state and input constraints will never
become active and the solution of Problem (12.41) will yield the unconstrained
optimal controller (10.31).
The discussion for the solution of the infinite horizon constrained linear quadratic
regulator (Section 12.3.3) by means of the batch approach can be repeated here
with one precaution. Since the unconstrained optimal controller (if it exists) is
PPWA the computation of the Maximal Invariant Set for the autonomous constrained piecewise linear system is more involved and requires algorithms which
will be presented later in Chapter 18.
Differently from the 2-norm case, here the use of dynamic programming for
computing the infinite horizon solution is a viable alternative to the batch approach.
Convergence conditions for the dynamic programming strategy and convergence
guarantees for the resulting possibly discontinuous closed-loop system are given
in [83]. A computationally efficient algorithm to obtain the infinite-time optimal
solution, based on a dynamic programming exploration strategy with a multiparametric linear programming solver and basic polyhedral manipulations, is also
presented in [83].
Example 12.5 We consider the double integrator system (12.23) from Example 12.4
with N = .
The partition of the state space for the time invariant optimal control law is shown in
Fig. 12.6(a) and consists of 202 polyhedral regions. In Fig. 12.6(b) the same control
law is represented where polyhedra with the same affine control law were merged.
12.5
12.5
235
min
subj. to
U0 ,N
(12.78)
c(x0 , u0 )
subj. to
x1 = Ax0 + Bu0
x0 X , u0 U
x1 Xf ,
u0
(12.79)
where c(x0 , u0 ) is any convex quadratic function. Let us assume that the solution of
1
the multiparametric program generates R1 regions {Pr1 }R
r=1 with the affine control
1
1
law u0 = Fr x + Gr in each region r. By construction we have
X0 = K1 (Xf ).
Continuing setting up simple multiparametric programs bring the states into Xf
in 2, 3, . . . steps, we have for step j
min
c(x0 , u0 )
subj. to
x1 = Ax0 + Bu0
x0 X , u0 U
x1 Kj1 (Xf ),
u0
(12.80)
j
j
which yields Rj regions {Prj }R
r=1 with the affine control law u0 = Fr x + Gr in each
region r. By construction we have
X0 = Kj (Xf ).
Thus to obtain K1 (Xf ), . . . , KN (Xf ) we need to solve N multiparametric programs
with a prediction horizon of 1. Since the overall complexity of a multiparametric
program is exponential in N , this scheme can be exploited to yield controllers
of lower complexity than the optimal control scheme introduced in the previous
sections.
Since N multiparametric programs have been solved, the controller regions overlap in general. In order to achieve minimum time behavior, the feedback law associated with the region computed for the smallest number of steps c, is selected for
any given state x.
Algorithm 12.1 Minimum-Time Controller: On-Line Application
236
12
repeat
Obtain state measurement x.
Find controller partition cmin = minc{0,...,N} c, s.t. x Kc (Xf ).
Find controller region r, such that x Prcmin and compute u0 = Frcmin x +Gcrmin .
Apply input u0 to system and go to Step 1.
Note that the region identification for this type of controller partition is much
more efficient than simply checking all the regions. The two steps of finding a
controller partition and finding a controller region in Algorithm 12.1 correspond
to two levels of a search tree, where the search is first performed over the feasible
c
sets Kc (Xf ) and then over the controller partition {Prc }R
r=1 . Furthermore, one
i
may discard all regions
S Pr which are completely covered by previously computed
i
controllers (i.e. Pr j{1,...,i1} Kj (Xf )) since they are not time optimal.
Example 12.6 Consider again the double integrator from Example 12.1. The MinimumTime Controller is computed that steers the system to the Maximal LQR Invariant
LQR
Set O
in the minimum number of time steps N . The Algorithm terminated afLQR
ter 11 iterations, covering the Maximal Controllable Set K (O
). The resulting
controller is defined over 33 regions. The regions are depicted in Fig. 12.7(a). The
LQR
Maximal LQR Invariant Set O
is the central shaded region.
The control law on this partition is depicted in Fig. 12.7(b). Note that, in general,
the minimum-time control law is not continuous as can be seen in Fig. 12.7(b).
12.6
12.6
examples we have usually observed a finite number, but cases can be constructed,
where the number can be proven to be infinite.
As we will argue in the following chapter, infinite horizon controllers based on
the 2-norm render the closed loop system exponentially stable, while controllers
based on the 1- or -norm render the closed loop system stable, only.
Among the controllers proposed in this chapter, the minimum-time controller
is usually the least complex involving the smallest number of regions. Minimum
time controllers are often considered to be too aggressive for practical use. The
controller here is different, however. It is minimum time only until it reaches the
terminal set Xf . Inside the terminal set a different unconstrained control law can
be used. Overall the operation of the minimum-time controller is observed to be
very similar to that of the other infinite-time controllers in this chapter.
237
238
12
5
x2 (1)
10
x2 (0)
10
0
-5
-10
-10
0
-5
-5
0
x1 (0)
10
-10
-10
-5
0
x1 (1)
10
10
10
5
x2 (3)
x2 (2)
0
-5
-10
-10
0
-5
-5
0
x1 (2)
10
-10
-10
-5
0
x1 (3)
10
10
10
5
x2 (5)
x2 (4)
0
-5
-10
-10
0
-5
-5
0
x1 (4)
10
-10
-10
-5
0
x1 (5)
10
12.6
5
x2
10
x2
10
0
-5
-10
-10
0
-5
-5
Partition
r
(N
= 117).
(a)
239
0
x1
before
10
merging
-10
-10
-5
Partition
r
(N
= 13).
(b)
0
x1
after
10
merging
240
12
5
x2 (1)
10
x2 (0)
10
0
-5
-10
-10
0
-5
-5
0
x1 (0)
10
-10
-10
-5
0
x1 (1)
10
10
10
5
x2 (3)
x2 (2)
0
-5
-10
-10
0
-5
-5
0
x1 (2)
10
-10
-10
-5
0
x1 (3)
10
10
10
5
x2 (5)
x2 (4)
0
-5
-10
-10
0
-5
-5
0
x1 (4)
10
-10
-10
-5
0
x1 (5)
10
12.6
5
x2
10
x2
10
0
-5
-10
-10
241
0
-5
-5
Partition
r
(N
= 202).
(a)
0
x1
before
10
merging
-10
-10
-5
0
x1
Partition
r
(N
= 26).
(b)
after
10
merging
10
5
u
x2
1
0
-5
-10
-10
0
-1
10
-5
0
x1
10
-5
5
0
0
-5 -10 5
x1
x2
Partition (N r =33).
(discontinuous).
Figure 12.7 Example 12.6: Double Integrator, minimum-time objective function. Partition of the state space and control law. Maximal LQR invariant
LQR
set O
is the central shaded region in (a).
242
12
Chapter 13
future
predicted outputs y(t + k|t)
t + Nm
t + Np
u(t)
manipulated inputs u(t + 1 + k)
t + 1t + 2
t + 1 + Nm t + 1 + Np
244
13
13.1
RHC Implementation
Consider the problem of regulating to the origin the discrete-time linear timeinvariant system
x(t + 1) = Ax(t) + Bu(t),
(13.1)
where x(t) Rn , u(t) Rm are the state and input vectors, respectively, subject
to the constraints
x(t) X , u(t) U, t 0,
(13.2)
13.1
RHC Implementation
245
minUtt+N |t
N
1
X
q(xt+k|t , ut+k|t )
k=0
subj. to
(13.3)
is solved at time t, where Utt+N |t = {ut|t , . . . , ut+N 1|t } and where xt+k|t denotes
the state vector at time t + k predicted at time t obtained by starting from the
current state xt|t = x(t) and applying to the system model
xt+k+1|t = Axt+k|t + But+k|t
the input sequence ut|t , . . . , ut+N 1|t . Often the symbol xt+k|t is read as the state
x at time t + k predicted at time t. Similarly, ut+k|t is read as the input u at
time t + k computed at time t. For instance, x3|1 represents the predicted state
at time 3 when the prediction is done at time t = 1 starting from the current state
x(1). It is different, in general, from x3|2 which is the predicted state at time 3
when the prediction is done at time t = 2 starting from the current state x(2).
Let Utt+N
|t = {ut|t , . . . , ut+N 1|t } be the optimal solution of (13.3) at time t
and Jt (x(t)) the corresponding value function. Then, the first element of Utt+N
|t
is applied to system (13.1)
(13.4)
u(t) = ut|t (x(t)).
The optimization problem (13.3) is repeated at time t + 1, based on the new state
xt+1|t+1 = x(t + 1), yielding a moving or receding horizon control strategy.
Let ft : Rn Rm denote the receding horizon control law that associates
the optimal input ut|t to the current state x(t), ft (x(t)) = ut|t (x(t)). Then the
closed-loop system obtained by controlling (13.1) with the RHC (13.3)-(13.4) is
x(k + 1) = Ax(k) + Bfk (x(k)) , fcl (x(k), k), k 0.
(13.5)
Note that the notation used in this chapter is slightly different from the one
used in Chapter 12. Because of the receding horizon strategy, there is the need
to distinguish between the input u (t + k) applied to the plant at time t + k, and
optimizer ut+k|t of the problem (13.3) at time t + k obtained by solving (13.3) at
time t with xt|t = x(t).
Consider problem (13.3). As the system, the constraints and the cost function
are time-invariant, the solution to problem (13.3) is a time-invariant function of
the initial state x(t). Therefore, in order to simplify the notation, we can set t = 0
in (13.3) and remove the term |0 since it is now redundant and rewrite (13.3) as
J0 (x(t)) = minU0
J0 (x(t), U0 ) , p(xN ) +
N
1
X
q(xk , uk )
k=0
subj. to
(13.6)
246
13
N
1
X
k=0
kQxk kp + kRuk kp .
(13.7)
If the squared Euclidian norm is used in (13.6), then we set p(xN ) = xN P xN and
q(xk , uk ) = xk Qxk + uk Ruk with P 0, Q 0 and R 0. The cost function is
rewritten as
N
1
X
J0 (x(0), U0 ) , xN P xN +
xk Qxk + uk Ruk .
(13.8)
k=0
(13.9)
(13.10)
13.1
RHC Implementation
247
Example 13.1 Consider the double integrator system (12.23) rewritten below:
x(t + 1) =
1
0
1
1
x(t) +
0
1
u(t)
(13.11)
The aim is to compute the receding horizon controller that solves the optimization
problem
(13.6) with p(xN ) = xN P xN , q(xk , uk ) = xk Qxk + uk Ruk N = 3, P = Q =
1 0
, R = 10, Xf = R2 subject to the input constraints
0 1
0.5 u(k) 0.5, k = 0, . . . , 3
(13.12)
5
5
, k = 0, . . . , 3.
x(k)
5
5
(13.13)
The QP problem associated with the RHC has the form (12.29) with
H=
, F =
10.50
10.00 0.50
20.50 10.00 9.50
, Y = [ 14.50
23.50
23.50
54.50 ]
(13.14)
and
G0 =
0.50
0.50
0.50
0.50
0.50
0.50
1.00
0.00
1.00
0.00
0.00
0.00
0.00
0.50
0.00
0.50
0.50
0.50
0.00
0.00
0.00
0.00
1.00
1.00
0.00
0.00
0.00
0.00
0.00
1.00
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.50
0.50
0.50
0.50
0.50
0.50
0.00
0.00
0.00
0.00
1.00 ,
1.00
0.00
0.50
0.00
0.50
0.50
0.50
0.00
0.00
0.00
0.00
E0 =
0.50
0.50
0.50
0.50
0.50
0.50
0.00
0.00
0.00
0.00
0.00
0.00
1.00
0.50
1.00
0.50
0.50
0.50
1.00
0.00
1.00
0.00
0.50
0.50
0.50
0.50
0.50
0.50
0.00
0.00
0.00
0.00
0.00
0.00 ,
1.00
0.50
1.00
0.50
1.50
1.50
0.00
1.00
0.00
1.00
0.50
0.50
5.00
5.00
5.00
5.00
5.00
5.00
5.00
5.00
0.50
w0 =
0.50
5.00
5.00
5.00
5.00
0.50
0.50
5.00
(13.15)
5.00
5.00
5.00
The RHC (13.6)-(13.9) algorithm becomes Algorithm 13.2. We refer to the solution
U0 of the QP (12.29) as [U0 , Flag] = QP(H, 2F x(0), G0 , w0 + E0 x(0)) where Flag
indicates if the QP was found to be feasible or not.
Algorithm 13.2 On-line receding horizon control QP-based
repeat
measure the state x(t) at time instant t
compute F = 2F x(t) and w
0 = w0 + E0 x(t)
obtain U0 (x(t)) by solving the optimization problem [U0 , Flag] = QP(H, F , G0 , w
0 )
if Flag=infeasible then problem infeasible stop
apply the first element u0 of U0 to the system
wait for the new sampling time t + 1
248
13
x2
2
1
0
-5 -4 -3 -2 -1 0 1 2 3 4 5
x1
Figure 13.2 Example 13.1. Closed-loop trajectories for the initial state
x(0)=[-4.5,2] (boxes) and x(0)=[-4.5,3] (circles).
Fig. 13.2 shows two closed-loop trajectories starting at state x(0) = [4.5, 2] and
x(0) = [4.5, 3]. The trajectory starting from x(0) = [4.5, 2] converges to the
origin and satisfies input and state constraints. The trajectory starting from x(0) =
[4.5, 3] stops at x(2) = [1, 2] because of infeasibility. At each time step, the openloop predictions are depicted with dashed lines. This shows that the closed-loop
trajectories are different from the open-loop predicted trajectories because of the
receding horizon nature of the controller.
In Fig. 13.3(a) the feasible state space was gridded and each point of the grid was
marked with a square if the RHC law (13.6)-(13.9) generates feasible closed-loop trajectories and with a circle if it does not. The set of all initial conditions generating
feasible closed-loop trajectories is the maximal positive invariant set O of the autonomous system (13.10). We remark that this set is different from the set X0 of
feasible initial conditions for the QP problem (12.29) with matrices (13.15). Both
sets O and X0 are depicted in figure 13.3(b). The computation of f0 is discussed
later in this chapter. Because of the nonlinear nature of f0 , the computation of O
for the system (13.10) is not an easy task. Therefore we will show how to choose a
terminal invariant set Xf such that O = X0 is guaranteed automatically.
Note that a feasible closed-loop trajectory does not necessarily converge to the origin.
Feasibility, convergence and stability of RHC are discussed in detail in the next
sections. Before that we want to illustrate these issues through another example.
Example 13.2 Consider the unstable system
1
2
1
u(t)
x(t) +
x(t + 1) =
0
0 0.5
(13.16)
(13.17)
(13.18)
In the following, we study the receding horizon control problem (13.6) with
p(xN ) = xN P xN , q(xk , uk ) = xk Qxk + uk Ruk for different horizons N and
replacemen
5
4
3
2
1
0
-1
-2
-3
-4
-5
-5 -4 -3 -2 -1 0 1 2 3 4 5
x1
249
x2
RHC Implementation
x2
13.1
5
4
3
2
1
0
-1
-2
-3
-4
-5
-5 -4 -3 -2 -1 0 1 2 3 4 5
x1
weights R. We set Q = I and omit both the terminal set constraint and the
terminal weight, i.e. Xf = R2 , P = 0.
Fig. 13.4 shows closed-loop trajectories for receding horizon control loops that
were obtained with the following parameter settings
Setting 1: N = 2, R = 10
Setting 2: N = 3, R = 2
Setting 3: N = 4, R = 1
For Setting 1 (Fig. 13.4(a)) there is evidently no initial state that can be steered
to the origin. Indeed, it turns out, that all non-zero initial states x(0) R2
diverge from the origin and eventually become infeasible. In contrast, Setting 2
leads to a receding horizon controller, that manages to get some of the initial
states converge to the origin, as seen in Fig. 13.4(b). Finally, Fig. 13.4(c) shows
that Setting 3 can expand the set of those initial states that can be brought to
the origin.
Note the behavior of particular initial states:
1. Closed-loop trajectories starting at state x(0) = [4, 7] behave differently
depending on the chosen setting. Both Setting 1 and Setting 2 cannot bring
this state to the origin, but the controller with Setting 3 succeeds.
2. There are initial states, e.g. x(0) = [4, 8.5], that always lead to infeasible
trajectories independent of the chosen settings. It turns out, that no setting
can be found that brings those states to the origin.
These results illustrate that the choice of parameters for receding horizon control influences the behavior of the resulting closed-loop trajectories in a complex
manner. A better understanding of the effect of parameter changes can be gained
from an inspection of maximal positive invariant sets O for the different settings, and the maximal control invariant set C as depicted in Fig. 13.5.
The maximal positive invariant set stemming from Setting 1 only contains the
origin (O = {0}) which explains why all non-zero initial states diverge from the
origin. For Setting 2 the maximal positive invariant set has grown considerably,
13
10
10
5
x2
x2
250
0
-5
-10
-8
0
-5
-4
0
x1
-10
-8
0
x1
-4
(b) Setting 2 : N = 3, R = 2.
10
x2
5
0
-5
-10
-8
-4
0
x1
(c) Setting 3 : N = 4, R = 1.
but does not contain the initial state x(0) = [4, 7], thus leading to infeasibility
eventually. Setting 3 leads to a maximal positive invariant set that contains this
state and thus keeps the closed-loop trajectory inside this set for all future time
steps.
From Fig. 13.5 we also see that a trajectory starting at x(0) = [4, 8.5] cannot
be kept inside any bounded set by any setting (indeed, by any controller) since
it is outside the maximal control invariant set C .
13.2
13.2
251
10
x2
5
0
-5
-10
-4
0
x1
Figure 13.5 Example 13.2. Maximal positive invariant sets O for different
parameter settings: Setting 1 (origin), Setting 2 (dark-gray) and Setting 3
(gray and dark-gray). Also depicted is the maximal control invariant set C
(white and gray and dark-gray).
controller has some nice properties. Most importantly, the differences between
the open-loop predicted and the actual closed-loop trajectories observed in Example 13.1 disappear. As a consequence, if the optimization problem is feasible, then
the closed-loop trajectories will be feasible for all times. If the optimization problem has a finite solution, then in closed-loop the states and inputs will converge to
the origin asymptotically.
In RHC, when we solve the optimization problem over a finite horizon repeatedly at each time step, we hope that the controller resulting from this shortsighted strategy will lead to a closed-loop behavior that mimics that of the infinite horizon controller. The examples in the last section indicated that at least two
problems may occur. First of all, the controller may lead us into a situation where
after a few steps the finite horizon optimal control problem that we need to solve
at each time step is infeasible, i.e., that there does not exist a sequence of control
inputs for which the constraints are obeyed. Second, even if the feasibility problem does not occur, the generated control inputs may not lead to trajectories that
converge to the origin, i.e., that the closed-loop system is asymptotically stable.
In general, stability and feasibility are not ensured by the RHC law (13.6)(13.9). In principle, we could analyze the RHC law for feasibility, stability and
convergence but this is difficult as the examples in the last section illustrated.
Therefore, conditions will be derived on how the terminal weight P and the terminal
constraint set Xf should be chosen such that closed-loop stability and feasibility
are ensured.
13.2.1
Feasibility of RHC
The examples in the last section illustrate that feasibility at the initial time x(0)
X0 does not necessarily imply feasibility for all future times. It is desirable to
design a RHC such that feasibility for all future times is guaranteed, a property we
refer to as persistent feasibility.
We would like to gain some insight when persistent feasibility occurs and how it
252
13
is affected by the formulation of the control problem and the choice of the controller
parameters. Let us first recall the various sets introduced in Section 11.2 and
Section 12.2 and how they are influenced.
C :
The maximal control invariant set C is only affected by the sets X and U,
the constraints on states and inputs. It is the largest set over which we can
expect any controller to work.
X0 :
A control input U0 can only be found, i.e. the control problem is feasible,
if x(0) X0 . The set X0 depends on X and U, on the controller horizon N
and on the controller terminal set Xf . It does not depend on the objective
function and it has generally no relation with C (it can be larger, smaller,
etc.).
O :
The maximal positive invariant set for the closed-loop system depends on
the controller and as such on all parameters affecting the controller, i.e., X , U,
N , Xf and the objective function with its parameters P , Q and R. Clearly
O X0 because if it were not there would be points in O for which
the control problem is not feasible. Because of invariance, the closed-loop is
persistently feasible for all states x(0) O .
We can now state necessary and sufficient conditions guaranteeing persistent feasibility by means of invariant set theory.
Lemma 13.1 Let O be the maximal positive invariant set for the closed-loop
system x(k + 1) = fcl (x(k)) in (13.10) with constraints (13.2). The RHC problem
is persistently feasible if and only if X0 = O
Proof: For the RHC problem to be persistently feasible X0 must be positive
invariant for the closed-loop system. We argued above that O X0 . As the
positive invariant set X0 cannot be larger than the maximal positive invariant set
O , it follows that X0 = O .
As X0 does not depend on the controller parameters P , Q and R but O does,
the requirement X0 = O for persistent feasibility shows that, in general, only
some P , Q and R are allowed. The parameters P , Q and R affect the performance.
The complex effect they have on persistent feasibility makes their choice extremely
difficult for the design engineer. In the following we will remedy this undesirable
situation. We will make use of the following important sufficient condition for
persistent feasibility.
Lemma 13.2 Consider the RHC law (13.6)-(13.9) with N 1. If X1 is a control
invariant set for system (13.1)-(13.2) then the RHC is persistently feasible. Also,
O = X0 is independent of P , Q and R.
Proof: If X1 is control invariant then, by definition, X1 Pre(X1 ). Also recall
that Pre(X1 ) = X0 from the properties of the feasible sets in equation (12.20)
(note that Pre(X1 ) X = Pre(X1 ) from control invariance). Pick some x X0
and some feasible control u for that x and define x+ = Ax + Bu X1 . Then
x+ X1 Pre(X1 ) = X0 . As u was arbitrary (as long as it is feasible) x+ X0
13.2
253
254
13.2.2
13
Stability of RHC
In this section we will derive the main stability result for RHC. Our objective is
to find a Lyapunov function for the closed-loop system. We will show next that if
the terminal cost and constraint are appropriately chosen, then the value function
J0 () is a Lyapunov function.
Theorem 13.2 Consider system (13.1)-(13.2), the RHC law (13.6)-(13.9), the
cost function (13.8) or (13.7) and the closed-loop system (13.10). Assume that
(A0)
(A1)
The sets X , Xf and U contain the origin in their interior and are closed.
(A2)
Xf is control invariant, Xf X .
(A3)
min
vU , Ax+BvXf
Then,
(i) the state of the closed-loop system (13.10) converges to the origin, i.e., limk x(k) =
0,
(ii) the origin of the closed-loop system (13.10) is asymptotically stable with domain
of attraction X0 .
Proof: Based on Lemma 13.1, 13.1 and Theorem 13.2, from hypothesis (A2) we
conclude that X0 = O is a positive invariant set for the closed-loop system (13.10)
for any choice of Q, R and P . Thus persistent feasibility for any feasible input is
guaranteed.
Next we prove convergence and stability. We establish that the function J0 ()
in (13.6) is a Lyapunov function for the closed-loop system. Because the cost J0 ,
the system and the constraints are time-invariant we can study the properties of
J0 between step k = 0 and step k + 1 = 1.
Consider problem (13.6) at time t = 0. Let x(0) X0 and let U0 = {u0 , . . . , uN 1 }
be the optimizer of problem (13.6) and x0 = {x(0), x1 , . . . , xN } be the corresponding optimal state trajectory. After the implementation of u0 we obtain
x(1) = x1 = Ax(0) + Bu0 . Consider now problem (13.6) for t = 1. We will con1 = {u , . . . , u , v}
struct an upper bound on J0 (x(1)). Consider the sequence U
1
N 1
1 =
and the corresponding state trajectory resulting from the initial state x(1), x
{x1 , . . . , xN , AxN + Bv}. Because xN Xf and (A2) there exists a feasible v such
1 = {u1 , . . . , u , v}
that xN +1 = AxN + Bv Xf and with this v the sequence U
N 1
(13.19)
(13.20)
13.2
255
Equation (13.20) and the hypothesis (A0) on the matrices R and Q ensure that
J0 (x) strictly decreases along the state trajectories of the closed-loop system (13.10)
for any x X0 , x 6= 0. In addition to the fact that J0 (x) decreases, J0 (x) is
lower-bounded by zero and since the state trajectories generated by the closed-loop
system (13.10) starting from any x(0) X0 lie in X0 for all k 0, equation (13.20)
is sufficient to ensure that the state of the closed-loop system converges to zero as
k 0 if the initial state lies in X0 . We have proven (i).
In order to prove stability via Theorem 8.2 we have to establish that J0 (x) is
a Lyapunov function. Positivity holds by definition, decrease follows from (13.20).
For continuity at the origin we will show that J0 (x) p(x), x Xf and as p(x)
is continuous at the origin J0 (x) must be continuous as well. From assumption
(A2), Xf is control invariant and thus for any x Xf there exists a feasible input
sequence {u0 , . . . , uN 1 } for problem (13.6) starting from the initial state x0 = x
whose corresponding state trajectory is {x0 , x1 , . . . , xN } stays in Xf , i.e., xi
Xf i = 0, . . . , N . Among all the aforementioned input sequences {u0 , . . . , uN 1 }
we focus on the one where ui satisfies assumption (A3) for all i = 0, . . . , N 1.
Such a sequence provides an upper bound on the function J0 :
!
N
1
X
q(xi , ui ) + p(xN ), xi Xf , i = 0, . . . , N,
(13.21)
J0 (x0 )
i=0
p(x
)
xi Xf , i = 0, . . . , N,
i
i
i+1
i
i=0
(13.22)
which from assumption (A3) yields
J0 (x) p(x), x Xf .
(13.23)
Remark 13.3 A function p(x) satisfying assumption (A3) of Theorem 13.2 is often
called control Lyapunov function .
256
13
(13.24)
(13.25)
13.3
257
which is the corresponding Lyapunov inequality for the 1, -norm case (8.50)
whose solution has been discussed in Section 8.5.3.
In general, if the unconstrained optimal controller (10.31) exists it is PPWA.
In this case the computation of the maximal invariant set Xf for the closed-loop
PWA system
x(k + 1) = (A + F i )x(k)
if H i x 0, i = 1, . . . , N r
(13.27)
13.3
(13.30)
We want to regulate the system to the origin by using the RHC problem (13.6)(13.9)
with cost (13.8), Q = [ 10 01 ], R = 0.01, and P = P where P solves the algebraic
Riccati equation (9.31). We consider three cases:
Case 1. N = 2, Xf = 0,
Case 2. N = 2, Xf is the positively invariant set of the closed-loop system x(k + 1) =
(A + BF ) where F is the infinite-time unconstrained optimal controller (9.32).
Case 3. No terminal state constraints: Xf = R2 and N = 6 = determinedness index+1.
From the results presented in this chapter, all three cases guarantee persistent feasibility for all cost functions and asymptotic stability of the origin with region of
attraction X0 (with X0 different for each case) . Next we will detail the matrices of
the quadratic program for the on-line solution as well as the explicit solution for the
258
13
three cases.
Case 1: Xf = 0. The mp-QP problem associated with the RHC has the form (12.29)
with
i
h
10.31 9.33
19.08 8.55
10.57 5.29
H=
and
0.00
0.00
0.00
1.00
0.00
1.00
1.00
1.00
1.00
1.00
0.00
1.00
0.00
1.00
1.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
1.00
1.00
1.00
1.00
G0 =
, F =
8.55 5.31
, Y =
10.59 5.29
9.33 10.37
0.00 0.00
1.00
0.00 0.00
1.00
1.00 1.00
1.00
1.00
10.00
10.00
1.00 1.00
10.00
1.00 1.00
10.00
0.00 0.00
10.00
1.00 1.00
10.00
10.00
0.00 0.00
10.00
1.00 1.00
5.00
1.00 1.00
5.00
1.00 1.00
5.00
1.00 1.00
1.00 1.00
, w0 = 5.00
E0 =
1.00
1.00 2.00
1.00
1.00 2.00
10.00
1.00 0.00
10.00
0.00 1.00
10.00
1.00 0.00
10.00
0.00 1.00
5.00
5.00
1.00 0.00
5.00
0.00 1.00
1.00 0.00
5.00
0.00 1.00
0.00 0.00
0.00
1.00 1.00
0.00
1.00
1.00
0.00
0.00
0.00
0.00
0.00
1.00
0.00
1.00
0.00
0.00
0.00
0.00
,
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
0.00
1.00
0.00
1.00
(13.31)
(13.32)
0.00
0.00
0.00 0.00
1.00 1.00
[ 0.61 1.61 ] x
[ 1.00 2.00 ] x
[ 1.00 2.00 ] x
0.72 1.72
[
]x
u=
[ 1.00 2.00 ] x
[ 1.00 2.00 ] x
[ 0.72 1.72 ] x
if
if
if
if
if
if
if
0.70
0.70
0.70
0.70
0.71
0.71
0.71
0.71
0.71 0.71
0.70
0.45
0.45
0.71
0.70
0.71
0.89
x
0.89
0.71
0.71
0.45 0.89
0.70 0.71
0.71 0.71
0.39
0.70
0.70
0.70
0.45
0.71
0.70
0.45
0.71
0.70
0.92
0.71
0.71
0.71
h 0.45 0.89 i
0.70 0.71
0.70 0.71
0.70 0.71
0.00
0.00
0.45
0.45
0.71
0.00
0.45
0.00
0.00
(Region #1)
0.39 0.92
0.00
0.00
0.00
0.89
0.71
0.71
x
0.89
0.71
0.71
0.71 0.71
0.70 0.71
0.00
0.54
0.00
0.00
0.00
0.45
0.00
0.00
0.54
0.00
0.00
0.00
(Region #2)
(Region #3)
0.45
0.71
0.00
0.45
0.00
0.00
(Region #4)
(Region #5)
(Region #6)
(Region #7)
The union of the regions depicted in Fig. 13.6(a) is X0 . From Theorem 13.2, X0 is
also the domain of attraction of the RHC law.
Case 2: Xf positively invariant set. The set Xf is
0.35617 0.93442
0.58043
0.35617 0.93442
0.58043
Xf = {x R2 :
x
0.71286 0.70131
1.9049 }
0.71286 0.70131
1.9049
(13.33)
259
x2
x2
13.3
x1
x1
(b) Polyhedral partition of X0 ,
LQR
Case 2. N = 2, Xf = O
,
r
N0 = 9.
x2
x1
(c) Polyhedral partition of X0 ,
Case 3. N = 6, Xf = R2 , N0r =
13.
260
13
13.4
(13.34)
Example 13.4 Consider the double integrator system (13.11) subject to the input
constraints
1 u(k) 1
(13.35)
5 x(k) 5
(13.36)
We want to regulate the system to the origin by using the RHC controller (13.6)
(13.9) with cost (13.7), p = , Q = [ 10 01 ], R = 20. We consider two cases:
Case 1. Xf = Rn , N = 6 (determinedness index+1) and P = Q
Case 2. Xf = Rn , N = 6 and P = kP k given in (10.34) measuring the infinitetime unconstrained optimal cost in (10.32).
From Corollary 13.2 in both cases persistent feasibility is guaranteed for all cost functions and X0 = C . However, in Case 1 the terminal cost P does not satisfy (13.26)
which is assumption (A3) in Theorem 13.2 and therefore the convergence to and the
stability of the origin cannot be guaranteed. In order to satisfy Assumption (A3) in
Theorem 13.2, in Case 2 we select the terminal cost to be equal to the infinite-time
unconstrained optimal cost computed in Example 10.1.
Next we will detail the explicit solutions for the two cases.
0
Case 1. The LP problem associated with the RHC has the form (12.52) with G
R12418 , S0 R1242 and c = [06 112 ]. The corresponding polyhedral partition of
the state-space is depicted in Fig. 13.7(a). The RHC law is:
261
x2
x2
13.4
x1
x1
x2
x2
Case 2.
x1
x1
Closed-loop
Case 2.
Trajectories,
(d)
1.
u=
[ 0.29
1.00
[ 0.29
1.00
if
1.71 ] x
1.43
if
if
1.71 ] x
1.43
if
if
0.16
0.16
1.00
1.00
0.99
0.99
0.00
0.00
1.00
0.16
1.00
0.16
0.00
0.99
0.00
0.99
0.16
1.00
0.71
1.00
0.20
0.16
0.24
0.45
0.32
1.00
0.24
0.20
0.16
1.00
0.16
0.71
0.45
1.00
x
x
0.99
0.00
0.71
0.00
0.98 x
0.99
0.97
0.89
0.95
0.00
0.16 0.99
1.00 0.00
0.16 0.99
0.32
0.82
5.00
5.00
5.00
0.82
5.00
1.40
(Region #1)
1.40
0.95
0.97
0.98
0.99
0.00 x
0.99
0.71
0.89
0.00
0.82
5.00
4.24
5.00
2.94
3.04
2.91
3.35
3.00
5.00
0.82
5.00
1.40
3.00
2.91
2.94
3.04
5.00
1.40
4.24
3.35
5.00
(Region #2)
(Region #3)
(Region #4)
(Region #5)
262
13
The union of the regions depicted in Fig. 13.7(a) is X0 and is shown in white in
Fig. 13.7(c). Since N is equal to the determinedness index plus one, X0 is a positive
invariant set for the closed-loop system and thus persistent feasibility is guaranteed
for all x(0) X0 . However, it can be noticed from Fig. 13.7(c) that convergence to
the origin is not guaranteed. Starting from the initial conditions [-4,2], [-2, 2], [0,
0.5], [4,-2], [-1,-1] and [2,-0.5], the closed-loop system converges to either [5, 0] or
[5, 0].
0
Case 2. The LP problem associated with the RHC has the form (12.52) with G
17418
1742
R
, S0 R
and c = [06 112 ]. The RHC law is defined over 21 regions and
the corresponding polyhedral partition of the state-space is depicted in Fig. 13.7(b).
The union of the regions depicted in Fig. 13.7(b) is X0 and is shown in white in
Fig. 13.7(d). Since N is equal to the determinedness index plus one, X0 is a positive
invariant set for the closed-loop system and thus persistent feasibility is guaranteed
for all x(0) X0 . Convergence to the origin is also guaranteed by the choice of P as
shown by the closed-loop trajectories in Fig. 13.7(d) starting from the same initial
conditions as in Case 1.
13.5
Objective Function
The squared 2-norm is employed more often as an indicator of control quality in
the objective function than the 1- or -norm. The former has many advantages.
The 1- or -norm formulation leads to an LP (12.51) from which the optimal control action is determined. The solution lies always at the intersection of
constraints and changes discontinuously as the tuning parameters are varied. This
makes the formulation of the control problem (what constraints need to be added
for good performance?) and the choice of the weights in the objective function
difficult.
The optimizer may be non-unique in the case of dual degeneracy. This may
cause the control action to vary randomly as the solver picks up different optimizers
in succeeding time steps.
The construction of the control invariant target set Xf and of a suitable terminal cost p(xN ) needed in Theorem 13.2 is more difficult for the 1- or -norm
13.5
263
264
13
over which the system can operate stably in the presence of input constraints and
is of eminent practical importance.
State/Output Constraints
State constraints arise from practical restrictions on the allowed operating range
of the system. Thus, contrary to input constraints, they are rarely hard. They
can lead to complications in the controller implementation, however. As it can
never be excluded that the state of the real system moves outside the constraint
range chosen for the controller design, special provisions must be made (patches in
the algorithm) to move the state back into the range. This is difficult and these
types of patches are exactly what one wanted to avoid by choosing MPC in the
first place.
Thus, typically, state constraints are softened in the MPC formulation. For
example,
x xmax
is approximated by
x xmax + , 0,
and a term l() is added to the objective function to penalize violations of the
constraint. This formulation may, however, lead to a violation of the constraint
even when a feasible input exists that avoids it but is not optimal for the new
objective function with the penalty term added. Let us analyze how to choose l()
such that this does not occur [68].
As an example take
J = minz
subj. to
f (z)
g(z) 0,
(13.37)
where g(z) is assumed to be scalar for simplicity and f (z) to be strictly convex.
Let us soften the constraints and add the penalty as suggested
p() = minz
subj. to
f (z) + l()
g(z) ,
(13.38)
J
0
(13.39)
p() >
p(0), > 0.
(13.40)
13.5
265
For (13.39) we require l(0) = 0. To construct l() to satisfy (13.40) we assume that
strong duality holds and u exists so that
J = min(f (z) + u g(z)),
(13.41)
(f (z) + u g(z))
g(z)
0.
(13.42)
minz
subj. to
(f (z) + u )
p() =
<
g(z) ,
minz
subj. to
(f (z) + u)
> 0, u > u .
g(z) ,
(13.43)
Thus l() = u with u > u 0 is a possible penalty term satisfying the requirements (13.39)-(13.40).
Because of smoothness
l() = u + v2 , u > u , v > 0
(13.44)
is preferable. On the other hand, note that l() = v2 does not satisfy an inequality
of the form (13.43). Therefore it should not be used as it can lead to optimizers
z in (13.38) which violate g(z) 0 even if a feasible optimizer to the original
problem exists.
These ideas can be extended to multiple constraints gj (z) 0, j = 1, . . . , r via
the penalty term
r
r
X
X
2j ,
(13.45)
j + v
l() = u
j=1
j=1
where
u>
max
j{1,...,r}
uj , v 0.
(13.46)
Formulations also exist where the necessary constraint violations are following
a prescribed order so that less important constraints are violated first [166].
266
13
If the underlying system is nonlinear, one often uses a locally linearized model
for the prediction and update it at each time step. Note that the results of Theorem 13.2 do not apply when the system model and/or the constraints are timevarying.
In all these cases, the optimization problem to be solved on line, (12.29) or (12.51),
does not change in structure but some of the defining matrices will now change at
each time step, which will increase the necessary on-line computational effort somewhat.
If the controller is to be computed explicitly off-line, then all the varying parameters (disturbances, constraints, references), which we will denote by , become parameters in the multi-parametric optimization problem and the resulting
controller becomes an explicit function of them: u(t) = F (x(t), ). As we have
learned, the complexity of the solution of mp-QP and mp-LP problems depends
primarily on the number of constraints. If the number of parameters affects the
number of constraints then this may only be possible for a relatively small number
of parameters. Thus, the possibilities to take into account time-varying control
problem features in explicit MPC are rather limited. Time-varying models cannot
be handled at all.
min
U0
subj. to
p(xNy ) +
q(xk , uk )
k=0
(13.47)
where K is some feedback gain, Ny , Nu , Nc are the prediction, input, and state
constraint horizons, respectively, with Nu Ny and Nc Ny . This formulation
reduces the number of constraints and as a consequence makes the long horizon
prediction used in the optimization less accurate as it is not forced to obey all the
constraints. As this approximation affects only the states far in the future, it is
hoped that it will not influence significantly the present control action.
Generalized Predictive Control (GPC) in its most common formulation [85] has
multiple horizons as an inherent feature but does not include constraints. Experience and theoretical analysis [53] have shown that it is very difficult to choose all
these horizons that affect not only performance but even stability in a non-intuitive
13.6
fashion. Thus, for problems where constraints are not important and the adaptive
features of GPC are not needed, it is much preferable to resort to the well established LQR and LQG controllers for which a wealth of stability, performance and
robustness results have been established.
Another more effective way to reduce the computational effort is move-blocking
where the manipulated variables are assumed to be fixed over time intervals in
the future thus reducing the degrees of freedom in the optimization problem. By
choosing the blocking strategies carefully, the available RHC stability results remain
applicable [69].
13.6
267
268
13
with d(t) Rnd . With abuse of notation we have used the same symbols for state
and outputs of system (13.1) and system (13.49). Later we will focus on specific
versions of the model (13.49).
The observer estimates both states and disturbances based on this augmented
model. Conditions for the observability of (13.49) are given in the following theorem.
Theorem 13.3 [209, 210, 218, 11] The augmented system (13.49) is observable
if and only if (C, A) is observable and
A I Bd
(13.50)
C
Cd
has full column rank.
iff
Again from the Hautus condition, the first set of rows is linearly independent iff
(C, A) is observable. The second set of rows is linearly independent from the first
n rows except possibly for = 1. Thus, for the augmented system the Hautus
condition needs to be checked for = 1 only, where it becomes (13.50).
Remark 13.4 Note that for condition (13.50) to be satisfied the number of disturbances in d needs to be smaller or equal to the number of available measurements in y,
nd 6 p. Condition (13.50) can be nicely interpreted. It requires that the model of the
disturbance effect on the output d y must not have a zero at (1, 0). Alternatively
we can look at the steady state of system (13.49)
0
x
A I Bd
,
(13.52)
=
y
d
C
Cd
where we have denoted the steady state values with a subscript and have omitted the forcing term u for simplicity. We note that from the observability condition (13.50) for system (13.49) equation (13.52) is required to have a unique solution,
which means, that we must be able to deduce a unique value for the disturbance d
from a measurement of y in steady state.
13.6
269
The state and disturbance estimator is designed based on the augmented model
as follows:
x
(t + 1)
x
(t)
A Bd
B
Lx
=
+
(ym (t)+C x
(t)+Cd d(t)),
u(t)+
+ 1)
0 I
0
Ld
d(t
d(t)
(13.54)
where Lx and Ld are chosen so that the estimator is stable. We remark that the
results below are independent of the choice of the method for computing Lx and
Ld . We then have the following property.
Lemma 13.3 Suppose the observer (13.54) is stable. Then, rank(Ld ) = nd .
Proof: From (13.54) it follows
x
(t + 1)
x
(t)
A + Lx C Bd + Lx Cd
B
Lx
=
+
ym (t).
u(t)
+ 1)
0
Ld C
I + Ld Cd
Ld
d(t
d(t)
(13.55)
By stability, the observer has no poles at (1, 0) and therefore
A I + Lx C Bd + Lx Cd
6= 0.
(13.56)
det
Ld C
Ld Cd
For (13.56) to hold, the last nd rows of the matrix have to be of full row rank.
A necessary condition is that Ld has full row rank.
In the rest of this section, we will focus on the case nd = p.
Lemma 13.4 Suppose the observer (13.54) is stable. Choose nd = p. The steady
state of the observer (13.54) satisfies:
AI
C
B
0
Bd d
ym, Cd d
(13.57)
where ym, and u are the steady state measured output and input of the system (13.48), x
and d are state and disturbance estimates from the observer (13.54)
at steady state, respectively.
Proof: From (13.54) we note that the disturbance estimate d converges only
if Ld (ym, + C x
+ Cd d ) = 0. As Ld is square by assumption and nonsingular
by Lemma 13.3 this implies that at steady state, the observer estimates (13.54)
satisfy
(13.58)
ym, + C x
+ Cd d = 0.
Equation (13.57) follows directly from (13.58) and (13.54).
270
13
r . The observer condition (13.57) suggests that at steady state the MPC should
satisfy
AI B
x
Bd d
,
(13.59)
=
HC
0
u
r HCd d
where x is the MPC
state. For x and u to exist for any d
state at steady
AI B
and r the matrix
must be of full row rank which implies m r.
HC
0
The MPC is designed as follows
N
1
X
minU0
(xN xt ) P (xN x
t ) +
subj. to
d0 = d(t),
k=0
(xk x
t ) Q(xk x
t ) + (uk ut ) R(uk u
t )
(13.60)
with the targets u
t and xt given by
AI B
x
t
Bd d(t)
=
HC
0
u
t
r(t) HCd d(t)
(13.61)
(13.62)
r(t)) = u (
Denote by c0 (
x(t), d(t),
0 x(t), d(t), r(t)) the control law when the esti
mated state and disturbance are x
(t) and d(t),
respectively. Then the closed-loop
system obtained by controlling (13.48) with the MPC (13.60)-(13.61)-(13.62) and
the observer (13.54) is:
r(t)))
x(t + 1) = f (x(t), c0 (
x(t), d(t),
+ Bc0 (
r(t)) Lx ym (t)
x(t + 1) = (A + Lx C)
x(t) + (Bd + Lx Cd )d(t)
x(t), d(t),
d(t + 1) = Ld C x
(t) + (I + Ld Cd )d(t) Ld ym (t).
(13.63)
Often in practice, one desires to track all measured outputs with zero offset.
Choosing nd = p = r is thus a natural choice. Such a zero-offset property continues
to hold if only a subset of the measured outputs are to be tracked, i.e., nd = p > r.
Next we provide a very simple proof for offset-free control when nd = p.
Theorem 13.4 Consider the case nd = p. Assume that for r(t) r as t ,
the MPC problem (13.60)-(13.61) is feasible for all t N+ , unconstrained for t j
, d , ym, , i.e.,
with j N+ and the closed-loop system (13.63) converges to x
d , ym (t) ym, as t . Then z(t) = Hym (t) r as
x
(t) x
, d(t)
t .
13.6
271
(13.64)
=
C
0
u
ym, Cd d
and the controller requirement (13.61)
AI B
Bd d
x
.
=
HC
0
u
r HCd d
(13.65)
Define x = x
x
, u = u u
and the offset = z r . Notice that the
steady state target values x
and u are both functions of r and d as given
by (13.65). Left multiplying the second row of (13.64) by H and subtracting (13.65)
from the result, we obtain
(A I)x + Bu = 0
HCx = .
(13.66)
d(t) = dk . Similarly, one can show that xk+1 = Axk + Buk . Then, the MPC
problem (13.60) becomes:
N
1
X
minu0 ,...,uN 1
xN P xN +
xk Qxk + uk Ruk
subj. to
k=0
(13.67)
Denote by KMP C the unconstrained MPC controller (13.67), i.e., u0 = KMP C x(t).
At steady state u0 u u
= u and x(t) x
x
= x. Therefore, at
steady state, u = KMP C x. From (13.66)
(A I + BKMP C )x = 0.
(13.68)
By assumption the unconstrained system with the MPC controller converges. Thus
KMP C is a stabilizing control law, which implies that (A I + BKMP C ) is nonsingular and hence x = 0.
Remark 13.6 Theorem 13.4 was proven in [218] by using a different approach.
272
13
Remark 13.7 Theorem 13.4 can be extended to prove local Lyapunov stability of
the closed-loop system (13.63) under standard regularity assumptions on the state
update function f in (13.63) [197].
Remark 13.8 The proof of Theorem 13.4 assumes only that the models used for the
control design (13.1) and the observer design (13.49) are identical in steady state in
the sense that they give rise to the same relation z = z(u, d, r). It does not make
any assumptions about the behavior of the real plant (13.48), i.e. the model-plant
mismatch, with the exception that the closed-loop system (13.63) must converge to
a fixed point. The models used in the controller and the observer could even be
different as long as they satisfy the same steady state relation.
Remark 13.10 Note that in order to achieve no offset we augmented the model of the
plant with as many disturbances (and integrators) as we have measurements (nd = p)
(cf. equation (13.55)). Our design procedure requires the addition of p integrators
even if we wish to control only a subset of r < p measured variables. This is actually
not necessary as we suspect from basic system theory. The design procedure for the
case nd = r < p is, however, more involved.
13.6
273
nd = r. The total size of the parameter vector can thus be reduced to n + 2r.
This is significant only if a small subset of the plant outputs are to be controlled.
A greater reduction of parameters can be achieved by the following method. By
Corollary 13.3, we are allowed to choose Bd = 0 in the disturbance model if the
plant has no integrators. Recall the target conditions 13.61 with Bd = 0
0
AI B
x
t
=
.
(13.70)
HC
0
ut
r(t) HCd d(t)
y(t) = Cx(t).
(13.73)
274
13
(13.74)
Minimum-Time Controller
In minimum-time control, the cost function minimizes the predicted number of
steps necessary to reach a target region, usually the invariant set associated with
the unconstrained LQR controller [162]. This scheme can reduce the on-line computation time significantly, especially for explicit controllers (Section 12.5). While
minimum-time MPC is computed and implemented differently from standard MPC
controllers, there is no difference between the two control schemes at steady-state.
In particular, one can choose the target region to be the unconstrained region
of (13.60)-(13.61). When the state and disturbance estimates and reference are
within this region, the control law is switched to (13.60)-(13.61). The analysis and
methods presented in this section therefore apply directly.
13.7
Literature Review
Although the basic idea of receding horizon control was already indicated by the
theoretical work of Propoi [230] in 1963 it did not gain much attention until the
mid-1970s, when Richalet and coauthors [239, 240] proposed the MPC technique
(they called it Model Predictive Heuristic Control (MPHC)). Shortly thereafter, Cutler and Ramaker [86] introduced the predictive control algorithm called
Dynamic Matrix Control (DMC) which has been hugely successful in the petrochemical industry. A vast variety of different names and methodologies followed,
such as Quadratic Dynamic Matrix Control (QDMC), Adaptive Predictive Control
(APC), Generalized Predictive Control (GPC), Sequential Open Loop Optimization (SOLO), and others.
While the mentioned algorithms are seemingly different, they all share the same
structural features: a model of the plant, the receding horizon idea, and an opti-
13.7
Literature Review
mization procedure to obtain the control action by optimizing the systems predicted evolution.
For complex constrained multivariable control problems, model predictive control has become the accepted standard in the process industries [231]. If in the finite
time optimal control problem solved by MPC at each time step, system model and
constraints are linear and the performance index is expressed as weighted sum of
2-norm, 1-norm or -norm vectors, then the resulting optimization problem can be
cast as a quadratic program (QP) or linear program (LP), respectively, for which
a rich variety of efficient active-set and interior-point solvers are available.
Some of the first industrial MPC algorithms like IDCOM [240] and DMC [86]
were developed for constrained MPC with quadratic performance indices. However,
in those algorithms input and output constraints were treated in an indirect ad-hoc
fashion. Only later, algorithms like QDMC [111] overcame this limitation by employing quadratic programming to solve constrained MPC problems with quadratic
performance indices. During the same period, the use of linear programming was
being studied by Gutman and coauthors [132, 133, 135]. Later an extensive theoretical effort was devoted to analyze such schemes, provide conditions for guaranteeing
feasibility and closed-loop stability, and highlight the relations between MPC and
the linear quadratic regulator (see the recent survey [198]).
An extensive treatment of conditions for feasibility and stability of MPC can also
be found in the surveys [198, 196]. Theorem 13.2 in this book presents the main
result on feasibility and stability of MPC ant it has been extracted from [198, 196].
The main idea of Theorem 13.2 dates back to Keerthi and Gilbert [163], the first to
propose specific choices of a terminal cost P and a terminal constraint Xf , namely
Xf = 0 and P = 0. Under this assumption (and a few other mild ones) Keerthi
and Gilbert prove the stability for general nonlinear performance functions and
nonlinear models. The work of Keerthi and Gilbert has been followed by a number
of stability result for RHC including the one in [163, 31, 1, 149, 80]. If properly
analyzed all these results have the main components required in Theorem 13.2.
The richness of theoretical and computational issues related to MPC have generated
in the past fifty years a large number of research studies appeared in conference
proceedings, archival journals and research monographs. It is not the intent of
this short paragraph to be an exhaustive literature review on Model Predictive
Control. The interested reader is also referred to recent books on Model Predictive
Control [72, 190, 238] for further reading on the topic.
275
276
13
Chapter 14
14.1
Introduction
This chapter will discuss approaches to define approximate explicit control laws
of desired complexity, that provide certificates of recursive feasibility and stability.
The methods introduced all have the following properties:
1. Operations are only on the problem data, and the methods do not require
the computation of the explicit control law.
2. Any convex parametric problem can be approximated, not just those for
which the explicit solution can be computed.
3. The desired level of sub-optimality or the desired complexity (in terms of
online storage and/or FLOPS) can be specified.
We begin by assuming that a controller with the desired properties has already
been defined via one of the predictive control methods introduced in Chapters 13,
but that solving the requisite optimization problem online, or evaluating the explicit
solution offline, is not possible due to computational limitations. This chapter
begins by itemizing the specific assumptions on this optimal control law. Then,
the key theorem is introduced, which provides sufficient conditions under which an
approximate control law will be stabilizing and recursively invariant. Section 14.3
describes an abstract algorithm that will provide these properties, and Section 14.4
reviews a number of specific implementations of this algorithm that have been
proposed in the literature.
278
14.2
14
J0 (x(t), U0 ) , p(xN ) +
minU0
N
1
X
q(xk , uk )
k=0
(14.1)
N
1
X
k=0
kQxk kp + kRuk kp
(14.2)
with with p = 1 or p = , or
J0 (x(0), U0 ) , xN P xN +
N
1
X
xk Qxk + uk Ruk .
(14.3)
k=0
0 = {
u0 , . . . , u
N 1 }. The
Denote a suboptimal feasible solution of (14.1) by U
corresponding suboptimal MPC law is
u(t) = f0 (x(t)) = u0 (x(t))
(14.4)
(14.5)
In the following we will prove that the stability of receding horizon control is
preserved when the suboptimal control law u
0 (x(t)) is used as long as the level of
suboptimality is less than the stage cost, i.e.
0 (x)) < J0 (x) + q(x, 0).
J0 (x, U
This is stated more precisely in the following Theorem 14.1.
Theorem 14.1 Assume
(A0)
(A1)
The sets X , Xf and U contain the origin in their interior and are closed.
(A2)
Xf is control invariant, Xf X .
(A3)
min
vU , Ax+BvXf
14.2
(A4)
(A5)
279
Then,
(i) the state of the closed-loop system (14.5) converges to the origin, i.e., limk x(k) =
0,
(ii) the origin of the closed-loop system (14.5) is asymptotically stable with domain
of attraction X0 .
Proof:
The proof follows the proof of Theorem 13.2. We focus only on the arguments
for convergence, which are different for the suboptimal MPC controller. We establish that the function J0 () in (14.1) strictly decreases for the closed-loop system.
Because the cost J0 , the system and the constraints are time-invariant we can study
the properties of J0 between step k = 0 and step k + 1 = 1.
0 = {
u0 , . . . , u
N 1 }
Consider problem (14.1) at time t = 0. Let x(0) X0 and let U
be a feasible suboptimal solution to problem (14.1) and x0 = {x(0), x1 , . . . , xN } be
the corresponding state trajectory. We denote by J0 (x(0)) the cost at x(0) when
0 is applied, J0 (x(0)) = J0 (x(0), U
0 ).
the feasible U
After the implementation of u0 we obtain x(1) = x1 = Ax(0) + B u
0 . Consider
now problem (14.1) for t = 1. We will construct an upper bound on J0 (x(1)).
1 = {
Consider the sequence U
u1 , . . . , u
N 1 , v} and the corresponding state trajec 1 = {x1 , . . . , xN , AxN + Bv}. Because
tory resulting from the initial state x(1), x
xN Xf and (A2) there exists a feasible v such that xN +1 = AxN + Bv Xf
1 = {
1 is not
and with this v the sequence U
u1 , . . . , uN 1 , v} is feasible. Because U
(14.6)
(14.7)
(14.8)
Equation (14.8), the hypothesis (A0) on the matrices Q and R and the hypothesis
(A5) ensure that J0 (x) strictly decreases along the state trajectories of the closed
loop system (14.5) for any x X0 , x 6= 0.
Theorem 14.1 has introduced sufficient conditions for stability for the approxi0 (x) based on its level of sub-optimality specified by the Assumpmate solution U
tions (A4) and (A5). In the following section we describe how these conditions
280
14
can be met by sampling the optimal control law and then using a technique called
barycentric interpolation.
To simplify the discussion we will adopt the notation for the general multiparametric programming problem from Chapter 6. We consider the multiparametric
program
J (x) = inf J(z, x)
z
(14.9)
subj. to g(z, x) 0
(14.11)
We denote by J (x) the real-valued function that expresses the dependence of the
minimum value of the objective function over K on x
J (x) = inf {J(z, x) : z R(x)},
z
(14.12)
and by Z (x) the point-to-set map which assigns the (possibly empty) set of optimizers z 2Z to a parameter x X
Z (x) = {z R(x) : J(z, x) = J (x)}.
(14.13)
14.3
Barycentric Interpolation
Several schemes have been proposed in the literature that compute an approximate
solution to various problems of the form (14.9) based on interpolating the optimizer
z (vi ) at a number of samples of the parameter V = {vi } K . The goal of
14.3
Barycentric Interpolation
281
interpolated control is to define a function z(x) over the convex hull of the sampled
points conv(V ), which has the required properties laid out in Theorem 14.1.
Section 14.3.1 will introduce the notion of barycentric interpolation, and will
demonstrate that such an approach will lead to an approximate function z(x),
which is feasible
z(x) R(x), x conv(V ).
(14.14)
Section 14.3.2 then introduces a convex optimization problem, whose optimal value,
if positive, verifies a specified level of suboptimality
J(
z (x), x) J (x) + (x), x conv(V ).
(14.15)
14.3.1
Feasibility
The class of interpolating functions z(x) satisfying (14.14) is called the class of
barycentric functions.
Definition 14.1 (Barycentric function) Let V = conv({v1 , . . . , vn }) Rn be a
polytope. The set of functions wv (x), v extreme(V ) is called barycentric if three
conditions hold for all x V
X
wv (x) 0 positivity
(14.16a)
(14.16b)
vextreme(V )
(14.16c)
vextreme(V )
z (v)wv (x)
(14.17)
vextreme(V )
The key statement that can be made about functions defined via barycentric
interpolation is that the function values lie within the convex hull of the function
values at the sample points. This property is illustrated in Figure 14.2, and formally
defined in the following Lemma.
282
14
vV
Proof: The statement holds if for each x conv(V ) there exists a set of
positive multipliers 0 , . . . , m such that
x
z(x)
m
X
i=0
vi
z (vi )
Pm
and i=0 i = 1. The properties of barycentric functions (14.16) clearly satisfy
this requirement.
Figure 14.2 Illustration of the Convex-Hull Property of Barycentric Interpolation. The first image shows a number of samples of a function, the
second the convex hull of these sample points and the third, a barycentric
interpolation that lies within this convex hull.
14.3.2
Sub-optimality
We next investigate the level of sub-optimality obtained for a barycentric interpolation.
We first demonstrate that the value of the function J (z, x) for the approximate
solution z(x) is upper-bounded by the barycentric interpolation of its optimal value
taken at the sample points. Then we introduce a convex optimization problem,
whose solution upper-bounds the approximation error.
vV
Proof: The lower bound follows from the fact that z(x) is feasible for the
optimization problem for all x V (Lemma 14.1).
The upper bound follows from convexity of the function J (z, x). Consider a
14.3
Barycentric Interpolation
283
wv (x)z (v),
vV
wv (x)v
vV
wv (x)J (z (v) , v)
vV
wv (x)J (v)
vV
The inequality in the above sequence follows because for a fixed value of the parameter x, the barycentric interpolating functions {wv } are all positive, and sum
to one.
The previous lemma demonstrates that due to the convexity of the value function J (z, x), the interpolation of the optimal value function at the sample points
provides an upper bound of the value function evaluated at the interpolated approximation z(x). We can use this fact to formulate a convex optimization problem,
whose solution will provide a certificate for (14.15).
Theorem 14.2 If V 0, then J (
z (x), x) J (x) + (x) for all x conv(V ),
where
X
V = min J (z, x) + (x)
v J (v)
(14.18)
x,z,v
s.t. x =
vV
vV
v v, v 0,
(x, z) C
v = 1
vV
Proof: We see that the sub-optimality bound is met within the convex hull
of V if and only if the following value is non-negative:
min
v ,xconv(V )
J (x) + (x) J (
z (x), x)
min
xconv(V )
J (x) + (x)
wv (x)J (v)
vV
and we obtain a further relaxation / lower bound by optimizing over all possible
barycentric functions
X
min
J (x) + (x)
v J (v)
xconv(V )
s.t.
x=
vV
vV
v v, v 0,
v = 1
vV
Recalling that J (x) = minz J (z, x) s.t. (x, z) C provides the desired result.
284
14
Figure 14.3 Illustration of the relationship between the optimal value function J (x), the value function evaluated for the sub-optimal function z(x)
and the convex hull of the optimal value at the sample points.
14.3.3
Constructive Algorithm
In this section, we introduce a simple high-level algorithm to incrementally construct a barycentric function that satisfies conditions (14.14)- (14.15) and therefore
can be used in the context of Theorem 14.1 to generate a stabilizing control law.
The idea is simple: begin with a set of sample points and solve the optimization
problem in Theorem 14.2 to test if the stability condition is met. If it is not,
then sample an additional set of points, sub-divide the resulting region into subregions and repeat the procedure for each such generated region. In the following
we formalize this idea, and demonstrate that this procedure will terminate.
Algorithm 14.1 Approximate Parametric Programming with Stability Guarantee
input: Parametric Optimization Problem (14.9). Initial set of samples V
output: Sample points V K and set of subsets R = {Rj }, Rj V such that
{conv(Rj )} forms a partition of conv(V ) and Rj 0 for all j
let R V
repeat
R argminRR R (from Problem (14.18))
if R < 0 then
V V {xR }, where xR is the optimizer of (14.18) for region R
R partition(V, R)
until R 0
return R
14.4
Remark 14.1 Several methods have been proposed in the literature that roughly
follow the scheme of Algorithm 14.1 []. Many of these approaches differ in the details
by adding caching steps to minimize computation of R , add multiple points at a
time, or take an incremental re-partitioning approach. All of these procedures can
significantly improve the computation speed of the algorithm, and should therefore
be incorporated into any practical approach.
14.4
14.4.1
Triangulation
The simplest approach to generate a barycentric function is to first triangulate
the sampled points V , and then define a piece-wise affine control law over the
triangulation. The resulting interpolation of the optimal value function at the
sampled points will result in a convex piecewise affine inner approximation of the
optimal value function, while the approximate control law will be a piecewise affine
function.
Definition 14.2 (Triangulation) A triangulation of a finite set of points V
Rn is a finite collection TV = {R0 , . . . , RL } such that
Ri = conv(Vi ) is an ndimensional simplex for some Vi V
conv(V ) = Ri and int Ri int Rj = for all i 6= j
If i 6= j then there is a common (possibly empty) face F of the boundaries of
Si and Sj such that Si Sj = F .
There are various triangulations possible; for example, the recursive triangulation developed in [223, 42] has the strong property of generating a simple hierarchy
that can significantly speed online evaluation of the resulting control law. The Delaunay triangulation [123], which has the nice property of minimizing the number
of skinny triangles, or those with small angles is a common choice for which incremental update algorithms are well-studied and readily available (i.e. computation
of TV {v} given TV ). A particularly suitable Delaunay triangulation can be defined
by using the optimal cost function as a weighting term, which causes the resulting
triangulation to closely match the optimal partition [154].
285
286
14
Given a discrete set of states V , we can now define an interpolated control law
z(x; TV ) : Rn 7 Rm .
Definition 14.3 If V X Rn is a finite set and T = {conv(V1 ), . . . , conv(VL )}
is a triangulation of V , then the interpolated function z(x; T ) : conv(V ) 7 Rm is
X
z (v)v , if x conv(Vj )
(14.19)
z(x; T ) =
vVj
where v 0,
v = 1 and x =
vVj
vv .
For each simplical region conv(Vj ) in the triangulation, the set of multipliers
is unique and can be found by solving the system of linear equations
1
v vn+1
x
= 1
1
1
1
where vi Vj .
The function z(x; T ) is piecewise affine, and given by:
T
z (v1 )
1
x
v1 vn+1
.
..
z(x; T ) =
if x Vj = conv({)v1 , . . . , vn+1 }
1
1
1
z (vn+1 )
The simplical nature of each region immediately demonstrates that the interpolation 14.19 is barycentric.
14.4.2
14.4
value function and an outer approximation of the optimal value function plus the
specified desired approximation error (Figure 14.4):
epi J P epi J ,
Once such an approximation is in hand, we will then use it to compute a polytopic
partition of K over which the approximate control law will be defined.
The quality of approximation is measured in terms of the Hausdorff distance,
which measures the distance between two sets X and Y as:
dH (X, Y ) = max supxX inf yY kx yk2 , supyY inf xX kx yk2
Our goal is to compute the Hausdorff distance between our polytopic approximation, and epi J , by solving a convex optimization problem. We first recall the
definition of the projection of a point v onto a convex set S
proj Sv = arg minxS kv xk22
The following lemma states that we can compute the projection of a point
onto the epigraph of the optimal value function by solving a convex optimization
problem.
Lemma 14.3 The projection of (v, Jv ) onto the epigraph epi J , is the optimizer
of the following problem, (x , J ):
min kx vk22 + kJ (z, x) + (x) Jv k22
z,x
s.t. (x, z) C
We can now use the fact that the maximizer of a convex function over a convex
set will always occur at an extreme point of the set to write the Hausdorff distance
as the maximum of the projection of the vertices of the polyhedron P onto the
epigraph, which can be done by testing a finite number of points, since a polyhedron
has a finite number of vertices.
Lemma 14.4 If P is a polyhedron, and P epi J , is a convex set, then the
Hausdorff distance is:
dH (P, epi J , ) = max(v,Jv )extreme(P ) kx (v) vk22 + kJ (v) + (x) Jv k22
where (x (v), J (v)) is the projection of (v, Jv ) onto the epigraph.
Algorithm 14.2 below will generate a sequence of outer polyhedral approximations of the epigraph that are monotonically decreasing in terms of the Hausdorff
distance. In each iteration of the algorithm, the Hausdorff distance is computed,
287
288
14
and the extreme point v of P at which the worst-case error occurs is removed from
P by adding a seperating hyperplane between the point v and the epigraph. The
addition of a seperating hyperplane ensures that the updated polyhedron remains
an outer approximation, and since the worst-case point is removed from the set,
we have that the approximation error is monotonically decreasing in the Hausdorff
distance. The algorithm terminates when all of the vertices of the P are contained
in the epigraph of J . Figure 14.6 illustrates the process of a single cycle of the
implicit double description algorithm. Figure 14.5 shows a progression of steps
until the termination criterion extreme(P ) epi J (x) is satisfied.
Figure 14.5 Several steps of the Double Description algorithm with the termination criterion extreme(P ) epi J (x) satisfied in the last step.
where wv,F (x) is a barycentric set of functions for the polytope RF . A method to
construct such a function will be described in the next sub-section.
14.4
289
The following theorem demonstrates that this approximate control law satisfies
the stability conditions of Theorem 14.1.
Theorem 14.3 If z(x) be defined as in (14.20) for a polyhedral approximation
P that satisfies the condition extreme(P ) epi J (x), then the conditions of the
stability theorem 14.1 are satisfied.
Proof:
J (x) J(
z (x), x)
X
vextreme(RF )
Convexity of J
Jv wv,F (x), if v RF
P epi J
(v,Jv )extreme(F )
min
(x,J)P
J (x) + (x)
P epi J ,
v
kv xk2
where v is the area of the polytope y [V 1xT ]y 0, (v x)T y = 1 ; i.e. the
area of the facet of the polar
P dual of S {x} corresponding to the vertex v x. The
function wv (x) = bv (x)/ v bv (x) is barycentric over the polytope S.
The areas of the facets of the polar duals v can be pre-computed offline and
stored. If there are d + 1 facets incident
v (i.e. v is simplical),
with the vertex
then the area of the polar facet is det a0 ad+1 , where {a0 , . . . , ad+1 } are
the normals of the incident facets. If the vertex is not simplical, then the area can
be easily computed by perturbing the incident facets [276]. Such computation is
straightforward because both the vertices and halfspaces of each region are available
due the double-description representation.
290
14.4.3
14
Second-Order Interpolants
This section summarizes the hierarchical grid-based approach proposed in [261].
The main idea is to partition the space into a grid, which is then extremely fast
to evaluate online. We first introduce an interpolation method based on secondorder interpolants and demonstrate that it is barycentric. We then discuss a sparse
hierarchical gridding structure, which allows for extremely sparse storage of the
data and rapid evaluation of the function.
Our goal is to define a set of basis functions, one for each sampled vertex of
the grid, whose sum is barycentric within each hypercube of the grid. We begin
by defining the one-dimensional scaling function (hat function), with support on
[1, 1] as:
(
1 |x| , if x [1, 1]
(x) =
0 ,
otherwise
For a particular grid point v, with grid width of w, we can extend the hat function
by scaling and shifting:
xv
v,w (x) =
w
We extend the function to d-dimensions by simply taking the tensor product:
v,w (x) =
d
Y
v[i],w (x[i]) ,
i=1
where v[i] and x[i] are the ith dimensions of the vectors v and x respectively.
The resulting basis function 0,1 (x) is illustrated in two dimensions in Figure 14.7.
Figure 14.7 Plot of the second order interpolant in 2D centered at zero, with
a width of one.
where w is the width of the grid, and G is the set of grid vertices.
Lemma 14.6 Let R be the hypercube defined by
R=
w
(B 1) v
2
14.5
Example
291
14.5
Example
In this section, we demonstrate and compare the three approximation algorithms
introduced above on a simple example. This comparison should not be generalized
to other problems, since the performance of any given approximation method will
depend strongly on the problem being considered.
Consider the following system:
1 1
1
x(k + 1) =
x(k) +
u(k)
0 1
0.5
with state and input constraints:
x X = {x | kxk 5}
u U = {u | kuk 1}
We approximate the control law resulting for a standard quadratic-cost MPC problem with a horizon of 10 and weighting matrices taken to be
0.1 0
Q=
R=1
0 2
The MPC problem formulation includes a terminal set and terminal weight, which
are computed as discussed in Chapter 13 as the maximum invariant set for the
system when controlled with the optimal LQR control law.
The optimal value function J for this problem can be seen in Figure 14.8, along
with the approximation target J (x) + (x) for (x) = x Qx.
Figure 14.8 Optimal cost J (x) (blue), and stability function J (x) +
(x)(red)
292
14
Figure 14.9 shows the evolution of the approximation algorithm using a triangulation approach, while Figure 14.16 shows the final approximate value function
and resulting approximate control law. Figure 14.19 shows the same evolution, but
for the double description/outer approximation approach. One can see that far
fewer sample points are required to reach a stability certificate.
The example has been approximated using the second-order interpolant method
until the stability certification was achieved. Figure 14.22 shows the resulting nodal
partition of the state space. Note that since it would take an infinite number of
hypercubes to approximate a non-orthogonal polytope, a soft-constrained version
of the problem is taken in the domain outside the feasible set (although the stability certificate is valid only within the feasible region). One can see the resulting
approximated value function, the approximate control law, and the resulting error
in Figures 14.24 and 14.23 respectively.
Table 14.1 shows the required online data storage and computation time for the
various methods introduced in this chapter for the studied example, as well as the
average and worst-case approximation errors (based on samples). One can see that
for this small example, the second-order interpolants are clearly the best solution.
However, this should not be taken as a clear indicator, or a recommendation for
the method beyond the others, since this would require a thorough computational
comparison of the various methods (or their variants in the literature) on a wide
range of practical problems.
Table 14.1 Comparison of the approximation methods introduced in Section 14.4
Triangulation
Polyhedral approx
Second-order interpolants
Data
(numbers)
120
720
187
FLOPS
1,2271
752
26
Approx error k
u(x) u (x)k
(avg)
(max)
3.3%
24.6%
1.1%
8.3%
Figure 14.9 Error function V (top). Points marked are the vertices with
worst-case fit. Partition (triangulation) of the sampled points (middle).
Upper-bound on approximate value function (convex hull of sample points)
(bottom).
14.5
Example
293
294
14
Chapter 15
15.1
296
15
In Algorithm 15.1, step (2) is critical and it is the only step whose efficiency can
be improved. A simple implementation of step (2) would require searching for the
polyhedral region that contains the state x(t) as in the following algorithm:
Algorithm 15.2
r
input: Current state x(t) and polyhedral partion {Pi }N
i=1 of the control law (15.1)
output: Index j of the polyhedron Pj in the control law (15.1) containing the current
state x(t)
i = 0, notfound=1;
while i N r and notfound
j = 0, stillfeasible=1
while j Nci and stillfeasible=1
if Hji x(t) > Kji then stillfeasible=0
else j = j + 1
end
15.1
297
In Algorithm 15.2 Hji denotes the j-th row of the matrix H i , Kji denotes the
j-th element of the vector K i and Nci is the number of constraints defining the
i-th polyhedron Pi . Algorithm 15.2 requires the storage of all polyhedra Pi , i.e.,
(n + 1)NC real numbers (n numbers for each row of the matrix H i plus one number
P r i
for the corresponding element in the matrix K i ), NC , N
i=1 Nc , and in the worst
case (the state is contained in the last region of the list) it will give a solution after
nNC multiplications, (n 1)NC sums and NC comparisons.
By using the properties of the value function, in this section we show how
Algorithm 15.2 can be replaced by more efficient algorithms that avoid storing the
polyhedral regions Pi , i = 1, . . . , N r , therefore reducing significantly the storage
demand, and that have a smaller computational complexity.
In the following we will distinguish between optimal control based on LP and
optimal control based on QP.
15.1.1
J (x) = T i x + V i , x Pi , i = 1, . . . , N r .
(15.2)
By exploiting the convexity of the value function the storage of the polyhedral
regions Pi can be avoided. From the equivalence of the representations of PWA
convex functions (see Section 3.2.5), the function J (x) in equation (15.2) can be
represented alternatively as
o
n
r
J (x) = max T i x + V i , i = 1, . . . , N r for x X = N
i=1 Pi .
(15.3)
(15.4)
x Pj T j x + V j = max T i x + V i , i = 1, . . . , N r .
Therefore, instead of searching for the polyhedron j that contains the point x
via Algorithm 15.2, we can just store the value function and identify region j by
searching for the maximum in the list of numbers composed of the single affine
298
15
J (x)
T4 x + V 4
T1 x + V 1
T3 x + V 3
T2 x + V 2
P1
P2
P3
P4
Figure 15.1 Example for Algorithm 15.3 in one dimension: For a given point
x P3 (x = 5) we have J (x) = max(T 1 x + V 1 , . . . , T 4 x + V 4 ).
Algorithm 15.3
The search Algorithm 15.3 requires the storage of (n + 1)N r real numbers
and will give a solution after nN r multiplications, (n 1)N r sums, and N r 1
comparisons. In Table 15.1 we compare the complexity of Algorithm 15.3 against
Algorithm 15.2 in terms of storage demand and number of flops. Algorithm 15.3
will outperform Algorithm 15.2 since typically NC N r .
Table 15.1 Complexity comparison of Algorithm 15.2 and Algorithm 15.3
15.1.2
Algorithm 15.2
(n + 1)NC
2nNC
Algorithm 15.3
(n + 1)N r
2nN r
15.1
299
(15.5)
f (x) = fi (x) , Ai x + B i , if x Pi
is called a descriptor function if
Ai 6= Aj , j Ci , i = 1, . . . , Nr .
(15.6)
Theorem 15.1 Let f (x) be a PWA descriptor function on the polyhedral partition
r
{Pi }N
i=1 .
i
Let Oi (x) RNc be a vector associated with region Pi , and let the j-th element
of Oi (x) be defined as
+1
fi (x) > fCi (j) (x)
Oji (x) =
(15.7)
1
fi (x) < fCi (j) (x).
Then Oi (x) has the following properties:
(i) Oi (x) = S i = const, x Pi , i = 1, . . . , Nr .
(ii) Oi (x) 6= S i , x
/ Pi , i = 1, . . . , Nr .
Proof: Let F = Pi PCi (j) be the common facet of Pi and PCi (j) . Define the
linear function
gji (x) = fi (x) fCi (j) (x).
(15.8)
From the continuity of f (x) it follows that gji (x) = 0, x F . As Pi and PCi (j) are
disjoint convex polyhedra and Ai 6= ACi (j) it follows that gji (i ) > 0 (or gji (i ) < 0,
but not both) for any interior point i of Pi . Similarly for any interior point Ci (j)
of PCi (j) we have gji (Ci (j) ) < 0 (or gij (i ) > 0, but not both). Consequently,
gji (x) = 0 is the separating hyperplane between Pi and PCi (j) .
(i) Because gji (x) = 0 is a separating hyperplane, the function gji (x) does not
change its sign for all x Pi , i.e., Oji (x) = sij , x Pi with sij = +1 or sij = 1.
The same reasoning can be applied to all neighbors of Pi to get the vector S i =
i
{sij } RNc .
300
15
f (x)
f4 (x)
f3 (x)
f2 (x)
f1 (x)
P1
P2
P3
P4
Figure 15.2 Example for Algorithm 15.4 in one dimension: For a given point
x P2 (x = 4) we have O2 (x) = [1 1] = S2 , while O1 (x) = 1 6= S1 = 1,
O3 (x) = [1 1] 6= S3 = [1 1] , O4 (x) = 1 6= S4 = 1.
(ii) x
/ Pi , j Ci such that Hji x > Kji . Since gji (x) = 0 is a separating
hyperplane Oji (x) = sij .
Equivalently, Theorem 15.1 states that
x Pi Oi (x) = S i ,
(15.9)
which means that the function Oi (x) and the vector S i uniquely characterize Pi .
Therefore, to check on-line if the polyhedral region i contains the state x it is
sufficient to compute the binary vector Oi (x) and compare it with S i . Vectors
S i are calculated off-line for all i = 1, . . . , N r , by comparing the values of fi (x)
and fCi (j) (x) for j = 1, . . . , Nci , for a point x belonging to Pi , for instance, the
Chebychev center of Pi .
In Figure 15.2 a one dimensional example illustrates the procedure with N r = 4
regions. The list of neighboring regions Ci and the vector S i can be constructed
by simply looking at the figure: C1 = {2}, C2 = {1, 3}, C3 = {2, 4}, C4 = {3},
S 1 = 1, S 2 = [1 1] , S 3 = [1 1] , S 4 = 1. The point x = 4 is in region 2 and
we have O2 (x) = [1 1] = S 2 , while O3 (x) = [1 1] 6= S 3 , O1 (x) = 1 6= S 1 ,
O4 (x) = 1 6= S 4 . The failure of a match Oi (x) = S i provides information on a
good search direction(s). The solution can be found by searching in the direction
where a constraint is violated, i.e., one should check the neighboring region Pj for
which Oji (x) 6= sij .
The overall procedure is composed of two parts:
1. (off-line) Construction of the PWA function f (x) in (15.5) satisfying (15.6)
and computation of the list of neighbors Ci and the vector S i ,
2. (on-line) Execution of the following algorithm
Algorithm 15.4
15.1
301
input: Current state x, the list of neighboring regions Ci and the vectors S i
output: Index i of the polyhedron Pj containing the current state x(t) in the control
law (15.1)
let i = 1, notfound=1;
while notfound
end
compute Oi (x)
if Oi (x) = S i then notfound=0
else i = Ci (q), where Oqi (x) 6= siq .
Algorithm 15.4 does not require the storage of the polyhedra Pi , but only
the storage of one linear function fi (x) per polyhedron, i.e., N r (n + 1) real numbers and the list of neighbors Ci which requires NC integers. In the worst case,
Algorithm 15.4 terminates after N r n multiplications, N r (n 1) sums and NC
comparisons.
In Table 15.2 we compare the complexity of Algorithm 15.4 against the standard
Algorithm 15.2 in terms of storage demand and number of flops.
Remark 15.1 Note that the computation of Oi (x) in Algorithm 15.4 requires the
evaluation of Nci linear functions, but the overall computation never exceeds N r linear
function evaluations. Consequently, Algorithm 15.4 will outperform Algorithm 15.2,
since typically NC N r .
Algorithm 15.2
(n + 1)NC
2nNC
Algorithm 15.4
(n + 1)N r
(2n 1)N r + NC
Now that we have shown how to locate polyhedron in which the state lies by
using a PWA descriptor function, we need a procedure for the construction of such
a function.
The image of the descriptor function is the set of real numbers R. In the
following we will show how the descriptor function can be generated from a vectorvalued function m : Rn Rs . This general result will be used in the following
subsections.
Definition 15.2 (Vector valued PWA descriptor function) A continuous vectorvalued PWA function
i , if x Pi ,
m(x) = Ai x + B
(15.10)
is called a vector-valued PWA descriptor function if
Ai 6= Aj , j Ci , i = 1, . . . , Nr ,
i Rs .
where Ai Rsn , B
(15.11)
302
15
Theorem 15.2 Given a vector-valued PWA descriptor function m(x) defined over
r
a polyhedral partition {Pi }N
i=1 it is possible to construct a PWA descriptor function
f (x) over the same polyhedral partition.
Proof: Let Ni,j be the null-space of (Ai Aj ) . Since by the definition Ai Aj 6=
0 it follows that Ni,j is not full-dimensional, i.e., Ni,j Rs1 . Consequently, it
is always possible to find a vector w Rs such that w(Ai Aj ) 6= 0 holds for all
i = 1, . . . , Nr and j Ci . Clearly, f (x) = w m(x) is then a valid PWA descriptor
function.
As shown in the proof of Theorem 15.2, once we have a vector-valued PWA
descriptor function, practically any randomly chosen vector w Rs is likely to be
satisfactory for the construction of PWA descriptor function. But, from a numerical
point of view, we would like to obtain a w that is as far away as possible from the
null-spaces Ni,j . We show one algorithm for finding such a vector w.
For a given vector-valued PWA descriptor function we form a set of vectors
ak Rs , kak k = 1, k = 1, . . . , NC /2, by choosing and normalizing one (and only
one) nonzero column from each matrix (Ai Aj ), j Ci , i = 1, . . . , Nr . The
vector w Rs satisfying the set of equations w ak 6= 0, k = 1, . . . , NC /2, can then
be constructed by using the following algorithm. Note that the index k goes to
NC /2 since the term (Aj Ai ) is the same as (Ai Aj ) and thus there is no need
to consider it twice.
Algorithm 15.5
input: Vectors ai Rs , i = 1, . . . , N .
output: The vector w Rs satisfying the set of equations w ai 6= 0, i = 1, . . . , N
let w [1, . . . , 1] , R 1
while k NC /2
d w ak
if 0 d R then w w + 12 (R d)ak , R 12 (R + d)
if R d < 0 then w w 12 (R + d)ak , R 12 (R d)
end
15.1
303
aTi x 0
aTi x 0
Rd
J (x) = qi (x) , x Qi x + T i x + V i , if x Pi , i = 1, . . . , N r .
(15.12)
In Section 7.3.4 we have proven that for non-degenerate problems the value
function J (x) is a C (1) function. We can obtain a vector-valued PWA descriptor
function by differentiating J (x). We first need to introduce the following theorem.
Theorem 15.3 ([19]) Assume that the CFTOC (12.9) problem leads to a nondegenerate mp-QP (12.31). Consider the value function J (x) in (15.12) and let
CRi , CRj be the closure of two neighboring critical regions corresponding to the
set of active constraints Ai and Aj , respectively, then
Qi Qj 0 or Qi Qj 0 and Qi 6= Qj
(15.13)
Qi Qj 0 iff Ai Aj .
(15.14)
and
Theorem 15.4 Consider the value function J (x) in (15.12) and assume that
the CFTOC (12.9) problem leads to a non-degenerate mp-QP (12.31). Then the
gradient m(x) , J (x), is a vector-valued PWA descriptor function.
Proof: From Theorem 7.9 we see that m(x) is continuous vector-valued PWA
function, while from equation (15.12) we get
m(x) , J (x) = 2Qi x + Ti
(15.15)
Since from Theorem 15.3 we know that Qi 6= Qj for all neighboring polyhedra,
it follows that m(x) satisfies all conditions for a vector-valued PWA descriptor
function.
Combining results of Theorem 15.4 and Theorem 15.2 it follows that by using
Algorithm 15.5 we can construct a PWA descriptor function from the gradient of
the value function J (x).
304
15
(15.16)
where F i Rsn and Gi Rs . We will assume that Pi are critical regions (as
defined in Section 7.1.2). Before going further we need the following lemma.
Lemma 15.1 Consider the state feedback solution (15.16) of the CFTOC problem (12.9) and assume that the CFTOC (12.9) leads to a non-degenerate mpQP (12.31). Let Pi , Pj be two neighboring polyhedra, then F i 6= F j .
Proof: The proof is a simple consequence of Theorem 15.3. As in Theorem 15.3,
without loss of generality we can assume that the set of active constraints Ai
associated with the critical region Pi is empty, i.e., Ai = . Suppose that the
optimizer is the same for both polyhedra, i.e., [F i Gi ] = [F j Gj ]. Then, the cost
functions qi (x) and qj (x) are also equal. From the proof of Theorem 15.3 this
implies that Pi = Pj , which is a contradiction. Thus we have [F i Gi ] 6= [F j Gj ].
Note that F i = F j cannot happen since, from the continuity of U0 (x), this would
imply Gi = Gj . Consequently we have F i 6= F j .
From Lemma 15.1 and Theorem 15.2 it follows that an appropriate PWA descriptor function f (x) can be calculated from the gradient of the optimizer U (x)
by using Algorithm 15.5.
Remark 15.2 Note that even if we are implementing a receding horizon control strategy, the construction of the PWA descriptor function is based on the full optimization
vector U (x) and the corresponding matrices F i and Gi .
Remark 15.3 In some cases the use of the optimal control profile U (x) for the
construction of descriptor function f (x) can be extremely simple. If there is a row
r, r m (m is the dimension of u) for which (F i )r 6= (F j )r , i = 1 . . . , N r , j Ci ,
15.1
15.1.3
305
Example
As an example, we compare the performance of Algorithm 15.2, 15.3 and 15.4 on
CFTOC problem for the discrete-time system
0
0
0
x(t) + 0 u(t)
x(t + 1) =
0
0
2
0
0
(15.17)
0
0
0.5
0
0
1
u
s4
sampled at Ts = 1, subject to the input constraint
(15.18)
y=
1 u(t) 1
(15.19)
10 y(t) 10.
(15.20)
CFTOC based on LP
To regulate (15.17), we design a receding horizon controller based on the optimization problem (13.6), (13.7)) where p = , N = 2, Q = diag{5, 10, 10, 10}, R = 0.8,
P = 0, Xf = R4 . The PPWA solution of the mp-LP problem comprises 136 regions.
In Table 15.3 we report the comparison between the complexity of Algorithm 15.2
and Algorithm 15.3 for this example.
The average on-line evaluation effort of the PPWA solution for a set of 1000
random points in the state space is 2259 flops (Algorithm 15.2), and 1088 flops
(Algorithm 15.3). We note that the solution using Matlabs LP solver (function
linprog.m with interior point algorithm and LargeScale set to off) takes 25459
flops on average.
Table 15.3 Complexity comparison of Algorithm 15.2 and Algorithm 15.3
for the example in Section 15.1.3
Algorithm 15.2
5690
9104
2259
Algorithm 15.3
680
1088
1088
CFTOC based on QP
To regulate (15.17), we design a receding horizon controller based on the optimization problem (13.6), (13.7)) where N = 7, Q = I, R = 0.01, P = 0, Xf = R4 .
The PPWA solution of the mp-QP problem comprises 213 regions. We obtained
306
15
a descriptor function from the value function and for this example the choice of
w = [1 0 0 0] is satisfactory. In Table 15.4 we report the comparison between the
complexity of Algorithm 15.2 and Algorithm 15.4 for this example.
The average computation effort of the PPWA solution for a set of 1000 random points in the state space is 2114 flops (Algorithm 15.2), and 175 flops (Algorithm 15.4). The solution of the corresponding quadratic program with Matlabs
QP solver (function quadprog.m and LargeScale set to off) takes 25221 flops on
average.
Table 15.4 Complexity comparison of Algorithm 15.2 and Algorithm 15.4
for the example in Section 15.1.3
15.1.4
Algorithm 15.2
9740
15584
2114
Algorithm 15.4
1065
3439
175
Literature Review
The problem considered in this Chapter has been approached by several other
researchers. For instance in [266] the authors propose to organize the controller
gains of the PWA control law on a balanced search tree. By doing so, the search for
the region containing the current state has on average a logarithmic computation
complexity although it is less efficient in terms of memory requirements. At the
expense of the optimality a similar computational complexity can be achieved with
the approximative point location algorithm described in [153].
The comparison of the proposed algorithms with other efficient solvers in the
literature [19, 52, 246, 99, 203, 206, 275, 58, 23] requires the simultaneous analysis
of several issues such as speed of computation, storage demand and real time code
verifiability.
15.2
PN 1
xN P xN + k=0 xk Qxk + uk Ruk
minU0
subj. to xk+1 = Axk + Buk , k = 0, . . . , N 1
xk X , uk U, k = 0, . . . , N 1
xN Xf
x0 = x(0),
(15.21)
15.2
307
minU0
subj. to
(15.22)
where U0 U N , U . . . U
= min (H) .
k = 0, . . . , N 1 ,
(15.24)
308
15
actuator constraints, (4.5) or (4.6) can be applied to solve (15.21). Even if the
input set U is an arbitrary polytope, there is the possibility of pre-computing an
explicit solution to (15.24) by means of multi-parametric programming with uf,k
as the parameter.
For some control problems, the feasible set of inputs U originates from a simple set that is being translated, uniformly scaled and/or rotated. In this case,
projection is as easy as on the original simple set according to the next lemma.
Lemma 15.2 (Projection on Sets after Translation, Rotation and Scaling)
Let K Rn be a closed convex set and K its associated projection operator. Projection of point z Rn on the set
b = {z Rn | z = W y + c, y K}
K
The separability property of the feasible set U N was used for simplifying the
projection. In case of state constraints, this beneficial property is lost, rendering
the projection a hard problem. So, the gradient methods in the presented form
cannot be used to solve MPC problems with state constraints in the primal domain,
but need to be applied in the dual domain. We study this case in the following.
is non-trivial in general, even if the convex sets U, X and Xf allow efficient projections individually. The inherent complication comes from the intersection of these
sets with the affine set determined by the state update equation.
For the application of gradient methods to the input- and state-constrained
MPC problem (15.21) in the dual domain (cf. Section 4.3.1), we rewrite it as the
convex multi-parametric quadratic program
f (x(0)) , min f (z) = z Hz
s.t. Gz = wx(0)
z X . . . X Xf U . . . U ,
(15.25)
15.3
In
0
0
0 0
A
In 0
0
B 0 0
..
.
..
.
.
.
.
.
.
.
. .
.
.
.
.
0 B
(15.27)
G , 0
.
.
.
.
.
..
.. 0
..
..
..
A In
0
0
0
A
In 0
0 B
Unlike in the input constrained case, we do not eliminate the states from the optimization vector. As above, by a separability argument, the projections necessary
can be carried out on the individual sets X , U and Xf . The Lipschitz constant of
the dual function can be calculated as Ld = kAH 1/2 k2 , as given in (4.45). With
these observations, Algorithms 4.1 or 4.2 can be applied to solve (15.21) with input
and state constraints in the dual domain.
15.3
309
310
15
Chapter 16
16.1
Problem Formulation
Consider the linear uncertain system
x(t + 1) = A(wp (t))x(t) + B(wp (t))u(t) + Ewa (t)
(16.1)
where x(t) Rn and u(t) Rm are the state and input vectors, respectively,
subject to the constraints
x(t) X , u(t) U, t 0.
(16.2)
312
16
The sets X Rn and U Rm are polytopes. Vectors wa (t) Rna and wp (t) Rnp
are unknown additive disturbances and parametric uncertainties, respectively. The
disturbance vector is
w(t) = [wa (t) , wp (t) ] W Rnw
(16.3)
i=1
where Ai Rnn and B i Rnm are given matrices for i = 0 . . . , np and wcp,i is
p,n
the i-th component of the vector wp , i.e., wp = [wcp,1 , . . . , wc p ].
Robust finite-time optimal control problems can be formulated by optimizing
over Open-Loop policies or Closed-Loop policies. We will discuss the two cases
next.
Open-Loop Predictions
Define the cost function as
J0 (x(0), U0 ) ,
h
i
P 1
q(x
,
u
)
JW p(xN ) + N
k
k
k=0
(16.5)
PN 1
where the operation JW evaluates the cost p(xN ) + k=0 q(xk , uk ) for the set of
uncertainties {w0 , . . . , wN 1 } W W and the input sequence U0 . The
following options are typically considered.
Nominal Cost.
In the simplest case, the objective function is evaluated for a single disturp
a
N
0p , . . . , w
N
bance profile w
0a , . . . , w
1 , w
1 :
JW =
h
i
PN 1
p(xN ) + k=0 q(xk , uk )
x0 = x(0)
where
a
w
=w
ka , wkp = w
kp ,
k
k = 0, . . . , N 1.
(16.6)
Expected Cost.
Another option is to consider the probability density function f (w) of the
disturbance w:
Z
f (w)dw.
Probability [w(t) W] = 1 =
wW
16.1
Problem Formulation
313
In this case we consider the expected cost over the admissible disturbance
set.
i
h
PN 1
JW = Ew0 ,...,wN 1 p(xN ) + k=0 q(xk , uk )
x0 = x(0)
where
p
wka W a , wk W p ,
k = 0, . . . , N 1.
(16.7)
x0 = x(0)
where
a
w
W a , wkp W p ,
k
k = 0, . . . , N 1.
(16.8)
If the 1-norm or -norm is used in the cost function of problems (16.6), (16.7)
and (16.8), then we set
p(xN ) = kP xN kp , q(xk , uk ) = kQxk kp + kRuk kp
(16.9)
(16.10)
Note that in (16.6), (16.7) and (16.8) xk denotes the state vector at time k obtained
by starting from the state x0 = x(0) and applying to the system model
xk+1 = A(wkp )xk + B(wkp )uk + Ewka
a
the input sequence u0 , . . . , uk1 and the disturbance sequences wa , {w0a , . . . , wN
1 },
p
p
wp , {w0 , . . . , wN 1 }.
Consider the optimal control problem
J0 (x(0)) ,
minU0
subj. to
J0 (x(0), U0 )
xk X , uk U
k = 0, . . . , N 1
xN Xf
x0 = x(0)
wka W a , wkp W p
k = 0, . . . , N 1
(16.11)
314
16
16.1
Problem Formulation
315
whole horizon {u0 , . . . , uN 1 }, reveals his plan to the opponent W , who dep
a
cides on his actions next {w0a , w0p , . . . , wN
1 , wN 1 } by solving (16.8). By
solving (16.8), (16.11), the player U selects the action corresponding to the
smallest worst-case cost.
Closed-Loop Predictions
The constrained robust optimal control problem based on closed-loop predictions
(CROC-CL) is defined as follows:
J0 (x(0)) ,
min0 (),...,N 1 ()
subj. to
J0 (x(0), U0 )
xk X , uk U
uk = k (xk )
k
= 0, . . . , N 1
x
Xf
x0 = x(0)
wka W a , wkp W p
k = 0, . . . , N 1
(16.13)
In (16.13) we allow the same cost functions as in (16.6), (16.7) and (16.8).
In problem (16.13) we look for a set of time-varying feedback policies 0 (), . . . , N 1 ()
which minimizes the performance index JW and generates input sequences 0 (x0 ), . . . , N 1 (xN 1 )
satisfying state and input constraints for all possible disturbance realizations.
We can formulate the search for the optimal policies 0 (), . . . , N 1 () as a
dynamic program.
Jj (xj ) , min Jj (xj , uj )
uj
xj X , uj U
wja W a , wjp W p
subj. to
A(wjp )xj + B(wjp )uj + Ewja Xj+1
(16.14)
(16.15)
JN
(xN ) = p(xN )
XN = Xf ,
(16.16a)
(16.16b)
JW = q(xj , uj ) + Jj+1
(A(wjp )xj + B(w
jp )uj + E wja )
(16.18)
316
16
Expected Cost.
JW =
(A(wp )x + B(wp )u + Ewa )
Ewja W a ,wjp W p q(xj , uj ) + Jj+1
j
j
j
j
j
(16.19)
(A(wp )x + B(wp )u + Ewa )
maxwja W a ,wjp W p q(xj , uj ) + Jj+1
j
j
j
j
j
(16.20)
If the 1-norm or -norm is used in the cost function of problems (16.18), (16.19)
and (16.20), then we set
p(xN ) = kP xN kp , q(xk , uk ) = kQxk kp + kRuk kp
(16.21)
(16.22)
The reason for including constraints (16.14) in the minimization problem and
not in the inner problem (16.15) is that in (16.15) wja and wjp are free to act
regardless of the state constraints. On the other hand in (16.14), the input uj has
the duty to keep the state within the constraints (16.14) for all possible disturbance
realization. As a result of solving problem (16.14) at step j of the dynamic program,
we obtain the function uj (xj ), i.e. the policy j ().
Again, the optimal control problem (16.14)(16.15) can be viewed as a game
between two players: the controller U and the disturbance W . This time the payer
U predicts that the game is going to be played as follows. At the generic time j
player U observes xj and responds with uj = j (xj ). Player W observes (xj , uj )
and responds with wja and wjp . Therefore, regardless of the cost function in (16.15)
the player U counteracts any feasible disturbance realization wj with a feedback
controller j (xj ). This prediction model takes into account that the disturbance
and the controller play one move at a time.
The game is played differently depending on the cost function JW .
In problem (16.14) and (16.18), among all the feedback policies uj = j (xj )
which robustly satisfy the constraints, the player U chooses the one which
minimizes a cost for a guessed action w
j played by W .
In problem (16.14) and (16.19), among all the feedback policies uj = j (xj )
which robustly satisfy the constraints, the player U chooses the one which
minimizes the expected cost at time j.
The optimal control problem (16.14)(16.20) is somehow more involved and
can be viewed as the solution of a zero-sum dynamic game. The player U
plays first. At the generic time j player U observes xj and responds with
uj = j (xj ). Player W observes (xj , uj ) and responds with the wja and wjp
which lead to the worst case cost. The player W will always play the worst
case action only if it has knowledge of both xj and uj = j (xj ). In fact,
wja and wjp in (16.20) are a function of xj and uj . If U does not reveal his
16.1
Problem Formulation
317
action to player W , then we can only claim that the player W might play
the worst case action. Problem (16.14)(16.20) is meaningful in both cases.
Robust constraint satisfaction and worst case minimization will always be
guaranteed.
Example 16.1 Consider the system
xk+1 = xk + uk + wk
(16.23)
where x, u and w are state, input and disturbance, respectively. Let uk {1, 0, 1}
and wk {1, 0, 1} be the feasible input and disturbance. Here {1, 0, 1} denotes
the set with three elements: -1, 0 and 1. Let x(0) = 0 be the initial state. The
objective for player U is to play two moves in order to keep the state x2 at time 2
in the set [1, 1]. If U is able to do so for any possible disturbance, then he will win
the game.
The open-loop formulation (16.11) is infeasible. In fact, in open-loop U can choose
from nine possible sequences: (0,0), (1,1), (-1,-1), (-1,1) (1,-1), (-1,0), (1,0), (0,1)
and (0,-1). For any of those sequences there will always exist a disturbance sequence
w0 , w1 which will bring x2 outside the feasible set [-1,1].
The closed-loop formulation (16.14)(16.20) is feasible and has a simple feasible solution: uk = xk . In this case system (16.23) becomes xk+1 = wk and x2 = w1 lies
in the feasible set [1, 1] for all admissible disturbances w1 . The optimal feedback
policy might be different from uk = xk and depends on the choice of the objective
function.
16.1.1
if
(16.24)
Hki and Kki in equation (16.24) are the matrices describing the i-th polyhedron
CRki = {x Rn : Hki x Kki } inside which the feedback optimal control law
u (k) at time k has the affine form Fki x + gki .
The optimal solution has the form (16.24) in the following cases.
Nominal or worst-case performance index based on 1-, -norm,
CROC-OL with no uncertainty in the dynamics matrix A (i.e., A() = A)
CROC-CL
Nominal performance index based on 2-norm,
CROC-OL with no uncertainty in the dynamics matrix A (i.e., A() = A)
318
16
16.2
Feasible Solutions
As in the nominal case (Section 12.2), there are two ways to rigorously define and
compute the robust feasible sets: the batch approach and the recursive approach.
While for systems without disturbances, both approaches yield the same result, in
the robust case the batch approach provides the feasible set XiOL of CROC with
open-loop predictions and the recursive approach provides the feasible set Xi of
CROC with closed-loop predictions. From the discussion in the previous sections,
clearly we have XiOL Xi . We will detail the batch approach and the recursive
approach next and at the end of the section we will show how to modify the batch
approach in order to compute Xi .
Thus for any initial state xi XiOL there exists a feasible sequence of inputs
Ui , [ui , . . . , uN 1 ] which keeps the state evolution in the feasible set X at future
time instants k = i + 1, . . . , N 1 and forces xN into Xf at time N for all feasible
disturbance sequences wka W a , wkp W p , k = i, . . . , N 1. Clearly XNOL = Xf .
Next we show how to compute XiOL for i = 0, . . . , N 1. Let the state and input
constraint sets X , Xf and U be the H-polyhedra Ax x bx , Af x bf , Au u bu ,
respectively. Assume that the disturbance sets are represented in terms of their
vertices: W a = conv{wa,1 , . . . , wa,nW a } and W p = conv{wp,1 , . . . , wp,nW p }. Define
16.2
Feasible Solutions
319
(16.25)
Au uk bu , k = i, . . . , N 1
State Constraints
(16.27)
(16.28)
In general, the constraint (16.28) is non-convex in the disturbance sequences
p
k1
a
wia , . . . , wk1
, wip , . . . , wk1
because of the product j=i
A(wjp ).
Assume that there is no uncertainty in the dynamics matrix A, i.e. A() = A.
In this case, since B() is an affine function of wp (cf. equation (16.4)) and since the
composition of a convex constraint with an affine map generates a convex constraint
(Section 1.2), we can use Lemma 11.1 to rewrite constraints (16.26) at time k as
Pk1
p
a
)uk1l + Ewk1l
) bx ,
Ax Aki1 xi + l=i Ali1 (B(wk1l
p
(16.29)
for all wa {wa,i }nW a , wp {wp,i }nW , j = i, . . . , k 1,
j
i=1
i=1
k = i, . . . , N 1.
Remark 16.3 When only additive disturbances are present (i.e., np = 0), vertex
enumeration is not required and a set of linear programs can be used to transform
the constraints (16.26) into a smaller number of constraints than (16.29) as explained
in Lemma 11.2.
Once the matrices Gi , Ei and Wi have been computed, the set XiOL is a polyhedron and can be computed by projecting the polyhedron Pi in (16.25) on the xi
space.
320
16
XN
(16.30)
The definition of Xi in (16.30) is recursive and it requires that for any feasible
initial state xi Xi there exists a feasible input ui which keeps the next state
A(wip )x + B(wip )u + Ewia in the feasible set Xi+1 for all feasible disturbances wia
W a , wip W p .
Initializing XN to Xf and solving (16.30) backward in time yields the feasible set
X0 for the CROC-CL (16.14)(16.16) which, as shown in example 16.1, is different
from X0OL .
Let Xi be the H-polyhedron AXi x bXi . Then the set Xi1 is the projection
of the following polyhedron
Au
0
bu
Wa
Wp
ui1 +
xi1
for all wia {wa,i }ni=1
0
Ax
bx
, wip {wp,i }ni=1
p
p
AXi B(wi )
AXi A(wi )
bXi Ewia
(16.31)
on the xi1 space. (Note that we have used Lemma 11.1 and imposed state constraints at all the vertices of the sets W a W p .)
Remark 16.4 When only additive disturbances are present (i.e., np = 0), vertex
enumeration is not required and a set of linear programs can be used to transform the
constraints in (16.30) into a smaller number of constraints than (16.31) as explained
in Lemma 11.2.
The backward evolution in time of the feasible sets Xi enjoys the properties
described by Theorems 12.1 and 12.2 for the nominal case. In particular if a
robust controlled invariant set is chosen as terminal constraint Xf , then set Xi
grows as i becomes smaller and stops growing when it becomes the maximal robust
stabilizable set.
Theorem 16.1 Let the terminal constraint set Xf be a robust control invariant
subset of X . Then,
1. The feasible set Xi , i = 0, . . . , N 1 is equal to the (N i)-step robust
stabilizable set:
Xi = KN i (Xf , W)
2. The feasible set Xi , i = 0, . . . , N 1 is robust control invariant and contained
within the maximal robust control invariant set:
Xi C
16.2
Feasible Solutions
321
Xi = K (Xf , W) if i N N
be written as
1
Ul = [u0 , uj11 , u
j22 , . . . u
]
NN1
where the lower index denotes time, and the upper index is used to associate a
different input for every disturbance realization. For instance, at time 0 the extreme realizations of additive and parametric uncertainty are nW a . At time 1
there is one input for each realization and thus j1 {1, . . . , nW a }. At time 2
the extreme realizations of the disturbance sequence w0 , w1 are n2W a and therefore j2 {1, . . . , n2W a }. By repeating this argument we have jk {1, . . . , nkW a },
N 1
k = 1, . . . , N 1. In total we will
have nW a different control sequences Ul and
N 1
2
m 1 + nW a + nW a + + nW a control variables to optimize over.
Since the approach is computationally demanding, we prefer to present the main
idea through a very simple example rather than including all the tedious details.
Example 16.2 Consider the system
xk+1 = xk + uk + wk
(16.32)
where x, u and w are state, input and disturbance, respectively. Let uk [1, 1]
and wk [1, 1] be the feasible input and disturbance. The objective for player U
is to play two moves in order to keep the state at time three x3 in the set Xf = [1, 1].
Batch approach
We rewrite the terminal constraint as
x3 = x0 + u0 + u1 + u2 + w0 + w1 + w2 [1, 1]
for all w0 [1, 1], w1 [1, 1], w2 [1, 1]
(16.33)
322
16
(16.34)
(16.35)
which is infeasible and thus X0 = . For the sake of clarity we continue with
the procedure as if we did not notice that constraint (16.35) was infeasible.
The set X0OL is the projection on the x0 space of the polyhedron P0 of combined
feasible input and state constraints
P0 = {(u0 , u1 , u2 , x0 ) R4 :
1
1
0
0
0
0
1
1
0
0
1
1
0
0
1
1
0
0
0
0
1
1
1
1
u0
u1 +
u2
The projection will provide an empty set since the last two
spond to the infeasible terminal state constraint (16.35).
0
0
0
0
0
0
1
1
1
1
x0 1
1
2
(16.36)
constraints corre-
Recursive approach
For the recursive approach we have X3 = Xf = [1, 1]. We rewrite the terminal
constraint as
x3 = x2 + u2 + w2 [1, 1] for all w2 [1, 1]
(16.37)
(16.38)
1
1
1
1
1
0
0
u2 1
0
1 x2
0
1
(16.39)
which yields X2 = [1, 1]. Since X2 = X3 one can conclude that X2 is the
maximal controllable robust invariant set and X0 = X1 = X2 = [1, 1].
16.3
323
1,
1,
1,
1,
w2
w2
w2
w2
4 different control
{u0 , u
21 , u
42 }, cor1, w1 = 1, w2 },
= 1, w1 = 1, w2 },
[1, 1]
[1, 1]
[1, 1]
[1, 1]
(16.40)
(16.41)
The set X0 can be obtained by projecting the polyhedron (16.41) in the (x0 , u0 , u
11 , u
21 , u
12 , u
22 , u
32 , u
42 )space on the x0 space. This yields X0 = [1, 1], the same as in the recursive
approach.
16.3
if
x CR0i , i = 1, . . . , N0r
(16.42)
324
16
where the polyhedral sets CR0i , {H0i x k0i }, i = 1, . . . , N0r , are a partition of the
feasible set X0OL . Moreover f0 can be found by solving an mp-LP for cost (16.9)
and an mp-QP for cost (16.10). The same result holds for u (k) = fkOL (x(0)), k =
1, . . . , N 1.
Proof: The proof can be easily derived from the results in Chapter 12. The
only difference is that the constraints have to be first robustified as explained in
Section 16.2.
Theorem 16.3 There exists a state-feedback control law u (k) = fk (x(k)), fk :
Xk Rn U Rm , solution of the CROC-CL (16.14)(16.16), (16.18) with
cost (16.21) and k = 0, . . . , N 1 which is time-varying, continuous and piecewise
affine on polyhedra
fk (x) = Fki x + gki
if
x CRki , i = 1, . . . , Nkr
(16.43)
16.4
minz
maxwW f (z, x, w)
subj. to g(z, x, w) 0 w W
(16.44)
min,z
subj. to f (z, x, w
i ), i = 1, . . . , NW
g(z, x, w
i ) 0, i = 1, . . . , NW .
(16.45)
Proof: Easily follows from the fact that the maximum of a convex function
over a convex set is attained at an extreme point of the set, cf. also [249].
16.4
Corollary 16.1 If f is convex and piecewise affine in (z, x), i.e. f (z, x, w) =
maxi=1,...,nf {Li (w)z + Hi (w)x + Ki (w)} and g is linear in (z, x) for all w W,
g(z, x, w) = Lg (w)z + Hg (w)x + Kg (w) (with Lg (), Hg (), Kg (), Li (), Hi (),
Ki (), i = 1, . . . , nf , convex functions), then the min-max multiparametric problem (16.44) is equivalent to the mp-LP problem
J (x) =
min,z
subj. to Lj (w
i )z + Hj (wi )x + Kj (wi ), i = 1, . . . , NW , j = 1, . . . , nf
Lg (w
i )z + Hg (w
i )x Kg (wi ), i = 1, . . . , NW
(16.46)
Remark 16.5 As discussed in Lemma 11.2, in the case g(z, x, w) = g1 (z, x) + g2 (w),
the
second constraint in (16.45) can be replaced by g1 (z, x)
g , where g ,
(16.47)
and g2i (w) denotes the i-th component of g2 (w). Similarly, if f (z, x, w) = f1 (z, x) +
f2 (w), the first constraint in (16.45) can be replaced by f1 (z, x) + f, where
fi = max f2i (w).
wW
(16.48)
Clearly, this has the advantage of reducing the number of constraints in the multiparametric program from NW ng to ng for the second constraint in (16.45) and from
NW nf to nf for the first constraint in (16.45).
16.4.1
if
x CR0i , i = 1, . . . , N0r
(16.49)
where the polyhedral sets CR0i , {H0i x k0i }, i = 1, . . . , N0r , are a partition of the
feasible set X0OL . Moreover f0 can be found by solving an mp-LP. The same result
holds for u (k) = fkOL (x(0)), k = 1, . . . , N 1.
Pk1
p
a
Proof: Since xk = Ak x0 + k=0 Ai [B(wk1i
)uk1i + Ewk1i
] is a linear
p
p
a
a
a
p
function of the disturbances w , {w0 , . . . , wN 1 }, and w , {w0 , . . . , wN
1 } for a
fixed input sequence and x0 , the cost function in the maximization problem (16.8) is
convex and piecewise affine with respect to the optimization vectors wa and wp and
the parameters U0 , x0 . The constraints in (16.11) are linear in U0 and x0 , for any
wa and wp . Therefore, by Corollary 16.1, problem (16.8)(16.11) can be solved by
325
326
16
Remark 16.7 In case of CROC-OL with additive disturbances only (wp (t) = 0) the
number of constraints in (16.11) can be reduced as explained in Remark 16.5.
16.4.2
if
x CRki , i = 1, . . . , Nkr
(16.50)
(16.51)
JN
1 (xN 1 ) , umin JN 1 (xN 1 , uN 1 )
N 1
xN 1 X , uN 1 U
p
p
a
A(wN
subj. to
1 )xN 1 + B(wN 1 )uN 1 + EwN 1 Xf
p
a
a
p
wN 1 W , wN 1 W
(16.52)
kQxN 1 kp + kRuN 1 kp +
p
+kP (A(wN
)xN 1 +
JN 1 (xN 1 , uN 1 ) , a
maxp
.
1
p
wN 1 W a , wN 1 W p
a
+B(wN
)u
+
Ew
)k
N 1 p
1 N 1
(16.53)
The cost function in the maximization problem (16.53) is piecewise affine and
p
a
convex with respect to the optimization vector wN
1 , wN 1 and the parameters
uN 1 , xN 1 . Moreover, the constraints in the minimization problem (16.52) are
p
a
linear in (uN 1 , xN 1 ) for all vectors wN
1 , wN 1 . Therefore, by Corollary 16.1,
16.4
JN
1 (xN 1 ), uN 1 (xN 1 ) and XN 1 are computable via the mp-LP:
JN
1 (xN 1 ) ,
min
,uN 1
(16.54a)
(16.54b)
(16.54c)
xN 1 X , uN 1 U
(16.54d)
W
Wa
are the vertices of the disturbance sets W a and
and {w
hp }nh=1
where {w
ia }ni=1
p
Remark 16.8 Let na and nb be the number of inequalities in (16.54b) and (16.54d),
respectively, for any i and h. In case of additive disturbances only (nW p = 0) the
total number of constraints in (16.54b) and (16.54d) for all i and h can be reduced
from (na + nb )nW a nW p to na + nb as shown in Remark 16.5.
Remark 16.9 The closed-loop solution u (k) = fk (x(k)) can be also obtained by using
the modified batch approach with closed-loop prediction as discussed in Section 16.2.
The idea there is to augment the number of free inputs by allowing one sequence
u
i0 , . . . u
iN1 for each vertex i of the disturbance set W a W a . . . W a . The large
{z
}
|
N1
number of extreme points of such a set and the consequent large number of inputs
and constraints make this approach computationally hard.
16.4.3
327
328
16
subj. to
Gz w + Sx.
(16.55)
(16.56)
J (x) , minz
J(z, x)
subj. to Cz c + Sx.
(16.57)
SnJ
J
are polyhedral sets with disjoint interiors, R , i=1
where {Ri }ni=1
Ri is a (possibly
non-convex) polyhedral set and Li , Hi and Ki are matrices of suitable dimensions.
Then the multiparametric optimization problem
is an mp-MILP.
Proof: By following the approach of [37] to transform piecewise affine functions into a set of mixed-integer linear inequalities, introduce the auxiliary binary
optimization variables i {0, 1}, defined as
(16.58)
[i = 1] [ xz ] Ri ,
PnJ
i = 1, and set
where i , i = 1, . . . , nJ , satisfy the exclusive-or condition i=1
J(z, x) =
nJ
X
qi
(16.59)
i=1
qi
[Li z + Hi x + Ki ]i
(16.60)
Next we present two theorems which describe how to use mp-MILP to solve
CROC-OL and CROC-CL problems.
1 In
16.5
Theorem 16.6 By solving two mp-MILPs, the solution U0 (x0 ) to the CROCOL (16.8), (16.9), (16.11) with additive disturbances only (np = 0) can be computed
in explicit piecewise affine form (16.49).
Proof: The objective function in the maximization problem (16.8) is convex
a
and piecewise affine with respect to the optimization vector wa = {w0a , . . . , wN
1 }
and the parameters U = {u0 , . . . , uN 1 }, x0 . By Lemma 16.2, it can be solved via
mp-MILP. By Theorem 7.10, the resulting value function J is a piecewise affine
function of U and x0 and the constraints in (16.11) are a linear function of the
disturbance wa for any given U and x0 . Then, by Lemma 16.2 and Corollary 16.1
the minimization problem is again solvable via mp-MILP, and the optimizer U =
{u0 ,. . . ,uN 1 } is a piecewise affine function of x0 .
Theorem 16.7 By solving 2N mp-MILPs, the solution of the CROC-CL (16.14)
(16.17) problem with cost (16.20), (16.21) and additive disturbances only (np = 0)
can be obtained in state feedback piecewise affine form (16.50)
Proof: Consider the first step j = N 1 of the dynamic programming solution
to CROC-CL. The cost-to-go at step N 1 is p(xN ) from (16.16). Therefore, the
cost function in the maximization problem is piecewise affine with respect to both
a
the optimization vector wN
1 and the parameters uN 1 , xN 1 . By Lemma 16.2,
the worst case cost function (16.20) can be computed via mp-MILP, and, from
Theorem 7.10 we know that JN 1 (uN 1 , xN 1 ) is a piecewise affine function. Then,
a
since constraints (16.52) are linear with respect to wN
1 for each uN 1 , xN 1 , we
can apply Corollary 16.1 and Remark 16.5 by solving LPs of the form (11.52).
16.5
329
330
16
PN 1
k=0
q(xk , uk )
(16.61)
xk X , uk U
k = 0, . . . , N 1
(16.62)
wka W a
k = 0, . . . , N 1
(16.63)
k
X
Lk,i xi + gi ,
i=0
k {0, . . . , N 1}
(16.64)
U0 = Lx + g,
where x =
[x0 , x1 , . . . , xN ]
and
L0,0
..
L= .
LN 1,0
0
..
.
..
.
LN 1,N 1
0
.. ,
.
0
g0
g = ...
gN 1
(16.65)
where L RmN nN and g RmN are unknown feedback control gain and offset,
respectively. With the parametrization (16.64) the robust control problem (16.61(16.63) becomes
PN 1
a
J0Lg (x(0), L, g) , maxw0a ,...,wN
p(xN ) + k=0 q(xk , uk )
1
wa W a ,
k
Pk
subj. to
uk = i=0 Lk,i xi + gi
k = 0, . . . , N 1
xk X , uk U
Ax + Buk + Ewka
xk+1 =
Pk k
subj. to
u = i=0 Lk,i xi + gi
k
x Xf
N
k = 0, . . . , N 1
(16.66)
(16.67)
wka W a
k = 0, . . . , N 1
(16.68)
16.5
331
We denote with X0Lg X the set of states x0 for which the robust optimal control
problem (16.67)-(16.68) is feasible, i.e.,
n
o
X0Lg =
x0 Rn : P0Lg (x0 ) 6=
P0Lg (x0 ) =
{L, g : xk X , uk U, k = 0, . . . , N 1, xN Xf
Pk
wka W a k = 0, . . . , N 1, where xk+1 = Axk + Buk + Ewka , uk = i=0 Lk,i xi + gi }
(16.69)
Problem (16.66) looks for the worst value of the performance index J0Lg (x0 , L, g)
and the corresponding worst sequences wa as a function of x0 and the controller
gain L and offset g.
Problem (16.67)(16.68) minimizes (over L and g) the worst performance subject to the constraint that the input sequence U0 = Lx + g must be feasible for
all possible disturbance realizations. Notice that formulation (16.66)(16.68) is
based on a closed-loop prediction. Unfortunately the set P0Lg (x0 ) is nonconvex,
in general [187]. Therefore, finding L and g for a given x0 may be difficult.
Consider now the parametrization of the control sequence in past disturbances
uk =
k1
X
Mk,i wi + vi ,
i=0
k {0, . . . , N 1},
(16.70)
0
M1,0
..
.
M =
MN 1,0
0
..
.
..
.
MN 1,N 2
0
0
.. ,
.
0
v0
..
.
..
.
.
v=
vN 1
(16.71)
k {0, . . . , N 1}.
wa W a ,
k
Pk1
subj. to
uk = i=0
Mk,i wi + vi
k = 0, . . . , N 1
(16.72)
332
16
xk X , uk U
Pk1
subj. to
uk = i=0 Mk,i wi + vi
x Xf
N
k = 0, . . . , N 1
(16.73)
wka W a
k = 0, . . . , N 1
(16.74)
Note that in general X0Mv and J0Mv (x0 ) are different from the corresponding
(16.76)
where x, u and w are state, input and disturbance, respectively. Let uk [1, 1] and
wk [1, 1] be the feasible input and disturbance. The objective for player U is to
play two moves in order to keep the state x3 in the set Xf = [1, 1].
Rewrite the terminal constraint as
x3 = x0 + u0 + u1 + u2 + w0 + w1 + w2 [1, 1]
w0 [1, 1], w1 [1, 1], w2 [1, 1].
(16.77)
(16.78)
16.5
333
(16.79)
There are two approaches to convert the infinite number of constraints (16.79) into
a set of finite number of constraint. The first approach has been already presented
in this chapter and resorts to disturbance vertex enumeration. The second approach
uses duality. We will present both approaches next.
Enumeration method
Constraint (16.79) are linear in w0 , w1 and w2 for fixed v0 , v1 , v2 , M1,0 , M2,0 ,
M2,1 . Therefore the worst case will occur at the extremes of the admissible disturbance realizations, i.e., {(1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1), (1, 1, 1),
(1, 1, 1), (1, 1, 1), (1, 1, 1)}. Therefore the constraints (16.79) can
be rewritten as:
x0 + v0 + v1 + v2 + (1 + M1,0 + M2,0 ) + (1 + M2,1 ) + 1 [1, 1]
x0 + v0 + v1 + v2 + (1 + M1,0 + M2,0 ) + (1 + M2,1 ) 1 [1, 1]
x0 + v0 + v1 + v2 (1 + M1,0 + M2,0 ) + (1 + M2,1 ) + 1 [1, 1]
x0 + v0 + v1 + v2 (1 + M1,0 + M2,0 ) + (1 + M2,1 ) 1 [1, 1]
x0 + v0 + v1 + v2 + (1 + M1,0 + M2,0 ) (1 + M2,1 ) + 1 [1, 1]
x0 + v0 + v1 + v2 + (1 + M1,0 + M2,0 ) (1 + M2,1 ) 1 [1, 1]
x0 + v0 + v1 + v2 (1 + M1,0 + M2,0 ) (1 + M2,1 ) + 1 [1, 1]
x0 + v0 + v1 + v2 (1 + M1,0 + M2,0 ) (1 + M2,1 ) 1 [1, 1]
u0 = v0 [1, 1]
u1 = v1 + M1,0 [1, 1]
u1 = v1 M1,0 [1, 1]
u2 = v2 + M2,0 + M2,1 [1, 1]
u2 = v2 + M2,0 M2,1 [1, 1]
u2 = v2 M2,0 + M2,1 [1, 1]
u2 = v2 M2,0 M2,1 [1, 1]
(16.80)
u0 = 0.8
u1 = w0
u2 = w1
(16.82)
u0 = 0.8
u1 = x1
u2 = x2
(16.83)
which corresponds to
334
16
Duality method
Duality can be used to avoid vertex enumeration. Consider the first constraint
in (16.79):
x0 + v0 + v1 + v2 + (1 + M1,0 + M2,0 )w0 + (1 + M2,1 )w1 + w2 1,
w0 [1, 1], w1 [1, 1], w2 [1, 1].
(16.84)
We want to replace it with the most stringent constraint, i.e.
x0 + v0 + v1 + v2 +J (M1,0 , M2,0 , M2,1 ) 1
(16.85)
where
J (M1,0 , M2,0 , M2,1 )=maxw0 ,w1 ,w2
subj. to
(16.87)
where
d (M1,0 , M2,0 , M2,1 ) = minu ,l
i
i
subject to
u0 + u1 + u2 + l0 + l1 + l2
1 + M1,0 + M2,0 + l0 u0 = 0
1 + M2,1 + l1 u1 = 0
1 + l2 u2 = 0
ui 0, li 0, i = 0, 1, 2
(16.88)
where ui and li are the dual variables corresponding to the the upper- and
lower-bounds of wi , respectively. From strong duality we know that for a given
v0 , v1 , v2 , M1,0 , M2,0 , M2,1 problem (16.85)-(16.86) is feasible if and only if
there exists a set of ui and li which together with v0 , v1 , v2 , M1,0 , M2,0 , M2,1
solves the system of equalities and inequalities
x0 + v0 + v1 + v2 +u0 + u1 + u2 + l0 + l1 + l2 1
1 + M1,0 + M2,0 + l0 u0 = 0
1 + M2,1 + l1 u1 = 0
1 + l2 u2 = 0
ui 0, li 0, i = 0, 1, 2
(16.89)
16.6
Example
335
constraints:
x0 + v0 + v1 + v2 + u0 + u1 + u2 + l0 + l1 + l2 1
x0 v0 v1 v2 + u0 + u1 + u2 + l0 + l1 + l2 1
1 + M1,0 + M2,0 + l0 u0 = 0
1 + M2,1 + l1 u1 = 0
1 + l2 u2 = 0
1 M1,0 M2,0 + l0 u0 = 0
1 M2,1 + l1 u1 = 0
1 l2 u2 = 0
ui , li , ui , li 0, i = 0, 1, 2
u0 = v0 [1, 1]
v1 + 0u + 0l 1
v1 + u0 + l0 1
M1,0 + 1l 1u = 0
M1,0 + l1 u1 = 0
u0 , l0 , 0u , 0l 0,
v2 + u0 + u1 + l0 + l1 1
v2 + 0u + 1u + 0l + 1l 1
M2,0 + l0 u0 = 0
M2,1 + l1 u1 = 0
M2,0 + 0l 0u = 0
M2,1 + 1l 1u = 0
ui , li , iu , il 0, i = 0, 1
16.6
(16.90)
Example
Example 16.4 Consider the problem of robustly regulating to the origin the double
integrator system
0
1 1
u(t) + wa (t)
x(t) +
x(t + 1) =
1
0 1
subject to the input constraints
U = {u R : 3 u 3}
and the state constraints
X = {x R2 : 10 x 10, k = 0, . . . , 3}
The two-dimensional disturbance wa is restricted to the set W a = {v : kwa k
1.5}.
We compute three robust controllers.
The state feedback control law u (k) = fkOL (x(0)), k = 1, . . . , N 1 obtained by solving the CROC-CL (16.14)(16.16),(16.21)(16.20) and the CROC-OL (16.5), (16.8), (16.9)
and (16.11). We use the cost function
kP xN k +
N1
X
k=0
(kQxk k + |Ruk |)
336
16
x2
x2
x1
(a) Feasible set X0OL of the solution to the CROC-OL problem.
x1
(b) Feasible set X0 of the solution
Figure 16.1 Example (16.4). Polyhedral partition of the state-space corresponding to the explicit solution of CROC-OL and CROC-CL at time t = 0
CROC-OL. The min-max problem is formulated as in (16.8)(16.11). The resulting
polyhedral partition consists of 22 regions and it is depicted in Figure 16.1(a). We remark that the CROC-OL problem (16.8)(16.11) is infeasible for a horizon N greater
than five.
CROC-CL. The min-max problem is formulated as in (16.14)(16.20) and solved
using the approach of Theorem 16.5. The resulting polyhedral partition consists of
64 regions and is depicted in Figure 16.1(b).
Parametrization.
16.7
16.7
337
RHC law (16.91) is piecewise affine and thus its on-line computation consists of a
function evaluation.
Convergence and persistent feasibility of the robust receding horizon controller
are not guaranteed as discussed in Chapter 13 for nominal receding horizon controllers. In the robust RHC case it is desirable to obtain robust convergence to a
set O Xf (rather than convergence to an equilibrium point) for all X0 . In other
words, the goal is to design a RHC control law which drives any feasible state in
X0 into the set O for all admissible disturbances and keeps the states inside the set
O for all future time and for all admissible disturbances. Clearly this is possible
only if O is a robust control invariant set for system (16.1)-(16.4).
We define a distance of a point x Rn from a nonempty set Y Rn as:
d(x, Y) , inf d(x, y)
yY
(16.93)
The following theorem presents sufficient conditions for convergence and persistent feasibility in the robust case and is a modified version of the one presented
in [21]. It can be proven by using the arguments of Theorem 13.2.
Theorem 16.9 Consider system (16.1)-(16.4), the closed-loop RHC law (16.91)
where f0 (x) is obtained by solving the CROC-CL (16.14)(16.16) with cost (16.20), (16.21)
for x(0) = x and the closed-loop system (16.92). Assume that
(A1)
(A2)
(A3)
J p (x) 0 x Xf where
J p (x)
= minu J c (x, u)
subj. to
uU
A(wp )x + B(wp )u + Ewa Xf wa W a wp W p
(16.94)
and
J c (x, u) =
(A4)
wa W a , wp W p
x+ = A(wp )x + B(wp )u + Ewa
(A5)
(16.95)
(16.96)
(16.97)
338
16
Remark 16.10 Assumptions A4 and A5 in Theorem 16.9 imply that stage cost
q(x, u) and terminal cost p(x) in (16.8) need to be zero in O. Note that this cannot
be obtained, for instance, with the cost (16.22) in (16.8).
Remark 16.11 Robust control invariance of the set Xf in Assumption (A2) of Theorem 16.9 implies that problem (16.94) in Assumption (A3) is always feasible.
Remark 16.12 Robust control invariance of the set O in Assumption (A2) of Theorem 16.9 is also implicitly guaranteed by Assumption (A3) as shown next.
From Remark 16.10, Assumption (A3) in O becomes minu maxwa ,wp (p(A(wp )x +
B(wp )u + Ewa ) 0 for all x O. From Assumption (A4), this can be verified only if
it is equal to zero, i.e. if there exists a u U such that A(wp )x + B(wp)u + Ewa O
for all wa W a wp W p (since O is the only place where p(x) can be zero). This
implies the robust control invariance of O.
16.8
Literature Review
An extensive treatment of robust invariant sets can be found in [54, 55, 49, 56].
The proof to Theorem 11.2 can be fond in [165, 94] For the derivation of the algorithms 11.4, 11.5 for computing robust invariant sets (and their finite termination)
see [9, 47, 165, 118, 158].
Min-max robust constrained optimal control was originally proposed by Witsenhausen [280]. In the context of robust MPC, the problem was tackled by Campo
and Morari [74], and further developed in [5] for SISO FIR plants. Kothare et
al. [172] optimize robust performance for polytopic/multi-model and linear fractional uncertainty, Scokaert and Mayne [249] for additive disturbances, and Lee and
Yu [182] for linear time-varying and time-invariant state-space models depending
on a vector of parameters , where is either an ellipsoid or a polyhedron. Other suboptimal CROC-Cl strategies have been proposed in [172, 26, 173].
For stability and feasibility of the robust RHC (16.1), (16.91) we refer the reader
to [38, 190, 198, 21].
The idea of the parametrization (16.70) appears in the work of Gartska & Wets
in 1974 in the context of stochastic optimization [112]. Recently, it re-appeared in
robust optimization work by Guslitzer and Ben-Tal [131, 43], and in the context
of robust MPC in the work of van Hessem & Bosgra, L
ofberg and Goulart &
Kerrigan [272, 187, 125].
Part V
Chapter 17
17.1
342
17
binary
inputs
discrete
dynamics
and logic
binary
outputs
mode
switches
events
real-valued
outputs
continuous
dynamics
real-valued
inputs
In Section 17.7 we will introduce the tool HYSDEL (HYbrid Systems DEscription Language), a high level language for modeling and simulating DHA. Therefore,
DHA will represent for us the starting point for modeling hybrid systems. We will
show that DHA, PWA, and MLD systems are equivalent model classes, and in particular that DHA systems can be converted to an equivalent PWA or MLD form
for solving optimal control problems.
After introducing PWA systems, we will go through the steps needed for modeling a system as a DHA. We will first detail the process of translating propositional
logic involving Boolean variables and linear threshold events over continuous variables into mixed-integer linear inequalities, generalizing several results available in
the literature, in order to get an equivalent MLD form of a DHA system. Finally, we
will briefly present the tool HYSDEL, that allows describing the DHA in a textual
form and obtain equivalent MLD and PWA representations in MATLABTM .
17.2
(17.1a)
(17.1b)
(17.1c)
17.2
343
x-space
C3
C4
C7
C1
C6
C2
C5
Figure 17.2 Piecewise affine (PWA) systems. Mode switches are triggered
by linear threshold events.
where x(t) Rn is the state vector at time t T and T , {0, 1, . . .} is the set of
nonnegative integers, u(t) Rm is the input vector, y(t) Rp is the output vector,
i(t) I , {1, . . . , s} is the current mode of the system, the matrices Ai(t) , B i(t) ,
f i(t) , C i(t) , Di(t) , g i(t) , H i(t) , J i(t) , K i(t) are constant and have suitable dimensions,
and the inequalities in (17.1c) should be interpreted component-wise. Each linear
inequality in (17.1c) defines a half-space in Rn and a corresponding hyperplane,
that will be also referred to as guardline. Each vector inequality (17.1c) defines a
polyhedron C i = {[ ux ] Rn+m : H i x + J i u K i } in the state+input space Rn+m
that will be referred to as cell, and the union of such polyhedral cells as partition.
We assume that C i are full-dimensional sets of Rn+m , for all i = 1, . . . , s.
A PWA system is called well-posed if it satisfies the following property [35]:
Definition 17.1 Let P be a PWA system of the form (17.1) and let C = si=1 C i
Rn+m be the polyhedral partition associated with it. System P is called well-posed
if for all pairs (x(t), u(t)) C there exists only one index i(t) satisfying (17.1).
Definition 17.1 implies that x(t+1), y(t) are single-valued functions of x(t) and u(t),
and therefore that state and output trajectories are uniquely determined by the
initial state and input trajectory. A relaxation of definition 17.1 is to let polyhedral
cells C i share one or more hyperplanes. In this case the index i(t) is not uniquely
defined, and therefore the PWA system is not well-posed. However, if the mappings
(x(t), u(t)) x(t + 1) and (x(t), u(t)) y(t) are continuous across the guardlines
that are facets of two or more cells (and, therefore, they are continuous on their
domain of definition), such mappings are still single valued.
344
17.2.1
17
Modeling Discontinuities
Discontinuous dynamical behaviors can be modeled by disconnecting the domain.
For instance, the state-update equation
1
2 x(t) + 1 if x(t) 0
x(t + 1) =
(17.2a)
0
if x(t) > 0
is discontinuous across x = 0. It can be modeled as
1
2 x(t) + 1 if x(t) 0
x(t + 1) =
0
if x(t)
(17.2b)
where > 0 is an arbitrarily small number, for instance the machine precision.
Clearly, system (17.2) is not defined for 0 < x(t) < , i.e., for the values of the state
that cannot be represented in the machine. However, the trajectories produced
by (17.2a) and (17.2b) are identical as long as x(t) > or x(t) 0, t N.
As remarked above, multiple definitions of the state-update and output functions over common boundaries of sets C i is a technical issue that arises only when
the PWA mapping is discontinuous. Rather than disconnecting the domain, another way of dealing with discontinuous PWA mappings is to allow strict inequalities in the definition of the polyhedral cells in (17.1), or by dealing with open
polyhedra and boundaries separately as in [257]. We prefer to assume that in the
definition of the PWA dynamics (17.1) the polyhedral cells C i(t) are closed sets:
As will be clear in the next chapter, the closed-polyhedra description is mainly
motivated by the fact that numerical solvers cannot handle open sets.
Example 17.1 The
x(t + 1)
y(t)
(t)
1 0 ] x(t) 0
if
[
3
=
3 if [ 1 0 ] x(t) < 0
is discontinuous at x = x02 , x2 6= 0. It can be described in form (17.1) as
1
3
0.4
u(t) if
1 0 x(t) 0
x(t) +
1
3
1
x(t + 1) =
3
0
1
u(t) if
1 0 x(t)
x(t) +
0.4
1
3
1
y(t) =
0 1 x(t)
(17.4)
for all x1 (, ] [0, +), x2 R, u R, and > 0.
Figure 17.3 shows the free response of the systems (open-loop simulation of the system
for a constant input u = 0) starting from the initial condition x(0) = [1 0] with
sampling time equal to 0.5s and = 106 .
In case the partition C does not cover the whole space Rn+m , well-posedness
does not imply that trajectories are persistent, i.e., that for all t N a successor
345
x1
17.2
mode
time
time
Figure 17.3 Example 17.1: Free response of state x1 for x(0) = [1 0].
state x(t + 1) and an output y(t) are defined. A typical case of C 6= Rn+m is
when we are dealing with bounded inputs and bounded states umin u(t) umax ,
xmin x(t) xmax . By embedding such ranges in the inequalities (17.1c), the
system becomes undefined outside the bounds, as no index i exists that satisfies
any of the set of inequalities (17.1c).
As will be clearer in the next chapter, when model (17.1) is used in an optimal control formulation, any input sequence and initial state that are feasible for
the related optimization problem automatically define unique trajectories over the
whole optimal control horizon.
PWA systems can model a large number of physical processes, as they can
model static nonlinearities through a piecewise affine approximation, or approximate nonlinear dynamics via multiple linearizations at different operating points.
Moreover, tools exist for obtaining piecewise affine approximations automatically
(see Section 17.8).
When the mode i(t) is an exogenous variable, condition (17.1c) disappears and
we refer to (17.1) as a switched affine system (SAS), see Section 17.3.1.
17.2.2
346
17
(17.5a)
x (t) u (t)
(17.5b)
where represents the logic operator and, can be represented in the PWA
form
0
xc
(t + 1) =
x
if
x 12 , u
1
2
if
x 12 , u
1
2
if
1
2
+ , u
1
2
if
1
2
+ , u
1
2
+
(17.5c)
+ .
b1 if u2 = 1
b2 if u2 = 0.
k1 x1 + d1 if x1 xm
k2 x1 + d2 if x1 > xm .
347
position x1
position x1
17.2
mode
mode
time
time
time
x(t+1) =
0.8956
0.0198
0.8956
0.0584
0.0198
0.0004
0.0195
0.0012
0.8956 0.3773
0.3773 0.5182
0.1044
x(t) + [ 0.0198
] u1 (t) +
0.0096
Mode 1
if x1 (t) 1, u2 (t) 0.5
0.0198
x(t) + [ 0.1044
0.0195 ] u1 (t) +
0.0711
Mode 2
if x1 (t) 1 + , u2 (t) 0.5
x(t) + [ 0.1044
0.3773 ] u1 (t) +
0.1044
Mode 3
if x1 (t) 1, u2 (t) 0.5
0.1459
0.3773
Mode 4
if x(t) 1 + , u2 (t) 0.5
(17.6)
for x1 (t) [5, 1] [1 + , 5], x2 (t) [5, 5], u1 (t) [10, 10], and for any arbitrary
small > 0.
0.8956 0.3463
1.0389 0.3529
x(t) + [ 0.1044
0.3463 ] u1 (t) +
0.7519
2.5972
Figure 17.4 shows the open-loop simulation of the system for a constant continuous
input u1 = 3, starting from zero initial conditions and for = 106 .
(17.7)
A logic state x2 [0, 1] stores the information whether the state of system (17.7) has
ever gone below a certain lower bound xlb or not:
_
x2 (t + 1) = x2 (t)
[x1 (t) xlb ],
(17.8)
Assume that the input coefficient is a function of the logic state:
b1 if x2 = 0
b=
b2 if x2 = 1.
(17.9)
x2
u
x2
x1
17
x1
348
time steps
time steps
(a) Scenario 1.
(b) Scenario 2.
Figure 17.5 Example 17.3: Open-loop simulation of system (17.10) for different sequences of the input u.
a
x(t+1) =
0
0
0
0
1
0
1
x(t) +
x(t) +
x(t) +
b2
0
b1
0
b2
0
u(t) +
0
1
if
1
0
0
1
u(t)
if
u(t)
if
x(t)
x(t) xlb
x(t)
xlb +
0.5
xlb +
0.5
(17.10)
for u(t) R, x1 (t) (, xlb ] [xlb + , +), x2 {0, 1}, and for any > 0.
Figure 17.5 shows two open-loop simulations of the system, for a = 0.5, b1 = 0.1,
b2 = 0.3, xlb = 1, = 106 . Note that when the continuous state x1 (t) goes below
xlb = 1 at time t, then x (t + 1) switches to 1 and the input has a stronger effect
on the states from time t + 2 on. Indeed, the steady state of x1 is a function of the
logic state x2 .
17.3
349
EEvent
Generator
(EG)
!ed(k)
!ed(k)
u!u(k)
FFinite State
Machine
(FSM)
xyc(k)
uvc(k)
y!(k)
uvc(k)
yc(k)
...
17.3
Switched
Affine
System
s
(SAS)
S
M
Mode
x!(k)
x
u!(k)
u
!e(k)
d
Selector
(MS)
i(k)
i
17.3.1
= C
i(t)
xc (t) + D
i(t)
uc (t) + g
i(t)
(17.11a)
(17.11b)
where t T is the time indicator, xc Rnc is the continuous state vector, uc Rmc
is the exogenous continuous input vector, yc Rpc is the continuous output vector,
{Ai , B i , f i , C i , Di , g i }iI is a collection of matrices of suitable dimensions, and the
mode i(t) I , {1, . . . , s} is an input signal that determines the affine state update
dynamics at time t. An SAS of the form (17.11) preserves the value of the state
when a mode switch occurs, but it is possible to implement reset maps on an SAS
as shown in [268].
17.3.2
350
17
where i denotes the ith component of a vector or the i-th row of a matrix, and A,
B, C are constant matrices of suitable dimensions. Then, events are defined as
e (t) = h(xc (t), uc (t))
(17.12)
In particular, state events are modeled as [e (t) = 1] [a xc (t) b]. Note that
time events can be modeled as in (17.12) by adding the continuous time as an
additional continuous and autonomous state variable, (t + 1) = (t) + Ts , where
Ts is the sampling time, and by letting [e (t) = 1] [tTs 0 ], where 0 is a given
time. By doing so, the hybrid model can be written as a time-invariant one. Clearly
the same approach can be used for time-varying events e (t) = h(xc (t), uc (t), t), by
using time-varying event conditions h : Rnc Rnc T {0, 1}ne .
17.3.3
Boolean Algebra
Before dealing in detail with the other blocks constituting the DHA and introduce
further notation, we recall here some basic definitions of Boolean algebra1 .
A variable is a Boolean variable if {0, 1}, where = 0 means something
is false, = 1 that it is true. A Boolean expression is obtained by combining
Boolean variables through the logic operators (not), (or), (and), (implied
by), (implies), and (iff). A Boolean function f : {0, 1}n1 7 {0, 1} is used
to define a Boolean variable n as a logic function of other variables 1 , . . . , n1 :
n = f (1 , 2 , . . . , n1 )
(17.13)
(17.14)
(CNF)
m
^
j=1
iPj
iNj
(17.15)
Nj , Pj {1, . . . , n}, j = 1, . . . , m.
As mentioned in Section 17.2.2, often we will use the term binary variable
and Boolean variable without distinction. An improper arithmetic operation over
Boolean variables should be understood as an arithmetic operation over corresponding binary variables and, vice versa, an improper Boolean function of binary
variables should be interpreted as the same function over the corresponding Boolean
variables. Therefore, from now on we will call binary variables both 0-1 variables
and Boolean variables.
1 A more comprehensive treatment of Boolean calculus can be found in digital circuit design
texts, e.g. [82, 137]. For a rigorous exposition see e.g. [202].
17.3
351
1
Red
1 u2
1 u2
2
Blue
Green
3 u1
2
u1 3
Figure 17.7 Example of finite state machine
17.3.4
(17.16a)
where y {0, 1}
and g : {0, 1}
{0, 1}
{0, 1}
ne
(17.16b)
p
7 {0, 1} .
Example 17.4 Figure 17.7 shows a finite state machine where u = [u1 u2 ] is
the input vector, and = [1 . . . 3 ] is a vector of signals coming from the event
generator. The Boolean state update function (also called state transition function)
is:
Green
if ((x (t) = Red) 1 u2 )
2 In this text we will only refer to synchronous finite state machines, where the transitions may
happen only at sampling times. The adjective synchronous will be omitted for brevity.
352
17
x2 (t + 1)
( x1 x2 1 u2 )
(x1 2 ) (x2 3 u1 ),
( x1 x2 1 u2 )
(x1 2 ) (x2 3 u1 ),
where the time index (t) has been omitted for brevity.
Since the Boolean state update function is deterministic, for each state the
conditions associated with all the outgoing arcs are mutually exclusive.
17.3.5
Mode Selector
In a DHA, the dynamic mode i(t) I = {1, . . . , s} of the SAS is a function
of the binary state x (t), the binary input u (t), and the events e (t). With a
slight abuse of notation, let us indicate the mode i(t) through its binary encoding,
i(t) {0, 1}ns where ns = log2 s, so that i(t) can be treated as a vector of
Boolean variables3. Then, we define the mode selector by the Boolean function
n
m
n
fM : {0, 1} {0, 1} {0, 1} e {0, 1}ns . The output of this function
i(t) = (x (t), u (t), e (t))
(17.18)
is called the active mode of the DHA at time t. We say that a mode switch occurs
at step t if i(t) 6= i(t 1). Note that, in contrast to continuous-time hybrid models
where switches can occur at any time, in our discrete-time setting, as mentioned
earlier, a mode switch can only occur at sampling instants.
17.3.6
DHA Trajectories
For a given initial condition
m
xc (0)
x (0)
uc (t)
u (t)
Rm c
{0, 1} , t T, the state x(t) of the system is computed for all t T by recursively
iterating the set of equations:
e (t) =
(17.19a)
i(t) =
yc (t) =
(17.19b)
(17.19c)
y (t) =
xc (t + 1) =
(17.19d)
(17.19e)
x (t + 1) =
(17.19f)
3 Any discrete variable { , . . . , } admits a Boolean encoding a {0, 1}d(j) , where d(j)
1
j
is the number of bits used to represent
1 , . . . , j . For example, {0, 1, 2} may be encoded
2
as a {0, 1} by associating 00 0, 01 1, 10 2.
17.4
17.4
17.4.1
P
P
1 iP1 i + iN1 (1 i )
..
(17.20)
P = {0, 1}n :
. P
1 iPm i + iNm (1 i )
The smallest polyhedron P associated with formula F has the following geometric interpretation: Assume to list all the 0-1 combinations of i s satisfying
353
354
17
relation
AND
OR
NOT
XOR
IMPLY
IFF
ASSIGNMENT
3 = 1 2
Boolean
1 2
1 X2
1
1 2
1 2
1 2
3 1 2
linear constraints
1 = 1, 2 = 1 or 1 + 2 2
1 + 2 1
1 = 0
1 + 2 = 1
1 2 0
1 2 = 0
1 + (1 3 ) 1
2 + (1 3 ) 1
(1 1 ) + (1 2 ) + 3 1
Table 17.1 Basic conversion of Boolean relations into mixed-integer inequalities. Relations involving the inverted literals can be obtained by substituting (1 ) for in the corresponding inequalities. More conversions
are reported in [204], or can be derived by (17.15)(17.20)
17.4.2
mi ei ,
(17.21a)
i = 1, . . . , ne ,
(17.21b)
where M i , mi are upper and lower bounds, respectively, on hi (xc (t), uc (t), t). As
we have pointed out in Section 17.2.1, from a computational viewpoint it is convenient to avoid strict inequalities. As suggested in [278], we modify the strict
inequality (17.21b) into
hi (xc (t), uc (t), t) + (mi )ei
(17.21c)
where is a small positive scalar (e.g., the machine precision). Clearly, as for the
case of discontinuous PWA discussed in Section 17.2.1, Equations (17.21) or (17.12)
are equivalent to (17.21c) only for hi (xc (t), uc (t), t) 0 and hi (xc (t), uc (t), t) .
Regarding switched affine dynamics, we first rewrite the state-update equation (17.11a) as the following combination of affine terms and if-then-else condi-
17.5
355
tions:
z1 (t) =
..
.
(17.22a)
(17.22b)
(17.22c)
i=1
IF THEN z = a1 x + b1 u + f 1 ELSE z = a2 x + b2 u + f 2 ,
(17.23)
(m1 M2 ) z
(m1 M2 )(1 ) + z
(m2 M1 )(1 ) z
a2 x + b 2 u + f 2 ,
2
(17.24a)
2
a x b u f ,
1
a x+b u+f ,
1
a x b u f ,
(17.24b)
(17.24c)
(17.24d)
17.5
(17.25a)
(17.25b)
(17.25c)
356
17
where x Rnc {0, 1}n is a vector of continuous and binary states, u Rmc
{0, 1}m are the inputs, y Rpc {0, 1}p the outputs, {0, 1}r are auxiliary
binary variables, z Rrc are continuous auxiliary variables which arise in the
transformation (see Example 17.5), and A, B1 , B2 , B3 , C, D1 , D2 , D3 , E1 ,. . . ,E5
are matrices of suitable dimensions. Given the current state x(t) and input u(t),
the time-evolution of (17.25) is determined by finding a feasible value for (t) and
z(t) satisfying (17.25c), and then by computing x(t + 1) and y(t) from (17.25a)
(17.25b).
As MLD models consist of a collection of linear difference equations involving
both real and binary variables and a set of linear inequality constraints, they are
model representations of hybrid systems that can be easily used in optimization
algorithms, as will be described in Chapter 18.
A definition of well-posedness of the MLD system (17.25) can be given similarly to Definition 17.1 by requiring that for all x(t) and u(t) within a given
bounded set the pair of variables (t), z(t) satisfying (17.25c) is unique, so that
the successor state x(t + 1) and output y(t) are also uniquely defined functions of
x(t), u(t) through (17.25a)(17.25b)4. Such a well-posedness assumption is usually guaranteed by the procedure described in Section 17.3.3 used to generate the
linear inequalities (17.25c). A numerical test for well-posedness is reported in [37,
Appendix 1].
Note that the constraints (17.25c) allow one to specify additional linear constraints on continuous variables (e.g., constraints over physical variables of the
system), and logical constraints over Boolean variables. The ability to include
constraints, constraint prioritization, and heuristics adds to the expressiveness and
generality of the MLD framework. Note also that despite the fact that the description (17.25) seems to be linear, clearly the nonlinearity is concentrated in the
integrality constraints over binary variables.
Example 17.5 Consider the following simple switched linear system [37]
0.8x(t) + u(t) if x(t) 0
x(t + 1) =
0.8x(t) + u(t) if x(t) < 0
(17.26)
where x(t) [10, 10], and u(t) [1, 1]. The condition x(t) 0 can be associated
to an event variable (t) {0, 1} defined as
[(t) = 1] [x(t) 0]
(17.27)
(M + )(t)
x(t) m
x(t)
(17.28a)
(17.28b)
where M = m = 10, and is an arbitrarily small positive scalar. Then (17.26) can
be rewritten as
x(t + 1) = 1.6(t)x(t) 0.8x(t) + u(t)
(17.29)
4 For
17.6
Model Equivalence
357
M (t)
(17.30a)
m(t)
(17.30b)
x(t) M (1 (t))
(17.30c)
(17.30d)
(17.31)
subject to the linear constraints (17.28) and (17.30). Therefore, the MLD equivalent
representation of (17.26) for x [10, ][0, 10] and u [1, 1] is given by collecting
Equations (17.31), (17.28) and (17.30).
17.6
Model Equivalence
In the previous chapters we have presented three different classes of discrete-time
hybrid models: PWA systems, DHA, and MLD systems. For what we described
in Section 17.3.3, under the assumption that the set of valid states and inputs
is bounded, DHA systems can always be equivalently described as MLD systems.
Also, a PWA system is a special case of a DHA whose threshold events and mode
selector function are defined by the PWA partition (17.1c). Therefore, a PWA
system with bounded partition C can always be described as an MLD system (an
efficient way of modeling PWA systems in MLD form is reported in [37]). The
converse result, namely that MLD systems (and therefore DHA) can be represented
as PWA systems, is less obvious. For any choice of , model (17.25) represented
an affine system defined over a polyhedral domain. Under the assumption of well
posedness these domains do not overlap. This result was proved formally in [32,
27, 115].
Such equivalence results are of interest because DHA are most suitable in the
modeling phase, but MLD systems are most suitable for solving open-loop finite
time optimal control problems, and PWA systems are most suitable for solving
finite time optimal control problems in state-feedback form, as will be described in
Chapter 18.
17.7
358
17
SYSTEM sample {
INTERFACE {
STATE {
REAL xr [-10, 10]; }
INPUT {
REAL ur [-2, 2]; }
}
IMPLEMENTATION {
AUX {
REAL z1, z2, z3;
BOOL de, df, d1, d2, d3; }
AD {
de = xr >= 0;
df = xr + ur - 1 >= 0; }
LOGIC {
d1 = ~de & ~df;
d2 = de;
d3 = ~de & df; }
DA {
z1 = {IF d1 THEN xr + ur - 1 };
z2 = {IF d2 THEN 2 * xr };
z3 = {IF (~de & df) THEN 2 }; }
CONTINUOUS {
xr = z1 + z2 + z3; }
}}
the technique presented in Section 17.4, and PWA form using either the approach
of [115] or the approach of [27]. HYSDEL can generate also a simulator that runs
as a function in MATLABTM . Both the HYSDEL compiler 5 and the Hybrid
Toolbox6 can import and convert HYSDEL models.
In this section we illustrate the functionality of HYSDEL through a set of
examples. For more examples and the detailed syntax we refer the interested
reader to [267].
Example 17.6 Consider the DHA system:
2,
if i(t) = 3,
e (t) = [xc (t) 0],
EG:
f (t) = [xc (t) + uc (t) 1 0],
i
h
e (t)
0
f (t) = [ 0 ] ,
1, if
MS: i(t) =
2, if he (t) =i 1,
e (t)
3, if
= [0].
f (t)
(17.32a)
(17.32b)
(17.32c)
The HYSDEL list is composed of two parts. The first one, called INTERFACE,
contains the declaration of all variables and parameters, so that it is possible to
make the proper type checks. The second part, IMPLEMENTATION, is composed
5 http://control.ee.ethz.ch/
~ Ch17-hybrid/hysdel
6 http://www.dii.unisi.it/Ch17-hybrid/toolbox
17.7
IMPLEMENTATION {
AUX { REAL z1,z2;
BOOL sign; }
AD { sign = x1<=0; }
DA
CONTINUOUS { x1 = z1;
x2 = z2+u; }
OUTPUT { y = x2;
}
}
Table 17.3 HYSDEL model of the PWA system described in Example 17.1
of specialized sections where the relations among the variables are described. These
sections are described next.
The HYSDEL section AUX contains the declaration of the auxiliary variables used
in the model. The HYSDEL section AD allows one to define Boolean variables from
continuous ones, and is based exactly on the semantics of the event generator (EG)
described earlier. The HYSDEL section DA defines continuous variables according
to if-then-else conditions. This section models part of the switched affine system
(SAS), namely the variables zi defined in (17.22a)(17.22b). The CONTINUOUS
section describes the linear dynamics, expressed as difference equations. This section
models (17.22c). The section LOGIC allows one to specify arbitrary functions of
Boolean variables.
Example 17.7 Consider again the PWA system described in Example 17.1. Assume
that [5, 5] [5, 5] is the set of states x(t) of interest and u(t) [1, 1]. By using
HYSDEL the PWA system (17.3) is described as in Table 17.3 and the equivalent
359
360
17
1
0
1 0
cc1
c1
0.4 c2
1
c1
0.4 c2
1 0
0
c1 (t) +
0.4 c2 x(t) + 0 .
1 0 z(t)
c1
0 1
c1
c
c2 0.4
1
c1
c1
0 1
0 1
0 1
c2 0.4
c2 0.4
c2
0.4
c1
0
0
where c1 = 4( 3 + 1), c2 = 0.4 3, = 106 . Note that in Table 17.3 the OUTPUT
section allows to specify a linear map for the output vector y.
Example 17.9 Consider again the system with a logic state described in Example 17.3.
The MLD model obtained by compiling the HYSDEL list of Table 17.5 is
1
0 0
z(t)
(t) +
x(t + 1) =
0
0 1
1
0
0
0
9
0
1
0
0
11
0
0
0.5 14
0.3
1
0
0
0
14
0
0.5
0.3
1
x(t) +
(t) + 1 z(t) 0.1 u(t) + 0.5
0
14
0
1
0.1
1
0
14
0
0
1
1
0
0
0
0
0
0
1
0
1
0
1
0
0
1
1
where we have assumed that [10, 10] is the set of states x1 and [10, 10] the set of
continuous inputs u of interest. In Table 17.5 the AUTOMATA section specifies the
state transition equations of the finite state machine (FSM) as a collection of Boolean
functions.
17.8
Literature Review
The lack of a general theory and of systematic design tools for systems having
such a heterogeneous dynamical discrete and continuous nature led to a considerable interest in the study of hybrid systems. After the seminal work published
1
10
14
14
0
0
0
0
0
17.8
Literature Review
361
/* Spring-Mass-Damper System
*/
SYSTEM springmass
/* spring region */
region = x1-xm <= 0;
}
DA {
zx11
zx12
zx21
zx22
zx31
zx32
zx41
zx42
=
=
=
=
=
=
=
=
{
{
{
{
{
{
{
{
IF
IF
IF
IF
IF
IF
IF
IF
region
region
region
region
~region
~region
~region
~region
&
&
&
&
&
&
&
&
u2
u2
~u2
~u2
u2
u2
~u2
~u2
THEN
THEN
THEN
THEN
THEN
THEN
THEN
THEN
A111*x1+A112*x2+B111*u1+B112};
A121*x1+A122*x2+B121*u1+B122};
A211*x1+A212*x2+B211*u1+B212};
A221*x1+A222*x2+B221*u1+B222};
A311*x1+A312*x2+B311*u1+B312};
A321*x1+A322*x2+B321*u1+B322};
A411*x1+A412*x2+B411*u1+B412};
A421*x1+A422*x2+B421*u1+B422};
}
CONTINUOUS {
x1=zx11+zx21+zx31+zx41;
x2=zx12+zx22+zx32+zx42;
}
}
}
362
17
ELSE a*x1+b1*u};
Table 17.5 HYSDEL model of the system with logic state described in Example 17.3
17.8
Literature Review
363
364
17
in Section 17.1, several other issues were considered. These include: equivalence of
hybrid models, stability and passivity analysis, reachability analysis and verification of safety properties, controller synthesis, observability analysis, state estimation and fault detection schemes, system identification, stochastic and event-driven
dynamics. We will briefly review some of these results in the next paragraphs and
provide pointers to some relevant literature references.
Stability Analysis
Piecewise quadratic Lyapunov stability is becoming a standard in the stability
analysis of hybrid systems [150, 90, 96, 224, 225]. It is a deductive way to prove
the stability of an equilibrium point of a subclass of hybrid systems (piecewise
affine systems). The computational burden is usually low, at the price of a convex relaxation of the problem, that leads to possibly conservative results. Such
conservativeness can be reduced by constructing piecewise polynomial Lyapunov
functions via semidefinite programming by means of the sum of squares (SOS) decomposition of multivariate polynomials [221]. SOS methods for analyzing stability
of continuous-time hybrid and switched systems are described in [229]. For the general class of switched systems of the form x = fi (x), i = 1, . . . , s, an extension of the
Lyapunov criterion based on multiple Lyapunov functions was introduced in [63].
The reader is also referred to the book of Liberzon [184].
The research on stability criteria for PWA systems has been motivated by the
fact that the stability of each component subsystem is not sufficient to guarantee
stability of a PWA system (and vice versa). Branicky [63], gives an example where
stable subsystems are suitably combined to generate an unstable PWA system.
Stable systems constructed from unstable ones have been reported in [270]. These
examples point out that restrictions on the switching have to be imposed in order
to guarantee that a PWA composition of stable components remains stable.
Passivity analysis of hybrid models has received very little attention, except
for the contributions of [73, 192, 286] and [227], in which notions of passivity for
continuous-time hybrid systems are formulated, and of [28], where passivity and
17.8
Literature Review
Control
The majority of the control approaches for hybrid systems is based on optimal
control ideas (see the survey [282]). For continuous-time hybrid systems, most
authors either studied necessary conditions for a trajectory to be optimal [226,
262], or focused on the computation of optimal/suboptimal solutions by means
of dynamic programming or the maximum principle [122, 236, 138, 139, 76, 283,
185, 252, 255, 257, 264, 66, 136, 189, 122, 65, 241, 67] The hybrid optimal control
problem becomes less complex when the dynamics is expressed in discrete-time,
as the main source of complexity becomes the combinatorial (yet finite) number
of possible switching sequences. As will be shown in Chapter 18, optimal control
problems can be solved for discrete-time hybrid systems using either the PWA
or the MLD models described in this chapter. The solution to optimal control
problems for discrete-time hybrid systems was first outlined by Sontag in [257].
365
366
17
In his plenary presentation [195] at the 2001 European Control Conference Mayne
presented an intuitively appealing characterization of the state feedback solution to
optimal control problems for linear hybrid systems with performance criteria based
on quadratic and piecewise linear norms. The detailed exposition presented in the
first part of Chapter 18 follows a similar line of argumentation and shows that the
state feedback solution to the finite time optimal control problem is a time-varying
piecewise affine feedback control law, possibly defined over non-convex regions.
Model Predictive Control for discrete-time PWA systems and their stability and
robustness properties have been studied in [64, 57, 29, 181, 178, 179]. Invariant
sets computation for PWA systems has also been studied in [180, 4].
Identification
Identification techniques for piecewise affine systems were recently developed [98,
245, 157, 34, 159, 273, 219], that allow one to derive models (or parts of models)
from input/output data.
Chapter 18
18.1
Problem Formulation
Consider the PWA system (17.1) subject to hard input and state constraints
Ex(t) + Lu(t) M
(18.1)
for t 0, and rewrite its restriction over the set of states and inputs defined
by (18.1) as
h
i
x(t)
x(t + 1) = Ai x(t) + B i u(t) + f i if u(t) Ci
(18.2)
s1
where {Ci }i=0
is the new polyhedral partition of the sets of state+input space
n+m
R
obtained by intersecting the sets C i in (17.1) with the polyhedron described
by (18.1). In this chapter we will assume that the sets C i are polytopes.
Define the cost function
J0 (x(0), U0 ) , p(xN ) +
N
1
X
q(xk , uk )
(18.3)
k=0
where xk denotes the state vector at time k obtained by starting from the state
x0 = x(0) and applying to the system model
xk+1 = Ai xk + B i uk + f i
if [ uxkk ] Ci
(18.4)
368
18
N
1
X
k=0
kQxk kp + kRuk kp
(18.5)
If the squared Euclidian norm is used in the cost function (18.3), then we set
p(xN ) = xN P xN and q(xk , uk ) = xk Qxk + uk Ruk with P 0, Q 0 and R 0.
Cost (18.3) is rewritten as
J0 (x(0), U0 ) , xN P xN +
N
1
X
xk Qxk + uk Ruk
(18.6)
k=0
min J0 (x(0), U0 )
(18.7a)
U0
xk+1 = Ai xk + B i uk + f i if [ uxkk ] Ci
subj. to
(18.7b)
x Xf
N
x0 = x(0)
where the column vector U0 , [u0 , . . . , uN 1 ] Rmc N {0, 1}mN , is the optimization vector, N is the time horizon and Xf is the terminal region.
In general, the optimal control problem (18.7) may not have a minimizer for
some feasible x(0). This is caused by discontinuity of the PWA system in the input
space. We will assume, however, that a minimizer U0 (x(0)) exists for all feasible
x(0). Also the optimizer function U0 may not be uniquely defined if the optimal
set of problem (18.7) is not a singleton for some x(0). In this case U0 denotes one
of the optimal solutions.
We will denote by Xk Rnc {0, 1}n the set of states xk that are feasible
for (18.7):
{1,
.
.
.
,
s}
n
nc
,
Xk = x R {0, 1} x
i
[ u ] C and
(18.8)
Ai x + B i u + f i Xk+1
k = 0, . . . , N 1,
XN = Xf .
The definition of Xi requires that for any initial state xi Xi there exists a feasible
sequence of inputs Ui , [ui , . . . , uN 1 ] which keeps the state evolution in the
feasible set X at future time instants k = i + 1, . . . , N 1 and forces xN into
Xf at time N . The sets Xk for i = 0, . . . , N play an important role in the the
solution of the optimal control problem. They are independent of the cost function
and of the algorithm used to compute the solution to problem (18.7). As in the
18.2
369
case of linear systems (see Chapter 12.2) there are two ways to rigourously define
and compute the sets Xi : the batch approach and the recursive approach. In this
chapter we will not discuss the details on the computation of Xi . Also, we will not
discuss invariant and reachable sets for hybrid systems. While the basic concepts
are identical to the one presented in Section 11.1 for linear systems, the discussion
of efficient algorithms requires a careful treatment which goes beyond the scope of
this book. The interested reader is referred to the work in [126, 114] for a detailed
discussion on reachable and invariant sets for hybrid systems.
In the following we need to distinguish between optimal control based on the
squared 2-norm and optimal control based on the 1-norm or -norm. Note that
the results of this chapter also hold when the number of switches is weighted in the
cost function (18.3), if this is meaningful in a particular situation.
In this chapter we will make use of the following definition. Consider system (18.2) and recall that, in general, x = [ xxc ] where xc Rnc are the continuous
states and x Rn are the binary states and u Rmc {0, 1}m where uc Rmc
are the continuous inputs and u Rm are the binary inputs (Section 17.2.2). We
will make the following assumption.
Assumption 18.1 For the discrete-time PWA system (18.2) the mapping (xc (t), uc (t)) 7
xc (t + 1) is continuous.
Assumption 18.1 requires that the PWA function that defines the update of the
continuous states is continuous on the boundaries of contiguous polyhedral cells,
and therefore allows one to work with the closure of the sets Ci without the need
of introducing multi-valued state update equations. With abuse of notation in the
next sections Ci will always denote the closure of Ci . Discontinuous PWA systems
will be discussed in Section 18.7.
18.2
(18.9)
where Rik , i = 1, . . . , Nk is a partition of the set Xk of feasible states x(k), and the
i of the sets Ri has the following form:
closure R
k
k
i , x : x(k) Li (j)x(k) + M i (j)x(k) N i (j) ,
R
k
k
k
k
(18.10)
j = 1, . . . , nik , k = 0, . . . , N 1,
and
x(k + 1) = Ai x(k)
+ B ii uk (x(k)) + f i
h
x(k)
if u (x(k)) Ci , i = {1, . . . , s}.
k
(18.11)
370
18
Proof: The piecewise linearity of the solution was first mentioned by Sontag
in [257]. In [195] Mayne sketched a proof. In the following we will give the proof
for u0 (x(0)), the same arguments can be repeated for u1 (x(1)), . . . , uN 1 (x(N 1)).
Case 1: (ml = nl = 0) no binary inputs and states
Depending on the initial state x(0) and on the input sequence U = [u0 ,. . .,
uk ], the state xk is either infeasible or it belongs to a certain polyhedron
Ci , k = 0, . . . , N 1. The number of all possible locations of the state
N
sequence x0 , . . . , xN 1 is equal to sN . Denote by {vi }si=1 the set of all possible
switching sequences over the horizon N , and by vik the k-th element of the
sequence vi , i.e., vik = j if xk Cj .
Fix a certain vi and constrain the state to switch according to the sequence
vi . Problem (18.3)-(18.7) becomes
Jvi (x(0)) ,
{U0 }
(18.12a)
k
k
k
xk+1 = Avi xk + B vi uk + f vi
[ uxkk ] Cvi
subj. to
(18.12b)
k = 0, . . . , N 1
X
f
N
x0 = x(0)
(18.13)
SN
where Di = j=1 T i,j is a polyhedral partition of the convex set Di of feasible
states x(0) for problem (18.12). N r i is the number of regions of the polyhedral
partition of the solution and it is a function of the number of constraints in
problem (18.12). The upper index i in (18.13) denotes that the input ui0 (x(0))
is optimal when the switching sequence vi is fixed.
SN
The set X0 of all feasible states at time 0 is X0 = si=1 Di and in general
it is not convex. Indeed, as some initial states can be feasible for different
switching sequences, the sets Di , i = 1, . . . , sN , in general, can overlap. The
solution u0 (x(0)) to the original problem (18.3)-(18.7) can be computed in
the following way. For every polyhedron T i,j in (18.13),
ri
(18.14)
18.2
371
u0 (x(0)) =
i,j ,
F i,j x(0) + G
l,m ,
F l,m x(0) + G
i,j
i,j or
F x(0) + G
l,m
l,m
F x(0) + G
x(0) T (i,j),(l,m) :
Jvi (x(0)) < Jvl (x(0))
x(0) T (i,j),(l,m) :
Jvi (x(0)) > Jvl (x(0))
(18.15)
x(0) T (i,j),(l,m) :
Jvi (x(0)) = Jvl (x(0))
Because Jvi and Jvl are quadratic functions of x(0) on T i,j and T l,m respectively, we find the expression (18.10) of the control law domain. The
sets T i,j \ T l,m and T l,m \ T i,j are two single feasibility non-Euclidean
polyhedra which can be partitioned into a set of single feasibility polyhedra, and thus be described through (18.10) with Lik = 0.
In order to conclude the proof, the general case of n intersecting polyhedra
has to be discussed. We follow three main steps. Step 1: generate one polyhedron of nth -ple feasibility and 2n 2 polyhedra, generally non-Euclidean
and possibly empty and disconnected, of single, double, . . ., (n 1)th -ple
feasibility. Step 2: the ith -ple feasibility non-Euclidean polyhedron is partitioned into several ith -ple feasibility polyhedra. Step 3: any ith -ple feasibility
polyhedron with i > 1 is further partitioned into at most i subsets (18.10)
where in each one of them a certain feasible value function is greater than all
the others. The procedure is depicted in Figure 18.1 when n = 3.
372
18
1,2
2
1,3
1,2,3
3
(a) Step 1.
3
(b) Step 2 and Step 3.
Figure 18.1 Graphical illustration of the main steps for the proof of Theorem 18.1 when 3 polyhedra intersect. Step 1: the three intersecting polyhedra are partitioned into: one polyhedron of triple feasibility (1,2,3), 2
polyhedra of double feasibility (1, 2) and (1, 3), 3 polyhedra of single feasibility (1),(2),(3). The sets (1), (2) and (1,2) are neither open nor closed
polyhedra. Step 2: the single feasibility sets of (1), (2) and (3) are partitioned into six polyhedra. Step 3: value functions are compared inside the
polyhedra of multiple feasibility.
From the result of the theorem above one immediately concludes that the value
function J0 is piecewise quadratic:
J0 (x(0)) = x(0) H1i x(0) + H2i x(0) + H3i if x(0) Ri0 ,
(18.16)
The proof of Theorem 18.1 gives useful insights into the properties of the sets
Rik in (18.10). We will summarize them next.
Each set Rik has an associated multiplicity j which means that j switching
sequences are feasible for problem (18.3)-(18.7) starting from a state x(k) Rik .
If j = 1, then Rik is a polyhedron. In general, if j > 1 the boundaries of Rik
can be described either by an affine function or by a quadratic function. In the
sequel boundaries which are described by quadratic functions but degenerate to
hyperplanes or sets of hyperplanes will be considered affine boundaries.
Quadratic boundaries arise from the comparison of value functions associated
with feasible switching sequences, thus a maximum of j 1 quadratic boundaries
can be present in a j-ple feasible set. The affine boundaries can be of three types.
Type a: they are inherited from the original j-ple feasible non-Euclidean
polyhedron. In this case across such boundaries the multiplicity of the feasibility changes.
Type b: they are artificial cuts needed to describe the original j-ple feasible
non-Euclidean polyhedron as a set of j-ple feasible polyhedra. Across type b
boundaries the multiplicity of the feasibility does not change.
18.2
Type c: they arise from the comparison of quadratic value functions which
degenerate in an affine boundary.
In conclusion, we can state the following lemma
Lemma 18.1 The value function Jk
1. is a quadratic function of the states inside each Rik
2. is continuous on quadratic and affine boundaries of type b and c
3. may be discontinuous on affine boundaries of type a,
and the optimizer uk
1. is an affine function of the states inside each Rik
2. is continuous across and unique on affine boundaries of type b
3. is non-unique on quadratic boundaries, except possibly at isolated points.
4. may be non-unique on affine boundaries of type c,
5. may be discontinuous across affine boundaries of type a.
Based on Lemma 18.1 above one can highlight the only source of discontinuity
of the value function: affine boundaries of type a. The following corollary gives a
useful insight on the class of possible value functions.
Corollary 18.1 J0 is a lower-semicontinuous PWQ function on X0 .
Proof: The proof follows from the result on the minimization of lower-semicontinuous
point-to-set maps in [44]. Below we give a simple proof without introducing the
notion of point-to-set maps.
Only points where a discontinuity occurs are relevant for the proof, i.e., states
belonging to boundaries of type a. From Assumption 18.1 it follows that the feasible
switching sequences for a given state x(0) are all the feasible switching sequences
j contains x(0). Consider a state x(0)
associated with any set Rj0 whose closure R
0
belonging to boundaries of type a and the proof of Theorem 18.1. The only case of
discontinuity can occur when (i) a j-ple feasible set P1 intersects an i-ple feasible
set P2 with i < j, (ii) there exists a point x(0) P1 , P2 and a neighborhood
N (x(0)) with x, y N (x(0)), x P1 , x
/ P2 and y P2 , y
/ P1 . The proof
follows from the previous statements and the fact that J0 (x(0)) is the minimum of
all Jvi (x(0)) for all feasible switching sequences vi .
The result of Corollary 18.1 will be used extensively in the next sections. Even
if value function and optimizer are discontinuous, one can work with the closure
Rjk of the original sets Rjk without explicitly considering their boundaries. In fact,
if a given state x(0) belongs to several regions R10 ,. . . ,Rp0 , then the minimum value
among the optimal values (18.16) associated with each region R10 , . . . , Rp0 allows us
to identify the region of the set R10 , . . . , Rp0 containing x(0).
Next we show some interesting properties of the optimal control law when we
restrict our attention to smaller classes of PWA systems.
373
374
18
Corollary 18.2 Assume that the PWA system (18.2) is continuous, and that E =
0 in (18.1) and Xf = Rn in (18.7) (which means that there are no state constraints,
i.e., P is unbounded in the x-space). Then, the value function J0 in (18.7) is
continuous.
Proof: Problem (18.7) becomes a multi-parametric program with only input
constraints when the state at time k is expressed as a function of the state at time 0
and the input sequence u0 , . . . , uk1 , i.e., xk = fP W A (( (fP W A (x0 , u0 ), u1 ), . . . , uk2 ), uk1 ).
J0 in (18.3) will be a continuous function of x0 and u0 , . . . , uN 1 since it is the
composition of continuous functions. By assumptions the input constraints on
u0 , . . . , uN 1 are convex and the resulting set is compact. The proof follows from
the continuity of J and Theorem 6.2.
Note that E = 0 is a sufficient condition for ensuring that constraints (18.1)
are convex in the optimization variables u0 , . . . , un . In general, even for continuous
PWA systems with state constraints it is difficult to find weak assumptions ensuring
the continuity of the value function J0 . Ensuring the continuity of the optimal
control law u(k) = uk (x(k)) is even more difficult. A list of sufficient conditions for
U0 to be continuous can be found in [100]. In general, they require the convexity (or
a relaxed form of it) of the cost J0 (U0 , x(0)) in U0 for each x(0) and the convexity
of the constraints in (18.7) in U0 for each x(0). Such conditions are clearly very
restrictive since the cost and the constraints in problem (18.7) are a composition
of quadratic and linear functions, respectively, with the piecewise affine dynamics
of the system.
The next theorem provides a condition under which the solution uk (x(k)) of
the optimal control problem (18.3)-(18.7) is a PPWA state-feedback control law.
Theorem 18.2 Assume that the optimizer U0 (x(0)) of (18.3)-(18.7) is unique
for all x(0). Then the solution to the optimal control problem (18.3)-(18.7) is a
PPWA state feedback control law of the form
uk (x(k)) = Fki x(k) + Gik if x(k) Pki k = 0, . . . , N 1,
(18.17)
18.3
0
1
2
u(t) if
x(t)
+
3
1
2
x(t + 1) =
1
1 2
u(t) if
x(t) +
0
1
1
u(t) [1, 1]
375
x(t) C 1 = {x : [0 1]x 0}
x(t) C2 = {x : [0 1]x < 0}
R = 10, P = Q, Xf = X .
1
1
(18.18)
1
,
15
The possible switching sequences are v1 = {1, 1}, v2 = {1, 2}, v3 = {2, 1}, v4 = {2, 2}.
The solution to problem (18.12) is depicted in Figure 18.2. In Figure 18.3(a) the
four solutions are intersected, the white region corresponds to polyhedra of multiple
feasibility. In Figure 18.3(b) we plot with different colors the regions of the statespace partition where the switching sequences v1 = {1, 1}, v2 = {1, 2}, v3 = {2, 1},
v4 = {2, 2} are optimal.
18.3
if x(k) Pki ,
(18.19)
(18.20)
and the value function Jvi is piecewise affine on polyhedra and convex. The rest
of the proof follows the proof of Theorem 18.1. Note that in this case the value
functions to be compared are piecewise affine and not piecewise quadratic.
x2
18
x2
376
x1
x1
(b) v2 = {1, 2}
x2
x2
(a) v1 = {1, 1}
x1
(c) v2 = {2, 1}
x1
(d) v4 = {2, 2}
Figure 18.2 Example 18.1: Problem (18.12) with cost (18.6) is solved for
different vi , i = 1, . . . , 4. Control law u0 (x0 ) is PPWA. State space partition
is shown.
18.4
x2
Computation of the Optimal Control Input via Mixed Integer Programming 377
x2
18.4
v1
v2
v2
v4
x1
v3
x1
Figure 18.3 Example 18.1: State-space partition corresponding to the optimal control law of
quadratic program, respectively, for which efficient branch and bound algorithms
exist.
Consider the equivalent MLD representation (17.25) of the PWA system (18.2).
Problem (18.7) is rewritten as:
J0 (x(0)) =
minU0 J0 (x(0), U0 )
(18.21a)
xk+1 = Axk + B1 uk + B2 k + B3 zk
X
N
f
x0 = x(0)
subj. to
N
1
X
k=0
(18.22)
when p = 1 or p = or
J0 (x(0), U0 ) , xN P xN +
N
1
X
xk Qxk + uk Ruk + k Q k + zk Qz zk
(18.23)
k=0
when p=2.
The optimal control problem (18.21) with the cost (18.22) can be formulated as
a Mixed Integer Linear Program (MILP). The optimal control problem (18.21) with
378
18
the cost (18.23) can be formulated as a Mixed Integer Quadratic Program (MIQP).
The compact form for both cases is
min
subj. to G w + Sx(0)
where H1 , H2 , H3 , c1 , c2 , G, w, S are matrices of suitable dimensions, = [c , d ]
where c , d represent continuous and discrete variables, respectively and H1 , H2 ,
H3 , are null matrices if problem (18.24) is an MILP.
The translation of (18.21) with cost (18.23) into (18.24) is simply obtained by
substituting the state update equation
xk = Ak x0 +
k1
X
(18.25)
j=0
(18.26)
u0 + . . . + uN 1 + x0 + . . . + xN
(18.27)
also solves the original problem, i.e., the same optimum J0 (x(0)) is achieved.
Therefore, problem (18.21) with cost (18.22) can be reformulated as the following MILP problem
min
J()
subj. to
1m uk
1n xk
1n xN
xk+1
E2 k
xN
x0
=
+
Ruk , k = 0, 1, . . . , N 1
Pk1
Q(Ak x0 + j=0 Aj (B1 uk1j +
B2 k1j + B3 zk1j )) k = 0, . . . , N 1
PN 1
P (AN x0 + j=0 Aj (B1 uk1j +
B2 k1j + B3 zk1j ))
Axk + B1 uk + B2 k + B3 zk , k 0
E3 zk E1 uk + E4 xk + E5 , k 0
Xf
x(0)
(18.28)
18.5
379
where the variable x(0) in (18.28) appears only in the constraints as a parameter
vector.
Given a value of the initial state x(0), the MIQP (18.24) or the MILP (18.28)
can be solved to get the optimizer (x(0)) and therefore the optimal input U0 (0).
In the next Section 18.5 we will show how multiparametric programming can be
also used to efficiently compute the piecewise affine state feedback optimal control
law (18.9) or (18.19).
Example 18.2 Consider the problem of steering in three steps the simple piecewise
affine system presented in Example 17.1 to a small region around the origin. The
system state update equations are
1
3
0.4
u(t) if
1 0 x(t) 0
x(t) +
1
3
1
x(t + 1) =
3
0
1
x(t)
+
u(t) if
1 0 x(t)
0.4
3
1
y(t) =
0 1 x(t)
(18.29)
subject to the constraints
x(t) [5, 5] [5, 5]
u(t) [1, 1]
(18.30)
18.5
380
18
which appears in the objective function and in the linear part of the right-hand side
of the constraints (18.24), can be considered as a vector of parameters. Then, for
performance indices based on the -norm or 1-norm, the optimization problem can
be treated as a multiparametric MILP (mp-MILP), while for performance indices
based on the 2-norm, the optimization problem can be treated as a multiparametric
MIQP (mp-MIQP). Solving an mp-MILP (mp-MIQP) amounts to expressing the
solution of the MILP (MIQP) (18.24) as a function of the parameters x(0) .
In Section 7.4.1 we have presented an algorithm for solving mp-MILP problems,
while, to the authors knowledge, there does not exist an efficient method for solving
general mp-MIQPs. In Section 18.6 we will present an algorithm that efficiently
solves the specific mp-MIQPs derived from optimal control problems for discretetime hybrid systems.
18.6
18.6.1
for
i = 1, . . . , NR ,
(18.31)
Fj x + Gj | x Rj and i < j : x
/ Ri ,
j {1, . . . , NR },
18.6
F1 x + G1
o (x) =
y FN x + GN
R
R
if
..
.
x P1
if
x PN R .
381
Note that given a PWA map (18.31) the corresponding ordered region single-valued
function o changes if the order used to store the regions Ri and the corresponding
affine gains change. For illustration purposes consider the example depicted in
Figure 18.4, where x R, NR = 2, F1 = 0, G1 = 0, R1 = [2, 1], F2 = 1, G2 = 0,
R2 = [0, 2].
In the following, we assume that the sets Rki in the optimal solution (18.9) can
overlap. When we refer to the PWA function uk (x(k)) in (18.9) we will implicitly
mean the ordered region single-valued function associated with the map (18.9).
Example 18.3 Let J1 : P1 R and J2 : P2 R be two quadratic functions,
J1 (x) , x L1 x + M1 x + N1 and J2 (x) , x L2 x + M2 x + N2 , where P1 and P2
are convex polyhedra and Ji (x) = + if x
/ Pi , i {1, 2}. Let u1 : P1 Rm ,
m
u2 : P2 R be vector functions. Let P1 P2 , P3 6= and define
J (x)
u (x)
(18.32)
(18.33)
J (x)
u (x)
J1 (x) if x P1
J2 (x) if x P2
u1 (x) if x P1
u2 (x) if x P2
(18.34)
(18.35)
u (x)
382
18
(x)
F2 x + G2
F1 x + G1
x
(a) Multi valued PWA map .
F1 x + G1 if x R1
12 (x) =
y F2 x + G2 if x R2
F2 x + G2 if x R2
21 (x) =
y F1 x + G1 if x R1
function 12 .
function 21 .
18.6
383
minx
subj. to
(18.38)
18.6.2
(18.39)
(18.40)
and it is equivalent to
subj. to f (x, u) P,
..
J (x) = min
.,
subj. to f (x, u) P
(18.41)
(18.42)
subj. to f (x, u) P
(18.43)
384
18
is a one to one problem and therefore it is solvable by an mp-QP. Let the solution
of the i-th mp-QPs be
i,j , x T i,j , j = 1, . . . , N r i ,
ui (x) = F i,j x + G
(18.44)
SN r i
where T i = j=1 T i,j is a polyhedral partition of the convex set T i of feasible x for
the ith sub-problem and N r i is the corresponding number of polyhedral regions.
The feasible set X satisfies X = T 1 = . . . = T d since the constraints of the d
sub-problems are identical.
The solution u (x) to the original problem (18.40) is obtained by comparing
and storing the solution of d mp-QP subproblems (18.42)-(18.43) as explained
in Example 18.3. Consider the case d = 2, and consider the intersection of the
polyhedra T 1,i and T 2,l for i = 1, . . . , N r 1 , l = 1, . . . , N r 2 . For all T 1,i T 2,l ,
T (1,i),(2,l) 6= the optimal solution is stored in an ordered way as described in
Example 18.3, while paying attention to the fact that a region could be already
stored. Moreover, when storing a new polyhedron with the corresponding value
function and optimizer, the relative order of the regions already stored must not
be changed. The result of this Intersect and Compare procedure is
u (x) = F i x + Gi if x Ri ,
Ri , {x : x Li (j)x + M i (j)x N i (j), j = 1, . . . , ni },
where R =
SNR
j=1
(18.45)
if x Dj , j = 1, . . . , N D ,
(18.46)
where Jj (x) are multiple quadratic functions defined over the convex polyhedra
Dj . The polyhedron Dj can contain several regions Ri or can coincide with one of
them. Note that (18.45) and (18.46) have to be considered as PWA and PPWQ
functions in the ordered region sense.
If d > 2 then the value function in (18.46) is intersected with the solution
of the third mp-QP sub-problem and the procedure is iterated by making sure
not to change the relative order of the polyhedra and corresponding gain of the
solution constructed in the previous steps. The solution will still have the same
form (18.45) (18.46).
18.6
385
program to solve is
subj. to f (x, u) P1
f (x, u) P
..,
J (x) = min
subj. to f (x, u) Pr
f (x, u) P
(18.47)
The proof follows the lines to the proof of the previous theorem with the exception that the constraints of the i-th mp-QP subproblem differ from the j-th mp-QP
subproblem, i 6= j.
The lower-semicontinuity assumption on q(x) allows one to use the closure of the
sets Pi in (18.47).The cost function in problem (18.39) is lower-semicontinuous since
it is a composition of a lower-semicontinuous function and a continuous function.
Then, since the domain is compact, problem (18.47) admits a minimum. Therefore
for a given x, there exists one mp-QP in problem (18.47) which yields the optimal
solution. The procedure based on solving mp-QPs and storing the results as in
S R j
Example 18.3 will be the same as in Lemma 18.3 but the domain R = N
j=1 R of
the solution can be a non-Euclidean polyhedron.
If f is PPWA defined over s regions then we have a s to X problem where X
can belong to any of the problems listed above. In particular, we have an s to
r problem of multiplicity d if f is PPWA and defined over s regions and q is a
multiple PPWQ function of multiplicity d, defined over r polyhedral regions. The
following lemma can be proven along the lines of the proofs given before.
18.6.3
i
where
n fPoW A : C R and fP W A (x, u) = A x + B u + f if [ u ] C , i = 1, . . . , s,
386
18
(18.7),
Jj (x(j)) ,
minuj
(18.49)
subj. to
fP W A (x(j), uj ) Xj+1
(18.50)
JN
(x) = x P x,
(18.51)
(18.52)
where Xj is the set of all states x(j) for which problem (18.49)(18.50) is feasible:
Xj = {x Rn | u, fP W A (x, u) Xj+1 }.
(18.53)
Equations (18.49)-(18.53) are the discrete-time version of the well known HamiltonJacobi-Bellman equations for continuous-time optimal control problems.
Assume for the moment that there are no binary inputs and binary states,
m = n = 0. The Bellman equations (18.49)(18.52) can be solved backwards in
time by using a multi-parametric quadratic programming solver and the results of
the previous section.
Consider the first step of the dynamic program (18.49)(18.52)
JN
1 (xN 1 ) ,
min{uN 1 }
xN 1 QxN 1 + uN 1 RuN 1 + JN
(fP W A (xN 1 , uN 1 ))
(18.54)
subj. to
fP W A (xN 1 , uN 1 ) Xf .
(18.55)
18.6
387
Algorithm 18.1
input: CFTOC problem (18.3)(18.7)
output: Solution (18.9) in the ordered region sense.
let RN = {Xf }
let JN,1
(x) = x P x
for k = N 1, . . . , 1,
for i = 1, . . . , |Rk+1 |,
for j = 1, . . . , s,
let Sk,i,j = {}
solve the mp-QP
Sk,i,j
minuk
subj. to
end
end
In Algorithm 18.1, the structure Sk,i,j stores the matrices defining quadratic
function Jk,i,j,l
(), affine function uk,i,j,l (), and polyhedra Rk,i,j,l , for all l:
Sk,i,j =
[
l
Jk,i,j,l
(x), uk,i,j,l (x), Rk,i,j,l
(18.56)
where the indices in (18.56) have the following meaning: k is the time step, i
indexes the piece of the cost-to-go function that the DP algorithm is considering,
j indexes the piece of the PWA dynamics the DP algorithm is considering, and l
indexes the polyhedron in the mp-QP solution of the (k, i, j)th mp-QP problem.
The KEEP only triplets Step of Algorithm 18.1 aims at discarding regions
Rk,h that are completely covered by some other regions that have lower cost. Obviously, if there are some parts of the region Rk,h that are not covered at all by other
regions (first condition) we need to keep them. Note that comparing the cost functions is, in general, non-convex optimization problem. One might consider solving
the problem exactly, but since algorithm works even if some removable regions are
kept, we usually formulate LMI relaxation of the problem at hand.
The output of Algorithm 18.1 is the state-feedback control law (18.9) considered
in the ordered region sense. The online implementation of the control law requires
simply the evaluation of the PWA controller (18.9) in the ordered region sense.
388
Examples
x2
Example 18.4 Consider the control problem of steering the piecewise affine system (18.29) to a small region around the origin. The constraints and cost function
of Example (18.2) are used. The state-feedback solution u0 (x(0)) was determined
by using the approach presented in the previous section. When the infinity norm
is used in the cost function, the feasible state-space X0 at time 0 is divided into 84
polyhedral regions and it is depicted in Fig. 18.5(a). When the squared Euclidean
norm is used in the cost function, the feasible state-space X0 at time 0 is divided into
92 polyhedral regions and is depicted in Fig. 18.5(b).
x2
18.6.4
18
x1
x1
Figure 18.5 Example 18.4: state space control partition for u0 (x(0))
Note that as explained in Section 18.6.2 for the case of the squared Euclidian norm,
the optimal control law is stored in a special data structure where:
18.7
389
ple 17.2
x(t+1) =
0.90
0.02
0.90
0.06
0.02
0.00
0.02
0.00
0.90 0.38
0.38 0.52
0.90 0.35
1.04 0.35
x(t) + [ 0.10
0.02 ] u1 (t) +
x(t) + [ 0.10
0.02 ] u1 (t) +
x(t) + [ 0.10
0.38 ] u1 (t) +
0.01
0.02
0.07
0.15
0.10
0.38
0.75
(18.58)
x(t) + [ 0.10
0.35 ] u1 (t) +
2.60
x2
x2
We solve the finite time constrained optimal control problem (18.7) with cost (18.6)
0
with N = 3, P = Q = [ 10 01 ], R = [ 0.2
0 1 ]. The state-feedback solution was determined by using the approach presented in the previous section in two cases:
without terminal constraint (Figure 18.6(a)) and with terminal constraint Xf =
[0.01 0.01] [0.01 0.01] (Figure 18.6(b)).
x1
x1
constraint (N r 3 = 183).
straint (N r 3 = 181).
18.7
390
18
Under the assumption that the optimizer exists for all states x(k), the approach explained in the previous sections can be applied to discontinuous systems
by considering three elements. First, the PWA system (18.2) has to be defined on
each polyhedron of its domain and all its lower dimensional facets. Secondly, dynamic programming has to be performed from and to any lower dimensional
facet of each polyhedron of the PWA domain. Finally, value functions are not
lower-semicontinuous, which implies that Lemma 18.4 cannot by used. Therefore,
when considering the closure of polyhedral domains in multi-parametric programming (18.47), a post-processing is necessary in order to remove multi-parametric
optimal solutions which do not belong to the original set but only to its closure.
The tedious details of the dynamic programming algorithm for discontinuous PWA
systems are not included here but can be immediately extracted from the results
of the previous sections.
In practice, the approach just described for discontinuous PWA systems can
easily be numerically prohibitive. The simplest approach from a practical point of
view is to introduce gaps between the boundaries of any two polyhedra belonging
to the PWA domain (or, equivalently, to shrink by a quantity the size of every
polyhedron of the original PWA system). This way, one deals with PWA systems
defined over a disconnected union of closed polyhedra. By doing so, one can use
the approach discussed in this chapter for continuous PWA systems. However, the
optimal controller will not be defined at the points in the gaps. Also, the computed
solution might be arbitrarily different from the original solution to problem (18.3)(18.7) at any feasible point x. Despite this, if the dimension of the gaps is
close to the machine precision and comparable to sensor/estimation errors, such an
approach can be very appealing in practice. To the best of our knowledge in some
cases this approach is the only one that is computationally tractable for computing
controllers for discontinuous hybrid systems fulfilling state and input constraints
that are implementable in real-time.
Without Assumption 18.1, problem (18.3)-(18.7) is well defined only if an optimizer exists for all x(0). In general, this is not easy to check. The dynamic
programming algorithm described here could be used for such a test but the details are not included in this book.
18.8
18.8
391
min
Jt (x(t), Utt+N |t )
(18.59a)
x
Ci
xt+k+1|t = Ai xt+k|t + B i ut+k|t + f i if ut+k|t
t+k|t
subj. to
(18.59b)
x
Xf
t+N |t
xt|t = x(t)
Utt+N |t
is solved at each time t, where Utt+N |t = {ut|t , . . . , ut+N 1|t }. Let Ut = {ut|t , . . . , ut+N 1|t }
be the optimal solution of (18.59) at time t. Then, the first sample of Ut is applied
to system (18.2):
u(t) = ut|t .
(18.60)
The optimization (18.59) is repeated at time t+1, based on the new state xt+1|t+1 =
x(t + 1), yielding a moving or receding horizon control strategy.
Based on the results of previous sections the state feedback receding horizon
controller (18.59)(18.60) can be immediately obtained in two ways: (i) solve the
MIQP/MILP (18.24) for xt|t = x(t) or (ii) by setting
u(t) = f0 (x(t)),
(18.61)
18.8.1
18.8.2
Examples
Example 18.6 Consider the problem of regulating the piecewise affine system (18.29)
to the origin. The finite time constrained optimal control (18.7) is solved with N = 3,
P = Q = [ 10 01 ], R = 1, and Xf = [0.01 0.01] [0.01 0.01]. Its state feedback
solution (18.9) u (x(0)) = f0 (x(0)) at time 0 is implemented in a receding horizon
fashion, i.e. u(x(k)) = f0 (x(k)).
392
18
The state feedback control law with cost (18.5) with p = has been computed in
Example 18.4 and it consists of 84 polyhedral regions. None of them has multiplicity
higher than 1. Figure 18.7 shows the corresponding closed-loop trajectories starting
from the initial state x(0) = [2 2] .
x1 and x2
The state feedback control law with cost (18.6) has been computed in Example 18.4
and it consists of 92 polyhedral regions, some of them have multiplicity higher than 1.
Figure 18.8 shows the corresponding closed-loop trajectories starting from the initial
state x(0) = [2 2] .
time
time
(b) Optimal input.
x1 and x2
Figure 18.7 Example 18.6: MPC control of system (18.29) when the infinity
norm is used in the objective.
time
(a) State trajectories (x1 dashed
time
(b) Optimal input.
18.8
393
system (18.57) described in Examples 17.2 and 18.5 to the origin. The finite time
constrained optimal control problem (18.7) with cost (18.6) is solved with N = 3,
0
P = Q = [ 10 01 ], R = [ 0.2
0 1 ]. Its state feedback solution (18.9) u (x(0)) = f0 (x(0)) at
x1 and x2
u1 and u2
Comparing the two controllers, we find that the terminal constraint leads to a more
aggressive behaviour. In addition, in the case with terminal constraint we observe the
coordination of the binary and continuous inputs. The small damping coefficient is
switched on to increase the effect of the continuous input. The damping is increased
again once the steady-state value is approached.
time
(a) State trajectories (x1 dashed
line and x2 solid line).
time
(b) Optimal inputs (u1 solid line
and the binary input u2 with a
dashed line)
Figure 18.9 Example 18.7: MPC control of system (18.57) without terminal
constraint
18
x1 and x2
u1 and u2
394
time
(a) State trajectories (x1 dashed
time
(b) Optimal inputs (u1 solid line
and the binary input u2 with a
dashed line)
Figure 18.10 Example 18.7: MPC control of system (18.57) with terminal
constraint
List of Figures
2.1
2.2
3.1
3.2
3.3
3.4
3.5
3.6
4.1
4.2
4.3
4.4
4.5
4.6
22
23
26
27
29
30
39
40
47
52
56
58
396
LIST OF FIGURES
4.7
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9
. . . . . . . . . . . . . . . 64
H-polyhedron. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
72
76
76
79
5.10 Two dimensional example: partition of the rest of the space Y\R0 .
5.11 Illustration of the Pontryagin difference and Minkowski sum operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.12 Illustration that (P Q) Q P. . . . . . . . . . . . . . . . . . .
5.13 Graphical illustration of Algorithm 5.3. . . . . . . . . . . . . . . .
. 81
. 82
. 83
. 87
6.1
6.2
6.3
6.4
6.5
6.6
Example 6.5: Open and not closed point-to-set map R(x). . . . . . . . . 100
7.1
7.2
7.3
7.4
LIST OF FIGURES
397
7.4
8.1
. 119
. 120
. 121
. 122
. 126
. 130
. 142
11.1 Example 11.1: one-step controllable set Pre(X ) X for system (11.8)
under constraints (11.9). . . . . . . . . . . . . . . . . . . . . . . . . .
11.2 Example 11.1: one-step reachable set for system (11.8). . . . . . . . . .
11.3 Example 11.2: one-step controllable set Pre(X ) X for system (11.15)
under constraints (11.16). . . . . . . . . . . . . . . . . . . . . . . . .
11.4 Example 11.2: one-step reachable sets for system (11.15) under constraints
(11.16). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
11.5 Example 11.3: one-step controllable sets Kj (S) for system (11.15) under
. 186
. 186
. 187
. 188
constraints (11.16) for N=1,2,3,4. Note that the sets are shifted along the
x-axis for a clearer visualization. . . . . . . . . . . . . . . . . . . . . . . 189
11.6 Example 11.3: N-step reachable sets for system (11.15) under constraints
(11.16). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
11.7 Maximal Positive Invariant Set of system (11.8) under constraints (11.9). 192
11.8 Maximal Control Invariant Set of system (11.15) subject to constraints (11.16). 194
11.9 Example 11.8: one-step robust controllable set Pre(X , W) X for system
(11.32) under constraints (11.33)-(11.34). . . . . . . . . . . . . . . . . . 199
11.10Example 11.8: one-step robust reachable set for system (11.32) under
constraints (11.34). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
11.11Example 11.9: one-step robust controllable set Pre(X , W) X for system
(11.42) under constraints (11.43)-(11.44). . . . . . . . . . . . . . . . . . 201
11.12Example 11.9: one-step robust reachable set for system (11.42) under
constraints (11.43)-(11.44). . . . . . . . . . . . . . . . . . . . . . . . . . 202
11.13Example 11.10: one-step robust controllable set Pre(X , W)X for system
(11.53) under constraints (11.54). . . . . . . . . . . . . . . . . . . . . . 204
398
LIST OF FIGURES
11.14Example 11.10: one-step robust reachable set Pre(X , W) X for system
(11.60) under constraints (11.61). . . . . . . . . . . . . . . . . . . . . . 205
11.15Maximal Robust Positive Invariant Set of system (11.65) subject to constraints (11.66). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
11.16Maximal Robust Control Invariant Set of system (11.42) subject to constraints (11.43). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
12.1 Example 12.1: Propagation of the feasible sets Xi to arrive at C (shaded). 217
12.2 CLQR Algorithm: Region Exploration. . . . . . . . . . . . . . . . . . . 224
12.3 Example 12.2: Double Integrator, 2-norm objective function, horizon
12.4
12.5
12.6
12.7
N = 6. Partition of the state space for the time-varying optimal control law. Polyhedra with the same control law were merged. . . . . . .
Example 12.3: Double Integrator, 2-norm objective function, horizon
N = . Partition of the state space for the time invariant optimal control
law. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Example 12.4: Double Integrator, -norm objective function, horizon
N = 6. Partition of the state space for the time-varying optimal control
law. Polyhedra with the same control law were merged. . . . . . . . . .
Example 12.5: Double Integrator, -norm objective function, horizon
N = . Partition of the state space for the time invariant optimal control
law. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Example 12.6: Double Integrator, minimum-time objective function. ParLQR
tition of the state space and control law. Maximal LQR invariant set O
is the central shaded region in (a). . . . . . . . . . . . . . . . . . . . .
. 238
. 239
. 240
. 241
. 241
. 250
. 251
. 259
. 261
LIST OF FIGURES
14.4 Polyhedron P constructed to be an outer approximation of epi J ,
and an inner approximation of epi J . . . . . . . . . . . . . . . . . . 287
14.5 Several steps of the Double Description algorithm with the termination criterion extreme(P ) epi J (x) satisfied in the last step. . . . 288
14.6 Example of one iteration of the implicit double-description algorithm...288
14.7 Plot of the second order interpolant in 2D centered at zero, with a
width of one. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
14.8 Optimal cost J (x) (blue), and stability function J (x) + (x)(red) . 291
14.9 Error function V (top). Points marked are the vertices with worstcase fit. Partition (triangulation) of the sampled points (middle).
Upper-bound on approximate value function (convex hull of sample
points) (bottom). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
14.10Sub-optimal solution with 35 vertices. Error: V = 1.26 . . . . . . 292
14.11Sub-optimal solution with 45 vertices. Error: V = 0.82 . . . . . . 292
14.12Sub-optimal solution with 55 vertices. Error: V = 0.23 . . . . . . 293
14.13Sub-optimal solution with 61 vertices. Error: V = 0, system stable . 293
14.14Approximated control law u
(x) . . . . . . . . . . . . . . . . . . . . . 293
14.15Approximation error u
(x) u (x) . . . . . . . . . . . . . . . . . . . 293
14.16Approximate control law using triangulation . . . . . . . . . . . . . . 293
14.17Sub-optimal solution with 5 facets. Error: V = . . . . . . . . . . . . 293
14.18Sub-optimal solution with 13 facets. Error: V = . . . . . . . . . . . 293
14.19Error function V (top). Points marked are the vertices with worstcase fit. Partition (triangulation) of the sampled points (middle).
Upper-bound on approximate value function (convex hull of sample
points) (bottom). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
14.20Sub-optimal solution with 21 facets. Error: V = . . . . . . . . . . . 293
14.21Sub-optimal solution with 31 facets. Error: V = 0, system stable . . 293
14.22Partition generated by the second-order interpolant method for Example ??. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
14.23Approximate control law for Example ?? using second-order interpolants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
399
400
LIST OF FIGURES
17.4 Example 17.2: Open-loop simulation of system (17.6) for u1 = 3 and zero
initial conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
17.5 Example 17.3: Open-loop simulation of system (17.10) for different sequences of the input u. . . . . . . . . . . . . . . . . . . . . . . . . . .
17.6 A discrete hybrid automaton (DHA) is the connection of a finite
state machine (FSM) and a switched affine system (SAS), through a
mode selector (MS) and an event generator (EG). The output signals
are omitted for clarity . . . . . . . . . . . . . . . . . . . . . . . . .
17.7 Example of finite state machine . . . . . . . . . . . . . . . . . . . .
. 347
. 348
. 349
. 351
18.1 Graphical illustration of the main steps for the proof of Theorem 18.1
18.2
18.3
18.4
18.5
18.6
18.7
372
376
377
382
388
389
Index
N -Step Controllable Set, 164
N -Step Robust Controllable Set, 176
P-collections
Union of, 69
Hybrid System
2-norm Optimal Control, 329
-norm Optimal Control, 335
Optimal Control, 327
Optimal Control via Mixed-Integer
acronyms, list of, xiii
Programming, 336
Active Constraints, 5
Hybrid Systems
Algebraic Riccati Equation, 148
Binary States, Inputs, and Outputs,
303
Complementary Slackness, 15
Discrete Hybrid Automata, 306
Constrained Least-Squares Problems, 32
Finite State Machine, 309
Constrained LQR, 189
Hysdel Modeling Language, 315
algorithm, 201
Logic and Mixed-Integer InequaliConstrained Optimal Control, 127
ties, 311
Batch Approach, 129
Mixed Logical Dynamical Systems,
Dynamic Programming, 130
313
Infinite Horizon, 132
Model Equivalences, 315
Recursive Approach, 130
Modeling Discontinuities, 302
Value Function Iteration, 133
Models, 299
Constraint Qualifications, 14
Piecewise Affine Systems, 300
Control Lyapunov Function, 233
Hysdel, 315
Convex Hull, 55
Convex Optimization Problems, 8
Invariant Set, 163
Convex Piecewise Linear Optimization,
control invariant, 171
LQR
28
max. LQR invariant (O
), 199
Convexity, 6
max. robust control inv. (C ), 171,
Operations Preserving, 7
185
Critical Region, 86
max. robust pos. inv. (O ), 169,
183
Discrete Hybrid Automata, 306
positive invariant, 169
Discrete Time Riccati Equation, 146
robust control invariant, 185
Dual variables, 12
robust positive invariant, 183
Efficient On-Line Algorithms, 286
1, -Norm Case, 287
2-Norm Case, 288
based on the optimal inputs, 293
based on the value function, 293
descriptor function, 289
Karush-Kuhn-Tucker Conditions, 16
Lagrange
dual function, 12
dual problem, 13
multipliers, 12
402
Index
Lagrange Duality, 12
Lagrangian, 12
Linear Constrained Optimal Control, 189
Feasible Sets, 192
state feedback solution, 198
Linear Program, 22
Dual of, 24
KKT Conditions, 25
Linear Quadratic Regulator, 143
Batch Approach, 144
Infinite Time Solution, 148
Recursive Approach, 145
Logic and Mixed-Integer Inequalities, 311
Transformation of Boolean Relations,
311
Lyapunov Equation, 140
Lyapunov Function, 138
1/ norm, 141
quadratic, 139
radially unbounded, 138
Lyapunov function
{1, }-norm, 141
Lyapunov Inequality, 140
Lyapunov Stability, 136
Definition, 136
Global, 138
algorithm, 113
properties, 105, 110
set of active constraints, 108
Multiparametric Mixed-Integer Program,
116
Multiparametric Program, 73
Example, 77
General Results, 76
main properties, 81
Reducing the Dimension of the Parameter Space, 87
with linear constraints, 85
norm
Matrix -norm, xiii
Matrix 1-norm, xiii
Optimization Problem, 3
Point-to-Set Maps, 79
Continuity, 79
Continuous Selections, 81
Polyhedron, 49
Affine Mappings of, 65
Chebychev Ball, 58
Envelope, 55
Minimal Representation, 54
Minkowski Sum, 62
Maximal Controllable Set, 173
Open, 52
Maximal Robust Controllable Set, 187
P-collection, 53
Minimum-Time Control, 213
Projection, 59
Minkowski Difference
ray, 51
of Polytopes, 62
Set-Difference, 60
Minkowski Differenceof P-collections, 66
Union, 64
Mixed Integer Linear Programming, 33
Valid Inequality, 51
Mixed Integer Quadratic Programming,
Vertex Enumeration, 56
33
Polytopal Complexes, 52
Mixed Logical Dynamical Systems, 313 Polytope, 49
MLD System
Pontryagin Difference
Optimal Control, 336
of Polytopes, 62
Model Predictive Control(MPC), 222
Pontryagin Differenceof P-collections, 66
mp-LP, 88
preface, ii
algorithm, 101
PWA System
formulation, 88, 104
2-norm Optimal Control, 329
nonunique optimizer, 93
-norm Optimal Control, 335
properties, 90, 99
Optimal Control, 327
set of active constraints, 95
mp-QP, 104
Quadratic Program, 29
Index
Dual of, 31
KKT Conditions, 31
reachable set, 164
Receding Horizon Control
Introduction, 134
receding horizon control, 224, 255, 256
Receding Horizon Controller (RHC), 222
RHC
feasibility issue, 229
Implementation, 222
Observer Design, 248
Offset-Free Reference Tracking, 248
stability and feasibility issues, 229
stability issue, 231
Robust Pre
Computation for Linear Systems with
Inputs, 178
Computation for Linear Systems with
Parametric Uncertainty, 181
Computation for Linear Systems without Inputs, 176
Definition, 175
Robust Reach
Definition, 175
Computation for Linear Systems with
Inputs, 178
Computation for Linear Systems with
Parametric Uncertainty, 181
Computation for Linear Systems without Inputs, 176
Robust Invariant Set, 175
Robust Reachable Set, 176
stability
asymptotic stability, 174
Strong Duality, 14
symbols, list of, xiii
Unconstrained Optimal Control
1/ Norm, 151
LQR, 143
403
404
Index
References
[1] L. Chisci A. Bemporad and E. Mosca. On the stabilizing property of
SIORHC. Automatica, 30(12):20132015, 1994.
[2] J. Acevedo and E.N. Pistikopoulos. A multiparametric programming approach for linear process engineering problems under uncertainty. Ind. Eng.
Chem. Res., 36:717728, 1997.
[3] I. Adler and R.D.C. Monteiro. A geometric view of parametric linear programming. Algorithmica, 8(2):161176, 1992.
[4] A. Alessio, M. Lazar, A. Bemporad, and W.P.M.H. Heemels. Squaring the
circle: An algorithm for obtaining polyhedral invariant sets from ellipsoidal
ones. Automatica, 43(12):20962103.
[5] J.C. Allwright and G.C. Papavasiliou. On linear programming and robust
model-predictive control using impulse-responses. Systems & Control Letters,
18:159164, 1992.
[6] R. Alur, C. Belta, F. Ivancic, V. Kumar, M. Mintz, G.J. Pappas, H. Rubin,
and J. Schug. Hybrid modeling and simulation of biomolecular networks.
In M.D. Di Benedetto and A. Sangiovanni Vincentelli, editors, Hybrid Systems: Computation and Control, volume 2034 of Lecture Notes in Computer
Science, pages 1933. Springer-Verlag, 2001.
[7] R. Alur, C. Courcoubetis, T.A. Henzinger, and P.H. Ho. Hybrid automata:
An algorithmic approach to the specification and verification of hybrid systems. In A.P. Ravn R.L. Grossman, A. Nerode and H. Rischel, editors, Hybrid
Systems, volume 736 of Lecture Notes in Computer Science, pages 209229.
Springer Verlag, 1993.
[8] P.J. Antsaklis. A brief introduction to the theory and applications of hybrid systems. Proc. IEEE, Special Issue on Hybrid Systems: Theory and
Applications, 88(7):879886, July 2000.
[9] J.P. Aubin. Viability theory. Systems & Control: Foundations & Applications. Birkh
auser, 1991.
[10] D. Avis. lrs: A revised implementation of the reverse search vertex enumeration algorithm. Polytopes, Combinatorics and Computation, pages 177198,
2000.
406
References
[11] T.A. Badgwell and K.R. Muske. Disturbance model design for linear model
predictive control. In Proceedings of the American Control Conference, volume 2, pages 16211626, 2002.
[12] V.L. Bageshwar and F. Borrelli. On a property of a class of offset-free model
predictive controllers. Automatic Control, IEEE Transactions on, 54(3):663
669, march 2009.
[13] E. Balas. Projection with a minimum system of inequalities. Computational
Optimization and Applications, 10:189193, 1998.
[14] A. Balluchi, L. Benvenuti, M. Di Benedetto, C. Pinello, and A. SangiovanniVincentelli. Automotive engine control and hybrid systems: Challenges and
opportunities. Proc. IEEE, 88(7):888912, 2000.
[15] A. Balluchi, L. Benvenuti, M.D. Di Benedetto, and A. SangiovanniVincentelli. Design of observers for hybrid systems. In C.J. Tomlin and M.R.
Greenstreet, editors, Hybrid Systems: Computation and Control, volume
2289 of Lecture Notes in Computer Science, pages 7689. Springer-Verlag,
Berlin Heidelberg New York, 2002.
[16] B. Bank, J. Guddat, D. Klatte, B. Kummer, and K. Tammer. Non-Linear
Parametric Optimization. Akademie-Verlag, Berlin, 1982.
[17] M. Baotic. An efficient algorithm for multi-parametric quadratic programming. Technical Report AUT02-04, Automatic Control Laboratory, ETH
Zurich, Switzerland, February 2002.
[18] M. Baotic. Gradient of the value function in parametric convex optimization
problems. Technical report, Faculty of Electrical Engineering and Computing.
University of Zagreb, Croatia, 2013. Technical Report.
[19] M. Baotic, F. Borrelli, A. Bemporad, and M. Morari. Efficient on-line computation of constrained optimal control. SIAM Journal on Control and Optimization, 5:24702489, September 2008.
[20] M. Baotic and F.D. Torrisi. Polycover. Technical Report AUT03-11,
Automatic Control Laboratory, ETHZ, Switzerland, 2003. Available from
http://control.ee.ethz.ch/research/publications/publications.msql?
[21] M. Baric. Constrained Control - Computations, Performance and Robustness.
PhD thesis, ETH Zurich, Switzerland, October 2008.
[22] T. Basar and G.J. Olsder. Dynamic Noncooperative Game Theory, Second
Edition. Classics in Applied Mathematics. SIAM, Philadelphia, 1998.
[23] Farhad Bayat, Tor Arne Johansen, and Ali Akbar Jalali. Using hash tables to
manage the time-storage complexity in a point location problem: Application
to explicit model predictive control. Automatica, 47(3):571 577, 2011.
[24] M.S. Bazaraa, H.D. Sherali, and C.M. Shetty. Nonlinear Programming
Theory and Algorithms. John Wiley & Sons, Inc., New York, second edition,
1993.
References
[25] A.G. Beccuti, G. Papafotiou, R. Frasca, and M. Morari. Explicit hybrid
model predictive control of the DC-DC boost converter. In IEEE PESC,
Orlando, Florida, USA, June 2007.
[26] A. Bemporad. Reducing conservativeness in predictive control of constrained
systems with disturbances. In Proc. 37th IEEE Conf. on Decision and Control, pages 13841391, Tampa, Florida, USA, 1998.
[27] A. Bemporad. Efficient conversion of mixed logical dynamical systems into an
equivalent piecewise affine form. IEEE Transactions on Automatic Control,
49(5):832838, 2004.
[28] A. Bemporad, G. Bianchini, and F. Brogi. Passivity analysis and passification
of discrete-time hybrid systems. IEEE Transactions on Automatic Control,
54(4):10041009, 2008.
[29] A. Bemporad, F. Borrelli, and M. Morari. Piecewise linear robust model
predictive control. In Proc. European Control Conf., Porto, Portugal, October
2001.
[30] A. Bemporad and S. Di Cairano. Optimal control of discrete hybrid stochastic
automata. In M. Morari and L. Thiele, editors, Hybrid Systems: Computation
and Control, number 3414 in Lecture Notes in Computer Science, pages 151
167. Springer-Verlag, 2005.
[31] A. Bemporad, A. Casavola, and E. Mosca. Nonlinear control of constrained
linear systems via predictive reference management. IEEE Transactions on
Automatic Control, AC-42(3):340349, 1997.
[32] A. Bemporad, G. Ferrari-Trecate, and M. Morari. Observability and controllability of piecewise affine and hybrid systems. IEEE Transactions on
Automatic Control, 45(10):18641876, 2000.
[33] A. Bemporad, K. Fukuda, and F.D. Torrisi. Convexity recognition of the
union of polyhedra. Computational Geometry, 18:141154, 2001.
[34] A. Bemporad, A. Garulli, S. Paoletti, and A. Vicino. A bounded-error approach to piecewise affine system identification. IEEE Transactions on Automatic Control, 50(10):15671580, October 2005.
[35] A. Bemporad, W.P.M.H. Heemels, and B. De Schutter. On hybrid systems
and closed-loop MPC systems. IEEE Transactions on Automatic Control,
47(5):863869, May 2002.
[36] A. Bemporad, D. Mignone, and M. Morari. Moving horizon estimation for
hybrid systems and fault detection. In Proc. American Contr. Conf., pages
24712475, Chicago, IL, June 1999.
[37] A. Bemporad and M. Morari. Control of systems integrating logic, dynamics,
and constraints. Automatica, 35(3):407427, March 1999.
407
408
References
[38] A. Bemporad and M. Morari. Robust model predictive control: A survey. In
A. Garulli, A. Tesi, and A. Vicino, editors, Robustness in Identification and
Control, number 245 in Lecture Notes in Control and Information Sciences,
pages 207226. Springer-Verlag, 1999.
[39] A. Bemporad, M. Morari, V. Dua, and E.N. Pistikopoulos. The explicit linear
quadratic regulator for constrained systems. Automatica, 38(1):320, 2002.
[40] A. Bemporad, M. Morari, V. Dua, and E.N. Pistikopoulos. The Explicit
Linear Quadratic Regulator for Constrained Systems. Automatica, 38(1):3
20, January 2002.
[41] A. Bemporad, F.D. Torrisi, and M. Morari. Discrete-time hybrid modeling and verification of the batch evaporator process benchmark. European
Journal of Control, 7(4):382399, 2001.
[42] Alberto Bemporad and Carlo Filippi. An algorithm for approximate multiparametric convex programming. Computational Optimization and Applications, V35(1):87108, 2006.
[43] A. Ben-Tal, A. Goryashko, E. Guslitzer, and A. Nemirovski. Adjustable
robust solutions of uncertain linear programs. Mathematical Programming,
99:351376, 2004.
[44] C. Berge. Topological Spaces. Dover Publications, inc., Mineola, New York,
1997.
[45] A.B. Berkelaar, K. Roos, and T. Terlaky. The optimal set and optimal
partition approach to linear and quadratic programming. In T. Gal and
H.J. Greenberg, editors, Advances in Sensitivity Analysis and Parametric
Programming, volume 6 of International Series in Operations Research &
Management Science, chapter 6. Kluwer Academic Publishers, 1997.
[46] Dimitri P Bertsekas. Nonlinear Programming. Athena Scientific, Belmont,
Massachusetts, 2nd edition, 1999.
[47] D.P. Bertsekas. Control of Uncertain Systems with a setmembership description of the uncertainty. PhD thesis, MIT, 1971.
[48] D.P. Bertsekas. Infinite-time reachability of state-space regions by using feedback control. IEEE Transactions on Automatic Control, 17:604613, October
1972.
[49] D.P. Bertsekas. Dynamic Programming and Optimal Control. Athena Scientific, Belmont, Massachusetts, 1995.
[50] D.P. Bertsekas. Dynamic Programming and Optimal Control, volume II.
Athena Scientific, Belmont, Massachusetts, 2nd edition, 2001.
[51] D.P. Bertsekas and I.B. Rhodes. On the minimax reachability of target sets
and target tubes. Automatica, 7:233247, 1971.
References
[52] L.T. Biegler and V.M. Zavala. Large-scale nonlinear programming using
ipopt: An integrating framework for enterprise-wide dynamic optimization.
Computers & Chemical Engineering, 33(3):575 582, 2009. Selected Papers
from the 17th European Symposium on Computer Aided Process Engineering
held in Bucharest, Romania, May 2007.
[53] R.R. Bitmead, M. Gevers, and V. Wertz. Adaptive Optimal Control: The
Thinking Mans GPC. International Series in Systems and Control Engineering. Prentice Hall, 1990.
[54] F. Blanchini. Ultimate boundedness control for uncertain discrete-time systems via set-induced Lyapunov functions. IEEE Transactions on Automatic
Control, 39(2):428433, February 1994.
[55] F. Blanchini. Set invariance in control a survey. Automatica, 35(11):1747
1768, November 1999.
[56] F. Blanchini and S. Miano. Set-Theoretic Methods in Control. Birkhauser,
2009.
[57] F. Borrelli, M. Baotic, A. Bemporad, and M. Morari. Dynamic programming for constrained optimal control of discrete-time linear hybrid systems.
Automatica, 41:17091721, October 2005.
[58] F. Borrelli, M. Baotic, J. Pekar, and G. Stewart. On the computation of
linear model predictive control laws. Automatica, 46(6):1035 1041, 2010.
[59] F. Borrelli, A. Bemporad, M. Fodor, and D. Hrovat. An MPC/hybrid system approach to traction control. IEEE Trans. Contr. Systems Technology,
14(3):541552, May 2006.
[60] F. Borrelli, A. Bemporad, and M. Morari. A geometric algorithm for multiparametric linear programming. J. Opt. Theory and Applications, 118(3),
September 2003.
[61] Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge
University Press, March 2004.
[62] M.S. Branicky. Studies in hybrid systems: modeling, analysis, and control.
PhD thesis, LIDS-TH 2304, Massachusetts Institute of Technology, Cambridge, MA, 1995.
[63] M.S. Branicky. Multiple Lyapunov functions and other analysis tools for
switched and hybrid systems. IEEE Transactions on Automatic Control,
43(4):475482, April 1998.
[64] M.S. Branicky, V.S. Borkar, and S.K. Mitter. A unified framework for hybrid
control: model and optimal control theory. IEEE Transactions on Automatic
Control, 43(1):3145, 1998.
[65] M.S. Branicky and S.K. Mitter. Algorithms for optimal hybrid control. In
Proc. 34th IEEE Conf. on Decision and Control, New Orleans, USA, December 1995.
409
410
References
[66] M.S. Branicky and G. Zhang. Solving hybrid control problems: Level sets and
behavioral programming. In Proc. American Contr. Conf., Chicago, Illinois
USA, June 2000.
[67] M. Buss, O. von Stryk, R. Bulirsch, and G. Schmidt. Towards hybrid optimal
control. AT - Automatisierungstechnik, 48:448459, 2000.
[68] Kerrigan E. C. and Maciejowski J. M. Soft constraints and exact penalty
functions in model predictive control. In Proc. UKACC International Conference (Control 2000), Cambridge, UK, September 2000.
[69] R. Cagienard, P. Grieder, E.C. Kerrigan, and M. Morari. Move blocking
strategies in receding horizon control. Journal of Process Control, 17(6):563
570, 2007.
[70] S. Di Cairano, A. Bemporad, and J. J
ulvez. Event-driven optimization-based
control of hybrid systems with integral continuous-time dynamics. Automatica, 45:12431251, 2009.
[71] S. Di Cairano, A. Bemporad, I. Kolmanovsky, and D. Hrovat. Model predictive control of magnetically actuated mass spring dampers for automotive
applications. Int. J. Control, 80(11):17011716, 2007.
[72] E.F. Camacho and C. Bordon. Model Predictive Control. Advanced Textbooks in Control and Signal Processing. Springer, London, 1999.
[73] M.K. Camlibel, W.P.M.H. Heemels, and J.M. Schumacher. On linear passive
complementarity systems. European Journal of Control, 8(3), 2002.
[74] P.J. Campo and M. Morari. Robust model predictive control. In Proc.
American Contr. Conf., volume 2, pages 10211026, 1987.
[75] P.J. Campo and M. Morari. Model predictive optimal averaging level control.
AIChE Journal, 35(4):579591, 1989.
[76] C.G. Cassandras, D.L. Pepyne, and Y.Wardi. Optimal control of a class of
hybrid systems. IEEE Transactions on Automatic Control, 46(3):3981415,
2001.
[77] T.M. Cavalier, P.M. Pardalos, and A.L. Soyster. Modeling and integer programming techniques applied to propositional calculus. Computers & Operations Research, 17(6):561570, 1990.
[78] S.N. Cernikov. Contraction of finite systems of linear inequalities (in Russian). Doklady Akademiia Nauk SSSR, 152(5):10751078, 1963. (English
translation in Soviet Mathematics - Doklady, Vol. 4, No. 5 (1963), pp.1520
1524).
[79] V. Chandru and J.N. Hooker. Optimization methods for logical inference.
Wiley-Interscience, 1999.
References
[80] H. Chen and F. Allg
ower. A quasi-infinite horizon nonlinear model predictive
control scheme with guaranteed stability. Automatica, 304(10):12051218,
1998.
[81] D. Chmielewski and V. Manousiouthakis. On constrained infinite-time linear quadratic optimal control. Systems & Control Letters, 29(3):121130,
November 1996.
[82] D. Christiansen. Electronics Engineers Handbook, 4th edition. IEEE Press/
McGraw Hill, Inc., 1997.
[83] F.J. Christophersen. Optimal Control and Analysis for Constrained Piecewise
Affine Systems. Dr. sc. ETH Zurich thesis, ETH Zurich, Zurich, Switzerland,
August 2006. Available from http://control.ee.ethz.ch.
[84] F.J. Christophersen and M. Morari. Further Results on Infinity Norms as
Lyapunov Functions for Linear Systems. IEEE Trans. on Automatic Control,
52(3):547553, March 2007.
[85] D.W. Clarke, C. Mohtadi, and P.S. Tuffs. Generalized Predictive Control
part i. the basic algorithm. Automatica, 23(2):137148, March 1987.
[86] C.R. Cuttler and B.C. Ramaker. Dynamic matrix controla computer control algorithm. In Proc. American Contr. Conf., volume WP5-B, San Francisco, USA, 1980.
[87] G.B. Dantzig, J. Folkman, and N. Shapiro. On the continuity of the minimum set of a continuous function. Journal of Mathematical Analysis and
Applications, 17:519548, 1967.
[88] B. De Schutter and B. De Moor. The extended linear complementarity
problem and the modeling and analysis of hybrid systems. In P. Antsaklis,
W. Kohn, M. Lemmon, A. Nerode, and S. Sastry, editors, Hybrid Systems V,
volume 1567 of Lecture Notes in Computer Science, pages 7085. Springer,
1999.
[89] B. De Schutter and T. van den Boom. Model predictive control for max-pluslinear discrete event systems. Automatica, 37(7):10491056, July 2001.
[90] R. DeCarlo, M. Branicky, S. Pettersson, and B. Lennartson. Perspectives
and results on the stability and stabilizability of hybrid systems. Proceedings
of the IEEE, 88(7):10691082, 2000.
[91] Olivier Devolder, Francois Glineur, and Yurii Nesterov. First-order methods
of smooth convex optimization with inexact oracle. Submitted to Mathematical Programming, 2011.
[92] A. Domahidi, A. Zgraggen, M.N. Zeilinger, M. Morari, and C.N. Jones. Efficient interior point methods for multistage problems arising in receding horizon control. In Conference on Decision and Control (CDC), pages 668674,
Maui, HI, USA, December 2012.
411
412
References
[93] J.A. De Don
a. Input Constrained Linear Control. PhD thesis, Control Group,
Department of Engineering, University of Cambridge, Cambridge, 2000.
[94] C.E.T. Dorea and J.C. Hennet. (a,b)-invariant polyhedral sets of linear
discrete-time systems. J. Optim. Theory Appl., 103(3):521542, 1999.
[95] V. Dua and E.N. Pistikopoulos. An algorithm for the solution of multiparametric mixed integer linear programming problems. Annals of Operations
Research, 99:123139, 2000. 10.1023/A:1019241000636.
[96] G. Ferrari-Trecate, F.A. Cuzzola, D. Mignone, and M. Morari. Analysis of
discrete-time piecewise affine and hybrid systems. Automatica, 38:21392146,
2002.
[97] G. Ferrari-Trecate, D. Mignone, and M. Morari. Moving horizon estimation
for hybrid systems. IEEE Transactions on Automatic Control, 47(10):1663
1676, 2002.
[98] G. Ferrari-Trecate, M. Muselli, D. Liberati, and M. Morari. A clustering
technique for the identification of piecewise affine systems. Automatica,
39(2):205217, February 2003.
[99] H.J. Ferreau and M. Diehl H.G. Bock. An online active set strategy to
overcome the limitations of explicit mpc. International Journal of Robust
and Nonlinear Control, 18:816830, 2008.
[100] A.V. Fiacco. Sensitivity analysis for nonlinear programming using penalty
methods. Mathematical Programming, 10(3):287311, 1976.
[101] A.V. Fiacco. Introduction to sensitivity and stability analysis in nonlinear
programming. Academic Press, London, U.K., 1983.
[102] R. Fletcher. Practical Methods of Optimization. Wiley-Interscience Publication, 1987. Second Edition.
[103] C.A. Floudas. Nonlinear and Mixed-Integer Optimization. Oxford University
Press, 1995.
[104] K. Fukuda. cdd/cdd+ Reference Manual. Institute for operations Research
ETH-Zentrum, ETH-Zentrum, CH-8092 Zurich, Switzerland, 0.61 (cdd) 0.75
(cdd+) edition, December 1997.
[105] K. Fukuda.
Polyhedral computation FAQ,
line document. Both html and ps versions
http://www.ifor.math.ethz.ch/staff/fukuda.
2000.
available
On
from
References
[108] T. Gal. Postoptimal Analyses, Parametric Programming, and Related Topics.
de Gruyter, Berlin, 2nd edition, 1995.
[109] T. Gal and H.J. Greenberg (Eds.). Advances in Sensitivity Analysis and
Parametric Programming, volume 6 of International Series in Operations
Research & Management Science. Kluwer Academic Publishers, 1997.
[110] T. Gal and J. Nedoma. Multiparametric linear programming. Management
Science, 18:406442, 1972.
[111] C.E. Garca and A.M. Morshedi. Quadratic programming solution of dynamic
matrix control (QDMC). Chem. Eng. Communications, 46:7387, 1986.
[112] S.J. Gartska and R.J.B. Wets. On decision rules in stochastic programming.
Mathematical Programming, 7:117143, 1974.
[113] S.I. Gass and T.L. Saaty. The computational algorithm for the parametric
objective function. Naval Research Logistics Quarterly, 2:3945, 1955.
[114] T. Geyer. Low Complexity Model Predictive Control in Power Electronics and
Power Systems. PhD thesis, Swiss Federal Institute of Technology (ETH),
ETH-Zentrum, CH-8092 Zurich, Switzerland, March 2005.
[115] T. Geyer, F.D. Torrisi, and M. Morari. Efficient Mode Enumeration of Compositional Hybrid Models. In A. Pnueli and O. Maler, editors, Hybrid Systems: Computation and Control, volume 2623 of Lecture Notes in Computer
Science, pages 216232. Springer-Verlag, 2003.
[116] T. Geyer, F.D. Torrisi, and M. Morari. Optimal complexity reduction of
polyhedral piecewise affine systems. Automatica, 44:17281740, July 2008.
[117] E.G. Gilbert and K.T. Tan. Linear systems with state and control constraints:
The theory and application of maximal output admissible sets. IEEE Transactions on Automatic Control, 36(9):10081020, September 1991.
[118] E.G. Gilbert and K.T. Tan. Linear systems with state and control constraints:
the theory and applications of maximal output admissible sets. IEEE Transactions on Automatic Control, 36(9):10081020, 1991.
[119] N. Giorgetti, A. Bemporad, H.E. Tseng, and D. Hrovat. Hybrid model predictive control application towards optimal semi-active suspension. Int. J.
Control, 79(5):521533, 2006.
[120] F. Glover. Improved linear integer programming formulations of nonlinear
integer problems. Management Science, 22(4):455460, 1975.
[121] R. Goebel, R.G. Sanfelice, and A.R. Teel. Hybrid dynamical systems. IEEE
Control Systems Magazine, 29(2):2893, 2009.
[122] K. Gokbayrak and C.G. Cassandras. A hierarchical decomposition method
for optimal control of hybrid systems. In Proc. 38th IEEE Conf. on Decision
and Control, pages 18161821, Phoenix, AZ, December 1999.
413
414
References
[123] Jacob E. Goodman and Joseph ORourke, editors. Handbook of Discrete and
Computational Geometry, chapter Voronoi Diagrams and Delaunay Triangulations, Steven Fortune, pages 513528. Chapman & Hall, 2nd edition,
2004.
[124] G.C. Goodwin and K. S. Sin. Adaptive Filtering, Prediction and Control.
Prentice-Hall, Englewood Cliffs, NJ, 1984.
[125] P.J. Goulart, E.C. Kerrigan, and J.M. Maciejowski. Optimization over state
feedback policies for robust control with constraints. Automatica, 42(4):523
533, 2006.
[126] P. Grieder. Efficient Computation of Feedback Controllers for Constrained
Systems. Dr. Sc. Techn. thesis, Swiss Federal Institute of Technology (ETH),
ETH-Zentrum, CH-8092 Zurich, Switzerland, 2004.
[127] P. Grieder, F. Borrelli, F.D. Torrisi, and M. Morari. Constrained infinite horizon linear quadratic regulator. Technical Report AUT02-66,
Automatic Control Laboratory, ETH Zurich, Switzerland, July 2002.
http://control.ee.ethz.ch.
[128] P. Grieder, F. Borrelli, F.D. Torrisi, and M. Morari. Computation of the
constrained infinite time linear quadratic regulator. Automatica, 40:701708,
April 2004.
[129] P. Grieder, M. Kvasnica, M. Baotic, and M. Morari. Low complexity control
of piecewise affine systems with stability guarantee. In Proc. of the American
Control Conference, pages 11961201, Boston, USA, June 2004.
[130] B. Gr
unbaum. Convex Polytopes. Springer-Verlag, 2nd edition, 2003.
[131] E. Guslitzer. Uncertainty-immunized solutions in linear programming. Masters thesis, Technion (Israel Institute of Technology), Haifa, Israel, 2002.
[132] P.O. Gutman. Online use of a linear programming controller. In G. Ferrate
and E.A. Puente, editors, Software for Computer Control 1982. Proceedings of
the Third IFAC/IFIP Symposium, pages 313318. Pergamon, Oxford, 1983.
[133] P.O. Gutman. A linear programming regulator applied to hydroelectric reservoir level control. Automatica, 22(5):533541, 1986.
[134] P.O. Gutman and M. Cwikel. Admissible sets and feedback control for
discrete-time linear dynamical systems with bounded control and states.
IEEE Transactions on Automatic Control, AC-31(4):373376, 1986.
[135] P.O. Gutman and M. Cwikel. An algorithm to find maximal state constraint
for discrete-time linear dynamical systems with bounded control and states.
IEEE Transactions on Automatic Control, AC-32(3):251254, 1987.
[136] A. Hassibi and S. Boyd. Quadratic stabilization and control of piecewiselinear systems. In Proc. American Contr. Conf., Philadelphia, Pennsylvania
USA, June 1998.
References
[137] J.P. Hayes. Introduction to Digital Logic Design. Addison-Wesley Publishing
Company, Inc., 1993.
[138] S. Hedlund and A. Rantzer. Optimal control of hybrid systems. In Proc.
38th IEEE Conf. on Decision and Control, pages 39723976, Phoenix, AZ,
December 1999.
[139] S. Hedlund and A. Rantzer. Convex dynamic programming for hybrid systems. IEEE Transactions on Automatic Control, 47(9):15361540, September
2002.
[140] W.P.H.M Heemels, B. de Schutter, and A. Bemporad. On the equivalence of
classes of hybrid dynamical models. In Proc. 40th IEEE Conf. on Decision
and Control, pages 364369, Orlando, Florida, 2001.
[141] W.P.M.H. Heemels. Linear complementarity systems: a study in hybrid dynamics. PhD thesis, Dept. of Electrical Engineering, Eindhoven University
of Technology, The Netherlands, 1999.
[142] W.P.M.H. Heemels, J.M. Schumacher, and S. Weiland. Linear complementarity systems. SIAM Journal on Applied Mathematics, 60(4):12341269,
2000.
[143] W.P.M.H. Heemels, B. De Schutter, and A. Bemporad. Equivalence of hybrid
dynamical models. Automatica, 37(7):10851091, July 2001.
[144] J.P. Hespanha, S. Bohacek, K. Obraczka, and J. Lee. Hybrid modeling of
TCP congestion control. In M.D. Di Benedetto and A. Sangiovanni Vincentelli, editors, Hybrid Systems: Computation and Control, volume 2034 of
Lecture Notes in Computer Science, pages 291304. Springer-Verlag, 2001.
[145] Jean-Baptiste Hiriart-Urruty and Claude Lemarechal. Fundamentals of Convex Analysis. Springer, 2001.
[146] W.M. Hogan. Point-to-set maps in mathematical programming. SIAM Review, 15(3):591603, July 1973.
[147] J.N. Hooker. Logic-Based Methods for Optimization: Combining Optimization and Constraint Satisfaction. Wiley, New York, 2000.
[148] T. Huerlimann. Reference Manual for the LPL Modeling Language, Version 4.42. Departement for Informatics, Universite de Fribourg, Switzerland,
http://www2-iiuf.unifr.ch/tcs/lpl/TonyHome.htm, 2001.
[149] A. Jadbabaie, Y. Jie, and J. Hauser. Stabilizing Receding Horizon Control of
nonlinear systems: a control Lyapunov function approach. In Proc. American
Contr. Conf., June 1999.
[150] M. Johannson and A. Rantzer. Computation of piece-wise quadratic Lyapunov functions for hybrid systems. IEEE Transactions on Automatic Control, 43(4):555559, 1998.
415
416
References
[151] T.A. Johansen, J. Kalkkuhl, J. L
udemann, and I. Petersen. Hybrid control
strategies in ABS. In Proc. American Contr. Conf., Arlington, Virginia, June
2001.
[152] K.H. Johansson, M. Egerstedt, J. Lygeros, and S. Sastry. On the regularization of Zeno hybrid automata. System & Control Letters, 38:141150, 1999.
[153] C. Jones, P. Grieder, and S. Rakovic. A Logarithmic-Time Solution to
the Point Location Problem for Closed-Form Linear MPC. In IFAC World
Congress, Prague, Czech Republic, July 2005.
[154] C.N. Jones, M. Baric, and M. Morari. Multiparametric Linear Programming
with Applications to Control. European Journal of Control, 13(2-3):152170,
March 2007.
[155] C.N. Jones, E.C. Kerrigan, and J.M. Maciejowski. Equality set projection: A new algorithm for the projection of polytopes in halfspace
representation.
Technical Report CUED Technical Report CUED/FINFENG/TR.463, Department of Engineering, Cambridge University, UK,
2004. http://www-control.eng.cam.ac.uk/ cnj22/.
[156] C.N. Jones and M. Morari. Polytopic approximation of explicit model predictive controllers. IEEE Transactions on Automatic Control, 55(11):25422553,
November 2010.
[157] P. Julian, M. Jordan, and A. Desages. Canonical piecewise-linear approximation of smooth functions. IEEE Trans. Circuits and Systems I: Fundamental Theory and Applications, 45(5):567571, May 1998.
[158] A.A. Julius and A.J. van der Schaft. The maximal controlled invariant set of
switched linear systems. In Proc. of the 41st IEEE Conference on Decision
and Control, Las Vegas, Nevada, USA, December 2002.
[159] A. Juloski, S. Weiland, and M.Heemels. A bayesian approach to identification
of hybrid systems. Proc. 43th IEEE Conf. on Decision and Control, 2004.
[160] David Kahaner, Cleve Moler, and Stephen Nash. Numerical Methods and
Software. Prentice Hall Series in Computational Mathematics, 1988.
[161] Narendra Krishna Karmarkar. A new polynomial-time algorithm for linear
programming. Combinatorica, 4:373395, 1984.
[162] S. Keerthi and E. Gilbert. Computation of minimum-time feedback control laws for discrete-time systems with state-control constraints. Automatic
Control, IEEE Transactions on, 32(5):432435, 1987.
[163] S.S. Keerthi and E.G. Gilbert. Optimal infinite-horizon feedback control laws
for a general class of constrained discrete-time systems: stability and movinghorizon approximations. J. Opt. Theory and Applications, 57:265293, 1988.
[164] S.S. Keerthi and K. Sridharan. Solution of parametrized linear inequalities
by Fourier elimination and its applications. J. Opt. Theory and Applications,
65(1):161169, 1990.
References
[165] E.C. Kerrigan. Robust Constraints Satisfaction: Invariant Sets and Predictive Control. PhD thesis, Department of Engineering, University of Cambridge, Cambridge, England, 2000.
[166] E.C. Kerrigan and J.M. Maciejowski. Designing model predictive controllers
with prioritised constraints and objectives. In Computer Aided Control System Design, 2002. Proceedings. 2002 IEEE International Symposium on,
pages 33 38, 2002.
[167] E.C. Kerrigan and J.M. Maciejowski. On robust optimization and the optimal
control of constrained linear systems with bounded state disturbances. In In
Proc. 2003 European Control Conference, Cambridge, UK, September 2003.
[168] H.K. Khalil. Nonlinear Systems. Prentice Hall, 2nd edition, 1996.
[169] H. Kiendl, J. Adamy, and P. Stelzner. Vector norms as Lyapunov functions
for linear systems. IEEE Transactions on Automatic Control, 37(6):839842,
June 1992.
[170] D. Klatte and G. Thiere. Error bounds for solutions of linear equations and
inequalities. ZOR - Mathematical Methods of Operations Research, 41:191
214, 1995.
[171] I. Kolmanovsky and E.G. Gilbert. Theory and computation of disturbance
invariant sets for discrete-time linear systems. Mathematical Problems in
Egineering, 4:317367, 1998.
[172] M.V. Kothare, V. Balakrishnan, and M. Morari. Robust constrained model
predictive control using linear matrix inequalities. Automatica, 32(10):1361
1379, 1996.
[173] B. Kouvaritakis, J.A. Rossiter, and J. Schuurmans. Efficient robust predictive
control. IEEE Transactions on Automatic Control, 45(8):15451549, 2000.
[174] M. Kvasnica, P. Grieder, and M. Baotic. Multi-Parametric Toolbox (MPT),
2004.
[175] G. Lafferriere, G.J. Pappas, and S. Sastry. Reachability analysis of hybrid
systems using bisimulations. In Proc. 37th IEEE Conf. on Decision and
Control, pages 16231628, Tampa, Florida, USA, 1998.
[176] J.P. LaSalle. Stability theory for difference equations. In Jack Hale, editor,
Studies in Ordinary Differential Equations, volume 14 of MAA Studies in
Mathematics, pages 131. 1977.
[177] J.P. LaSalle. The Stability and Control of Discrete Processes, volume 62 of
Applied Mathematical Sciences. Springer-Verlag, New York, 1986.
[178] M. Lazar and W.P.M.H. Heemels.
Predictive control of hybrid systems: Input-to-state stability results for sub-optimal solutions. Automatica,
45(1):180185, 2009.
417
418
References
[179] M. Lazar, W.P.M.H. Heemels, and A.R. Teel. Lyapunov functions, stability
and input-to-state stability subtleties for discrete-time discontinuous systems.
IEEE Transactions on Automatic Control, 54(10):24212425, 2009.
[180] M. Lazar, W.P.M.H. Heemels, S. Weiland, and A. Bemporad. Stabilizing
model predictive control of hybrid systems. IEEE Transactions on Automatic
Control, 51(11):18131818, 2006.
[181] M. Lazar, D. M. de la Pe
na, W.P.M.H. Heemels, and T. Alamo. On inputto-state stability of min-max nonlinear model predictive control. Systems &
Control Letters, 57:3948, 2008.
[182] J. H. Lee and Z. Yu. Worst-case formulations of model predictive control for
systems with bounded parameters. Automatica, 33(5):763781, 1997.
[183] F.L. Lewis and V.L. Syrmos. Optimal Control. John Wiley & Sons, Inc.,
New York, 1995.
[184] D. Liberzon. Switching in Systems and Control. Systems and Control: Foundations and Application. Birkhauser, Boston, MA, June 2003.
[185] B. Lincoln and A. Rantzer. Optimizing linear system switching. In Proc.
40th IEEE Conf. on Decision and Control, pages 20632068, Orlando, FL,
USA, 2001.
[186] Y.-C. Liu and C. B. Brosilow. Simulation of large scale dynamic systems
I. modular integration methods. Computers & Chemical Engineering,
11(3):241253, 1987.
[187] J. L
ofberg. Minimax approaches to robust model predictive control. PhD
thesis, Link
oping University, Sweden, April 2003.
[188] J. Lygeros, D.N. Godbole, and S. Sastry. A game theoretic approach to
hybrid system design. In R. Alur and T. Henzinger, editors, Hybrid Systems
III, volume 1066 of Lecture Notes in Computer Science, pages 112. Springer
Verlag, 1996.
[189] J. Lygeros, C. Tomlin, and S. Sastry. Controllers for reachability specifications for hybrid systems. Automatica, 35(3):349370, 1999.
[190] J.M. Maciejowski. Predictive Control with Constraints. Prentice Hall, 2002.
[191] U. Maeder, F. Borrelli, and M. Morari. Linear offset-free Model Predictive
Control. Automatica, 45(10):2214 2222, 2009.
[192] S. Mahapatra. Stability of Hybrid Haptic Systems. PhD thesis, University of
Illinois, Chicago, Illinois, 2003.
[193] O.L. Mangasarian and J.B. Rosen. Inequalities for stochastic nonlinear programming problems. Operations Research, 12(1):143154, 1964.
[194] Jacob Mattingley and Stephen Boyd. CVXGEN: A code generator for embedded convex optimization. Optimization and Engineering, 13:127, 2012.
References
[195] D.Q. Mayne. Constrained Optimal Control. European Control Conference,
Seminario de Vilar, Porto, Portugal, Plenary Lecture, September 2001.
[196] D.Q. Mayne. Control of constrained dynamic systems. European Jornal of
Control, 7:8799, 2001.
[197] D.Q. Mayne, J.B. Rawlings, C.V. Rao, and P.O.M. Scokaert. Constrained
model predictive control: Stability and optimality. Automatica, 36:789814,
2000.
[198] D.Q. Mayne, J.B. Rawlings, C.V. Rao, and P.O.M. Scokaert. Constrained
model predictive control: Stability and optimality. Automatica, 36(6):789
814, June 2000.
[199] T.A. Meadowcroft, G. Stephanopoulos, and C. Brosilow. The Modular Multivariable Controller: 1: Steady-state properties. AIChE Journal, 38(8):1254
1278, 1992.
[200] S. Mehrotra and R.D.C. Monteiro. Parametric and range analysis for interior
point methods. Technical report, Dept. of Systems and Industrial Engineering, University of Arizona, Tucson, USA, 1992.
[201] Sanjay Mehrotra. On the implementation of a primal-dual interior point
method. SIAM Journal on Optimization, 2(4):575601, November 1992.
[202] E. Mendelson. Introduction to mathematical logic. Van Nostrand, 1964.
[203] J.A. Mendez, B. Kouvaritakis, and J.A. Rossiter. State space approach to
interpolation in MPC. Int. J. Robust Nonlinear Control, 10(1):2738, January
2000.
[204] D. Mignone. Control and Estimation of Hybrid Systems with Mathematical
Optimization. PhD thesis, Automatic Control Labotatory - ETH, Zurich,
2002.
[205] D. Mignone, A. Bemporad, and M. Morari. A framework for control, fault
detection, state estimation and verification of hybrid systems. Proc. American
Contr. Conf., pages 134138, June 1999.
[206] R. Milman and E.J. Davison. A fast mpc algorithm using nonfeasible active
set methods. Journal of Optimization Theory and Applications, 139(3):591
616, 2008.
[207] G. Mitra, C. Lucas, and S. Moody. Tool for reformulating logical forms into
zero-one mixed integer programs. European Journal of Operational Research,
71:262276, 1994.
[208] R. M
obus, M. Baotic, and M. Morari. Multi-objective adaptive cruise control. In O. Maler and A. Pnueli, editors, Hybrid Systems: Computation and
Control, number 2623 in Lecture Notes in Computer Science, pages 359374.
Springer-Verlag, 2003.
419
420
References
[209] M. Morari and G. Stephanopoulos. Minimizing unobservability in inferential
control schemes. Int. J. Control, 31:367377, 1980.
[210] M. Morari and G. Stephanopoulos. Studies in the synthesis of control structures for chemical processes; Part III: Optimal selection of secondary measurements within the framework of state estimation in the presence of persistent unknown disturbances. AIChE J., 26:247260, 1980.
[211] Katta G. Murty. Linear Programming. Wiley-Interscience Publication, 1983.
First Edition.
[212] K.R. Muske and T.A. Badgwell. Disturbance modeling for offset-free linear
model predictive control. Journal of Process Control, 12:617632, 2002.
[213] G.L. Nemhauser and L.A. Wolsey. Integer and Combinatorial Optimization.
Wiley, 1988.
[214] Yurii Nesterov. A method for solving a convex programming problem with
convergence rate 1/k 2 . Soviet Math. Dokl., 27(2):372376, 1983.
[215] Yurii Nesterov. Introductory Lectures on Convex Optimization. A Basic
Course. Springer, 2004.
[216] Yurii Nesterov and Arkadi Nemirovski. Interior-Point Polynomial Algorithms
in Convex Programming. Society for Industrial Mathematics, 1994.
[217] G. Pannocchia. Robust disturbance modeling for model predictive control
with application to multivariable ill-conditioned processes. J. Process Control, 13(8):693701, 2003.
[218] G. Pannocchia and J.B. Rawlings. Disturbance models for offset-free model
predictive control. AIChE Journal, 49(2):426437, 2003.
[219] S. Paoletti. Identification of Piecewise Affine Models. PhD thesis, Dept.
Information Engineering, University of Siena, Italy, 2004.
[220] T. Park and P.I. Barton. Implicit model checking of logic-based control
systems. AIChE Journal, 43(9):22462260, 1997.
[221] P.A. Parrilo. Structured semidefinite programs and semialgebraic geometry
methods in robustness and optimization. Ph.D. thesis, California Institute of
Technology, Pasadena, CA, USA, 2000.
[222] Panagiotis Patrinos and Haralambos Sarimveis. Convex parametric piecewise
quadratic optimization: Theory and Algorithms. Automatica, 47(8):1770
1777, 2011.
[223] D. M. De La Pena, A. Bemporad, and C. Filippi. Robust explicit MPC based
on approximate multiparametric convex programming. Automatic Control,
IEEE Transactions on, 51(8):13991403, 2006.
References
[224] S. Pettersson and B. Lennartson. Stability and robustness for hybrid systems.
In Proc. 35th IEEE Conf. on Decision and Control, pages 12021207, Kobe,
Japan, 1996.
[225] S. Pettersson and B. Lennartson. Exponential stability of hybrid systems
using piecewise quadratic lyapunov functions resulting in LMIs. In Proc.
IFAC World Congress, pages 103108, Beijing, China, July 1999.
[226] B. Piccoli. Necessary conditions for hybrid optimization. In Proc. 38th IEEE
Conf. on Decision and Control, Phoenix, Arizona USA, December 1999.
[227] A. Pogromski, M. Jirstrand, and P. Spangeus. On stability and passivity of a
class of hybrid systems. In Proc. 37th IEEE Conf. on Decision and Control,
pages 37053710, Tampa, Florida, USA, 1998.
[228] A. Polanski. On infinity norms as Lyapunov functions for linear systems.
IEEE Transactions on Automatic Control, 40(7):12701274, July 1995.
[229] S. Prajna and A. Papachristodoulou. Analysis of switched and hybrid systems beyond piecewise quadratic methods. In Proceedings of the American
Control Conference, volume 4, pages 27792784, 2003.
[230] A.I. Propoi. Use of linear programming methods for synthesizing sampled 21 data automatic systems. Automation and Remote Control, 24(7):837844,
1963.
[231] S.J. Qin and T.A. Badgwell. An overview of industrial model predictive
control technology. In Chemical Process Control - V, volume 93, no. 316,
pages 232256. AIChE Symposium Series - American Institute of Chemical
Engineers, 1997.
[232] S.J. Qin and T.A. Badgwell. A survey of industrial model predictive control
technology. Control Engineering Practice, 11:733764, 2003.
[233] S.V. Rakovic, P. Grieder, M. Kvasnica, D.Q. Mayne, and M. Morari. Computation of invariant sets for piecewise affine discrete time systems subject to
bounded disturbances. In IEEE Conference on Decision and Control, pages
14181423, December 2004.
[234] S.V. Rakovic, E.C. Kerrigan, and D.Q. Mayne. Reachability computations
for constrained discrete-time systems with state- and input-dependent disturbances. In Proc. of the Conf. on Decision and Control, Maui, Hawaii,
USA, pages 39053910, December 2003.
[235] R. Raman and I.E. Grossmann. Relation between MILP modeling and logical
inference for chemical process synthesis. Computers Chem. Engng., 15(2):73
84, 1991.
[236] A. Rantzer and M. Johansson. Piecewise linear quadratic optimal control.
IEEE Transactions on Automatic Control, 45(4):629637, April 2000.
421
422
References
[237] Christopher V. Rao, Stephen J. Wright, and James B. Rawlings. Application
of Interior-Point methods to model predictive control. Journal of Optimization Theory and Applications, 99(3):723757, December 1998.
[238] James B. Rawlings and David Q. Mayne. Model Predictive Control: Theory
and Design. Nob Hill Publishing, 2009.
[239] J. Richalet, A. Rault, J.L. Testud, and J. Papon. Algorithmic control of
industrial processes. In Fourth IFAC symposium on identixcation and system
parameter estimation, volume WP5-B, pages 11191167, 1976.
[240] J. Richalet, A. Rault, J.L. Testud, and J. Papon. Model predictive heuristic
control-application to industrial processes. Automatica, 14:413428, 1978.
[241] P. Riedinger, F.Kratz, C. Iung, and C. Zanne. Linear quadratic optimization
for hybrid systems. In Proc. 38th IEEE Conf. on Decision and Control,
Phoenix, Arizona USA, December 1999.
[242] S.M. Robinson. Some continuity properties of polyhedral multifunctions.
Mathematical Proggraming Study, 14:206214, 1981.
[243] S.M. Robinson and R.H. Day. A sufficient condition for continuity of optimal
sets in mathematical programming. Journal of Mathematical Analysis and
Applications, 45:506511, 1974.
[244] Tyrrell R. Rockafellar. Convex analysis. Princeton University Press, 1997.
[245] J. Roll, A. Bemporad, and L. Ljung. Identification of piecewise affine systems
via mixed-integer programming. Automatica, 40(1):3750, 2004.
[246] L.O. Santos, P.A.F.N.A. Afonso, J.A.A.M. Castro, N.M.C. Oliveira, and L.T.
Biegler. On-line implementation of nonlinear mpc: an experimental case
study. Control Engineering Practice, 9(8):847 857, 2001.
[247] M. Schechter. Polyhedral functions and multiparametric linear programming.
J. Opt. Theory and Applications, 53(2):269280, May 1987.
[248] C. Scherer and S. Weiland. Linear Matrix Inequalities in Control. Technical report, Delft Center for Systems and Control, Delft University
of Technology, Delft, The Netherlands, January 2005. Available from
http://www.dcsc.tudelft.nl/~cscherer/2416/lmi05.pdf.
[249] P.O.M. Scokaert and D.Q. Mayne. Min-max feedback model predictive control for constrained linear systems. IEEE Transactions on Automatic Control,
43(8):11361142, 1998.
[250] P.O.M. Scokaert and J.B. Rawlings. Constrained linear quadratic regulation.
IEEE Transactions on Automatic Control, 43(8):11631169, 1998.
[251] C. Seatzu, D. Corona, A. Giua, and A. Bemporad. Optimal control of
continuous-time switched affine systems. IEEE Transactions on Automatic
Control, 51(5):726741, 2006.
References
[252] C. Seatzu, D. Corona, A. Giua, and A. Bemporad. Optimal control of
continuous-time switched affine systems. Automatic Control, IEEE Transactions on, 51(5):726 741, may 2006.
[253] M.M. Seron, J.A. DeDon
a, and G.C. Goodwin. Global analytical model predictive control with input constraints. In Proc. 39th IEEE Conf. on Decision
and Control, pages 154159, 2000.
[254] J. Serra. Image Analysis and Mathematical Morphology, Vol II: Theoretical
advances. Academic Press, 1988.
[255] M.S. Shaikh and P.E. Caines. On the optimal control of hybrid systems:
optimization of trajectories, switching times and location schedules. In 6th
Int. Workshop on Hybrid Systems: Computation and Control, Prague, The
Czech Republic, 2003.
[256] B.I. Silva, O. Stursberg, B.H. Krogh, and S. Engell. An assessment of the
current status of algorithmic approaches to the verification of hybrid systems. In Proc. 40th IEEE Conf. on Decision and Control, pages 28672874,
Orlando, Florida, December 2001.
[257] E.D. Sontag. Nonlinear regulation: The piecewise linear approach. IEEE
Transactions on Automatic Control, 26(2):346358, April 1981.
[258] E.D. Sontag. Interconnected automata and linear systems: A theoretical
framework in discrete-time. In R. Alur, T.A. Henzinger, and E.D. Sontag,
editors, Hybrid Systems III - Verification and Control, number 1066 in Lecture Notes in Computer Science, pages 436448. Springer-Verlag, 1996.
[259] J. Spjotvold, E.C. Kerrigan, C.N. Jones, P. Tondel, and T.A Johansen. On
the facet-to-facet property of solutions to convex parametric quadratic programs. Automatica, 42(12):22092214, December 2006.
[260] R. Suard, J. L
ofberg, P. Grieder, M. Kvasnica, and M. Morari. Efficient
computation of controller partitions in multi-parametric programming. In
Proc. 43th IEEE Conf. on Decision and Control, pages 36433648, Bahamas,
December 2004.
[261] S. Summers, C.N. Jones, J. Lygeros, and M. Morari. A multiresolution approximation method for fast explicit model predictive control. IEEE Transactions on Automatic Control, October 2011.
[262] H.J. Sussmann. A maximum principle for hybrid optimal control problems.
In Proc. 38th IEEE Conf. on Decision and Control, Phoenix, Arizona USA,
December 1999.
[263] M. Sznaier and M.J. Damborg. Suboptimal control of linear systems with
state and control inequality constraints. In Proc. 26th IEEE Conf. on Decision and Control, volume 1, pages 761762, 1987.
[264] C.J. Tomlin, J. Lygeros, and S.S. Sastry. A game theoretic approach to
controller design for hybrid systems. Proceeding of IEEE, 88, July 2000.
423
424
References
[265] P. Tndel, T.A. Johansen, and A. Bemporad. An algorithm for multiparametric quadratic programming and explicit MPC solutions. In Proc.
40th IEEE Conf. on Decision and Control, December 2001.
[266] P. Tndel, T.A. Johansen, and A. Bemporad. Evaluation of piecewise affine
control via binary search tree. Automatica, 39(5):945950, 2003.
[267] F. Torrisi, A. Bemporad, G. Bertini, P. Hertach, D. Jost, and D. Mignone.
Hysdel 2.0.5 - User Manual. Technical Report AUT02-28, Automatic Control
Laboratory, ETH Zurich, 2002.
[268] F.D. Torrisi and A. Bemporad. HYSDEL A tool for generating computational hybrid models. IEEE Trans. Contr. Systems Technology, 12(2):235
249, March 2004.
[269] M.L. Tyler and M. Morari. Propositional logic in control and monitoring
problems. Automatica, 35(4):565582, 1999.
[270] V.I. Utkin. Variable structure systems with sliding modes. IEEE Transactions on Automatic Control, 22(2):212222, April 1977.
[271] A.J. van der Schaft and J.M. Schumacher. Complementarity modelling of
hybrid systems. IEEE Transactions on Automatic Control, 43:483490, 1998.
[272] D.H. vanHessem and O.H. Bosgra. A conic reformulation of model predictive
control including bounded and stochastic disturbances under state and input
constraint. In Proc. 41th IEEE Conf. on Decision and Control, pages 4643
4648, Las Vegas, Nevada, USA, 2002.
[273] R. Vidal, S. Soatto, Y. Ma, and S. Sastry. An algebraic geometric approach
to the identification of a class of linear hybrid systems. In Proc. 42th IEEE
Conf. on Decision and Control, pages 167172, Maui, Hawaii, 2003.
[274] D.W. Walkup and R.J.-B. Wets. A lipschitzian characterizations of convex
polyhedra. Proceeding of the American Mathematical Society, 20:167173,
1969.
[275] Y. Wang and S. Boyd. Fast model predictive control using online optimization. IEEE Transactions on Control Systems Technology, 18(2):267 278,
March 2010.
[276] J. Warren, S. Schaefer, A. Hirani, and M. Desbrun. Barycentric coordinates
for convex sets. Advances in Computational Mathematics, 27(3):319338,
2007.
[277] H.P. Williams. Logical problems and integer programming. Bulletin of the
Institute of Mathematics and Its Applications, 13:1820, 1977.
[278] H.P. Williams. Model Building in Mathematical Programming. John Wiley
& Sons, Third Edition, 1993.
[279] H. Witsenhausen. A class of hybrid-state continuous-time dynamic systems.
IEEE Transactions on Automatic Control, 11(2):161167, 1966.
References
425
[280] H.S. Witsenhausen. A min-max control problem for sampled linear systems.
IEEE Transactions on Automatic Control, 13(1):521, 1968.
[281] Stephen J. Wright. Primal-Dual Interior Point Methods. SIAM, 1997.
[282] X. Xu and P.J. Antsaklis. Results and perspectives on computational methods for optimal control of switched systems. In O. Maler and A. Pnueli,
editors, Hybrid Systems: Computation and Control, number 2623 in Lecture
Notes in Computer Science, pages 540555. Springer-Verlag, 2003.
[283] X. Xu and P.J. Antsaklis. Optimal control of switched systems based on
parameterization of the switching instants. IEEE Transactions on Automatic
Control, 49(1):216, 2004.
[284] L.A. Zadeh and L.H. Whalen. On optimal control and linear programming.
IRE Trans. Automatic Control, 7:4546, 1962.
[285] E. Zafiriou and M. Morari. A general controller synthesis methodology based
on the IMC structure and the H2 -, H - and -optimal control theories.
Computers & Chemical Engineering, 12(7):757765, 1988.
[286] M. Zefran, F. Bullo, and M. Stein. A notion of passivity for hybrid systems.
In Proc. 40th IEEE Conf. on Decision and Control, pages 768773, Orlando,
FL, 2001.
[287] G.M. Ziegler. Lectures on Polytopes.
Springer-Verlag, 1994.