COMPARISON BETWEEN QR AND CHOLESKY METHOD

IN DYNAMICS SYSTEM

THRESEA DALAN

A report Submitted in partial fulfillment of the

requirements for the award of the degree of
Bachelor of Engineering (Electrical - Control and Instrumentation)

Faculty of Electrical Engineering

Universiti Teknologi Malaysia

MAY 2011

iii

To my beloved family especially mother. The love you gave me has been
tremendous and invaluable in my life. I would not have succeeded without you as
my other half.

iv

ACNOWLEGMENT

Praise to God for His guidance and blessing, this project was finally
completed.First and foremost, I would like to express my heartily gratitude to my
supervisor, Dr. Norhaliza Binti Hj. Wahab for the guidance and enthusiasm given
throughout the progress of this project.
My appreciation also goes to my family who has been so tolerant and
supports me all these years. Thanks for their prayers, encouragement, love and
emotional supports that they had given to me. Without them, I feel very weak to face
every constraint in my life.
I would also like thanks for all of lectures and tutors that had given me so
much knowledge. May God give his blessing for all of their kindness. Nevertheless,
my great appreciation dedicated to all my best friends and SEI members. To friends
whom I have spent the year with - thanks for all the ideas and being supportive. Also
to those whom involve directly or indirectly with this project. There isno such
meaningful word than. Thank You So Much.

ABSTRACT

In the complex system, often computation time is one of the main problems
in direct adaptive predictive controller. This is basically related to the identification
part to retrieve the model online. This thesis will tackle the problem of computation
time in the identification part using subspace identification algorithm (i.e N4SID).
The main idea is to solve for linear predictor equation, Yf LwWp LuU f using
least square solution. Both RQ and Cholesky methods will be used

Yf

to solve

.These two algorithms basically used to reduces the effects noise in measured data
and use to extrapolating future plat input-output behavior from past input-output
data. Both methods of algorithms are compared in terms of computation time.

vi

ABSTRAK

Dalam sistem yang kompleks, sering waktu pengkomputeran yang

merupakan salah satu masalah utama dalam pengawal aditif ramalan langsung. Hal
ini pada dasarnya berkaitan dengan bahagian pengenalan untuk memuatkan online
model. Tesis ini akan mengatasi masalah waktu pengkomputeran di bahagian
pengenalan menggunakan algoritma Subspace (N4SID). Idea utamanya adalah untuk
menyelesaikan persamaan linier prediktor, Yf LwWp LuU f menggunakan
penyelesaian kuadrat terkecil. Kedua-dua RQ dan kaedah Cholesky akan digunakan
untuk menyelesaikan

Yf

. Kedua-dua algoritma pada dasarnya digunakan untuk

mengurangkan kesan noise dalam data diukur dan digunakan untuk ekstrapolasi plat
perilaku input-output masa depan dari data input-output terkini. Kedua-dua kaedah
algoritma

tersebut

berbanding

dalam

hal

masa

pengkomputeran.

vii

TABLE OF CONTENTS

CHAPTER

TITLE

PAGE

DECLARATION

ii

DEDICATION

iii

ACKNOWLEDGEMENT

iv

ABSTRACT

ABSTRAK

vi

TABLE OF CONTENTS

vii

LIST OF TABLES

LIST OF FIGURES

xi

LIST OF APPENDICES

xii

INTRODUCTION

1.1

Background

1.2

Objectives

1.3

Scope

1.4

Summary

LITERATURE REVIEW

2.1 Introduction

2.2 Least Square Subspace Predictor

2.3 QR method

2.4 Cholesky method

2.5 Fast updating and down dating of Lw and Lu

10

2.6 Comparison between QR and Cholesky

11

viii
3

METHODOLOGY

13

3.1 Introduction

13

3.2 Create the MIMO Aircraft System

14

3.3 Create The Hankel matrices

15

3.4 Create QR algorithm

16

3.5 Create Cholesky algorithm

17

3.6 Updating and Down-dating data using QR method

17

3.7 Updating and down-date data using Cholesky method

18

3.8 Measure computation time for QR and Cholesky

method
4

18

RESULTS AND DISCUSSIONS

19

4.1 Discussions

19

4.2 Results

21

CONCLUSIONS AND FUTURE WORK

28

5.1 Conclusions

28

5.2 Future Works

29

REFERENCES

30

Appendices A-D

31-39

ix

LIST OF TABLES

TABLE NO.
1.1

TITLE
Comparison

between

PAGE
QR

and

12

Time taken by QR method to

21

Cholesky method
4.1

determine Lu and Lw
4.2

Time taken by QR method to

22

updating data
4.3

Time taken by QR to down dating

23

data
4.4

Time taken by Cholesky method to

24

determine Lu and Lw
4.5

Time taken by Cholesky method to

25

updating data
4.6

Time taken by Choclesky method to

down dating data

26

LIST OF FIGURE

FIGURE NO.

TITLE

PAGE

1.1

Flow Chart

4.1

The performance of Aircraft system

27

xi

LIST OF APPENDICES

APPENDIX

TITLE

PAGE

GANTTS CHART FOR FINAL

YEAR PROJECT 1

31

GANTTS CHART FOR FINAL

YEAR PROJECT 1

32

MATLAB
PROGRAM
CHOLESKY METHOD

FOR

33

MATLAB PROGRAM FOR QR

METHOD

36

CHAPTER 1

INTRODUCTION

1.1 Background

Nowadays, most of dynamics system use online tracking to monitor the

performance of the system. Online method make the engineer easier to monitor and
analysis the performance of the system especially dynamics system. The traditional
method requires more time to get the data from the plant and then do the analysis. In
traditional method the engineer need to calibrate every instrument in plant if
something wrong goes into the plant, so it take a long time. In online system the
engineer can detect the error of the system in second by monitoring the system and
control the system in control room. So in online system the engineer no needs to go
to the plant to do inspection.
However, online systems still have the problem in computation time in direct
adaptive predictive controller. So to tackle the problem, the subspace identification
algorithm was introduced in this paper. The subspace system identification methods
have recently gained much popularity for identification of linear time-invariant (LTI)
systems. The technique is typically employed in two step process: firstly, the best
least square subspace predictor is derived from available experimental data;
secondly, the predictor is used to derive a state space model of the dynamics system.
[1]

2
A fast algorithm is very important to updating and computing a subspace
predictor data when new experiment data are available. For example in-flight modal
analysis of airplanes, a fast analysis of the test data is very important because test
flights are expensive. A batch of data samples is collected a used to estimate the
model parameters. While processing the data, the pilot has waited for the next
instructions. Online and in-flight exploration of the data would allow more direct
exploration of the flight domain, with improved confidence and reduced costs. [2] So
the fast algorithms used in this paper are RQ and Cholesky methods. Besides the
faster processing of the data, the algorithm also reduces the constraints on the
amount of data that can be processed so that more sensors can be used in dynamics
system. For example, is an airplane. In this project, the data from airplane will be
used.

1.2 Objective of Project

The main objective of this project is to tackle the problem of computation

time in identification part using subspace algorithms. Second is to solve for linear
predictor equation Yf LwWp LuU f using RQ and Cholesky method. The
comparison between RQ and Cholesky in term mrthods of time is done. The last
objective is to extrapolate future plant input-output data behavior from past inputoutput data.

1.3 Scope of Project

In order to achieve objective of the project, there are several scopes have
been outlined. The scope of this project includes implement matrix algorithm for
subspace predictor, Yf LwWp LuU f and interpret the data available to it, study on
RQ and Choclesky algorithm methods, to build / design matlab programming and
last but not least is compute the data using matlab Programming.

1.4 Summary of Work

Implementation and works of the project are summarized into the flow chart
as shown in figure 1.1. Gantt chart as shown in figure 1.2 and 1.3 shows the detail of
the work of the project that had been implemented in the first semester and the
planning of work for the second semester.

Figure 1.1 Flow Chart

CHAPTER 2

THEORY AND LITERATURE REVIEW

2.1 Introduction

This chapter includes the least square predictor, method use to solve least
square predictor in the adaptive control and use in-flight modal analysis of airplane.
So in this chapter the research about QR and Cholesky has done by read the journal
and book. Besides that, this chapter also discusses on floating point and the memory
require.

2.2 Least Square Subspace Predictor

The summary of the least square subspace predictor is in [1]. The

development of the predictor begins with experimental input- output data. Consider
input- output data of length n from plan m inputs uk Rm and l outputs yk Rl . The
designer then chooses a prediction horizon, i which should be chosen to be larger
than the expected order of the plant (if the plant is LTI). Typically i is chosen to be 2
and 3 times larger than the expected plant order. [1] The data set is then broken into j
prediction problems, where j= n -2i+ 1. Usually there is a relatively large amount of
data, so that j>> i. The goal is to find a single predictor that simultaneously optimizes
(in the least squares sense) the j prediction problems.

The j prediction problem can be formalized as follows. Define the block Hankel
matrices from the data as

Up

u0 u1

u1 u2

ui 1 ui

u j 1

uj

ui j 2

(2.1)

The matrix in equation (2.1) represent past input.

ui ui 1
u
i 1 ui 2
Uf

u2i 1 u2i

ui j 1
ui j

u2i j 2

(2.2)

The matrix in equation (2.2) represents future input.

Yp

y0
y
1

y2i 1

y1
y2

y2i

y j 1

y j

y2i j 2

(2.3)

The matrix in equation (2.3) represents past output.

yi
y
i 1
Y f

y2i 1

yi 1
yi 2

y2 i

yi j 1
yi j

y2i j 2

(2.4)

7
The matrix in equation (2.4) represents future output.

The predictor is determined by choosing Lw and Lu to simultaneously optimize

the prediction indicated by the matrix in equation (2.1), (2.2), (2.3) and (2.4).
The subscripts p and f essentially represent past and future time. Define wp ,
which contains all the past

U p
Wp

Yp

(25)

To obtaining the best linear squares predictor of L , given U and Y can be written as
a Fresenius norm minimization.
min
Lw , Lu

Y f Lw

W p
Lu

U f

(2.6)

The least square represent below:

Yf LwWp LuU f
The

Lu

and

(2.7)

Lw use to form an estimate of the future outputs

y k

y k i 1

uk i

uk 1
Lw
Lu
y
k i

yk 1

uk

uk i 1

(2.8)

8
The equation (2.8) will be used to extrapolate future plant input-output behavior from
past input-output data.

2.3 QR method

The QR factorization of an m-by-n matrix A is given:A= QR

(2.9)

The matrix A have full column rank then the first n column of Q form an
orthogonal basis for range(A).The RQ factorization is one way to compute an
orthogonal basis for set of vectors. They have five method in RQ factorization;
Householder, block householder, Givens and fast givens transformations.
So in this project the fast givens transformation will be to calculate the least
square predictor. The RQ decomposition forms the basis of a computationally
efficient and numerically reliable method to finding Lw and Lu. [1]
QR decomposition.

W p
R11

T
T
U f R Q R21
Yf
R31

0
R22
R32

0
0
R33

QT

(2.10)

Then

Lw

Lu R31 R32

R11
R
21

0
R22

(2.11)

9
The indicated pseudo- inverse is usually calculated using the singular value
decomposition (SVD), so that

R11
R
21

Lw

Lu R31

0
T

U
V

svd
svd
R22
11

0
R32 Vq

1
2

(2.12)

0 T
U

q1

(2.13)

The QR decomposition has computational complexity o(i 2 j ) and the SVD

algorithm has complexity o(i3), which produces an overall computational complexity
o(i 2 j i 3 ) . The storage requirement for the RQ/SVD algorithm is large.

2.4 Cholesky Method

In order to enable an adaptive algorithm, rapid computation of Lw and Lu is
required. The following method is based on the cholesky factorization.

Let

W p

A U f
yf

AAT RT QT QR

(2.14)

(2.15)

10

Where R= chol(AAT) is the cholesky factorization of AAT. Since the

dimensions of AAT are much smaller than A when j>>i, the Cholesky factorization of
AAT can take significantly fewer computations to perform than the RQ
decomposition of A. The cholesky factorization is in fact O (i3) [1]. However, one
must also account for the computation of AAT.
The brute force method of computing AAT is O(i2j). Fortunately, the Hankel
structure of A can be exploited to drastically reduce this number.[1]
Let
U p
U
U f

Yp
Y
Y f

U p

U f U
B
Y
Y
p
Y f

(2.16)

By inspection, A is a permutation of B, thus AAT is a permutation of BBT, so

computing

UU T
BBT T
YU

UY T

YY T

(2.17)

is equivalent to computing AAT. Since BBT is symmetric, it is sufficient to compute

only the bloxk upper triangle of UUT and YYT, and to compute only UYT.

2.5

Fast updating and down dating of Lw and Lu

According to Woodley et al [1]. To update and downdate the data (sliding

window), the oldest experimental data are removed from R, and new data are added.
This sliding window procedure requires the storage of experimental data over the
whole window. In this case, the matrix R plays the role of maintaining a summary of

11
all past information in order to update the predictor as new experimental data
installed. When the data are used to update R, then the Lw and Lu are computed
from R using SVD.

2.6

Comparison on RQ and Cholesky

Table 1.1 Comparison between QR and Cholesky Method

Floating point

RQ

Cholesky

3.2 x 108

3.4 x 105

1.08x106

1.4x104

operation
Memory(Bytes)

From B.R Woodley et.al [2], the result show that RQ method require more
floating point operation and memory byte in do the analysis. The Cholesky method is
the reduced storage requirements. The recursive method of computing BBT does not
require the complete formation of U or Y. Since any block i x j Hankel matrix can be
represented by a block vector of length i+j-1, the storage requirement. Based on this
result, we can expect outcome for this project is the cholesky method faster.

CHAPTER 3

METHODOLOGY

3.1 Introduction

In this chapter discuss about the method and approach to implement the RQ
and Cholesky method in airplane analysis. Below the step will be taken to implement
this project.
a)

Design matrix algorithm for subspace predictor, Yf LwWp LuU f

b)

Write or design programming matlab for RQ method to solve the Y f .

c)

Write or design programming matlab for Cholesky method to solve subspace

predictor.

d)

Compute the data Y f using RQ and Cholesky. Record the time might be taken
for RQ and cholesky method to solve the Y f

14

3.2

Create the MIMO Aircraft system

In this thesis the MIMO Aircraft system has been use to test the QR
algorithm and Cholesky algorithm. This system has two inputs and two outputs. That
mean this system has four states, beta (sideslip angle), phi (bank angle), yaw rate and
roll rate and also have two inputs, the rudder and aileron deflections. The units are
radians for beta (sideslip angle) and phi (bank angle) and radians/sec for yaw (yaw
rate) and roll (roll rate). The rudder and aileron are in degrees. Below show the
transfer function for this system:

Transfer function from input rudder to output

Yaw rate:

-0.475 s^3 0.2479 s^2 0.1187 s 0.05633

----------------------------------------------------------------s^4 + 0.6358 s^3 + 0.9389 s^2 + 0.5116 s + 0.003674

(3.1)

Transfer function from input rudder to output

Bank angle:

0.1148 s^2 0.2004 s 1.373

-------------------------------------------------------------------- (3.2)
s^4 + 0.6358 s^3 + 0.9389 s^2 + 0.5116 s + 0.003674

Transfer function from input aileron to output

Yaw rate:

0.0077 s^3 0.0005372 s^2 + 0.008688 s + 0.004523

---------------------------------------------------------------------- (3.3)
s^4 + 0.6358 s^3 + 0.9389 s^2 + 0.5116 s + 0.003674

15

Transfer function from input aileron to output

Bank angle:

0.1436 s^2 + 0.02737 s + 0.1104

-------------------------------------------------------------------- (3.4)
s^4 + 0.6358 s^3 + 0.9389 s^2 + 0.5116 s + 0.003674

The equation (3.1), (3.2), (3.3) and (3.4) are form of the transfer function of
airplane, the MATLAB programming or command was created and these systems
become are input and output for the Hankel matrix. In this programming the system
was representing in the A, B, C and D matrices. Then the step input was use in this
programming to test the response or performance of this MIMO Aircraft system.

3.3

Create the Hankel matrices

The block Hankel matrices were define from the Aircraft data. The input of
the the Hankel were represent as U p and U f. Then the output were represent as are

Y f and Yp . Then the MATLAB programming for block Hankel matrices was
created as an appendix C and D.

16
3.4

Create QR algorithm

In this thesis the QR method is use to solve least square predictor, L and L .
w

The MATLAB programming was use to create QR algorithm. In this programming,

term H was represent block Hankel matrices contained U f , U p , y p and y f . Then
the matrix H was factorizing using QR method. Matrix R and Q then transposed to
get lower triangular R. After that the matrix R divided in to two part, which is RL1
and pseudo-inverse RL 2 . The Singular Value Decomposition (SVD) was use to solve
the matrix pseudo-inverse RL 2 .
Matrix

W p

U f
Yf

L2

0
R11 0
R21 R22 0 Q T
R31 R32 R33
Matrix

RL1 R31

Pseudo-inverse

(3.5)

L1

R32

L2

R11
=
R21

(3.6)

0
R22

(3.7)

17
3.5

Create Cholesky algorithm

In this project another method that was use to calculate least square predictor
was cholesky method. In this section, the MATLAB programming for Cholesky
algorithm are created. The block hankel for this method was similar with QR
method, which is contained U f , U p , y p and y f . For Cholesky method, the Matrix
R was equal to cholesky factorization of matrix hankel H multiply by Matrix hankel
H transpose. To get R lower triangular, the command [R,p]=chol(B,lower);
was used. Matrix B is equal to Matrix H multiply with transpose H. After that the
matrix R was divided in to two part, which is RL1 and pseudo-inverse RL 2 . Then the
same method (SVD) in QR was use in this section in order to solve Pseudo-inverse
RL 2 . Last but not least, the command for Lw and

Lu was created.

R= chol(AAT)

3.7

(3.8)

Updating and Down dating Data using QR method

In dynamic system updating and down dating the new and old data is very
important in order to engineer analysis data in short time. So in this section the QR
method has been use to update the data. The MATLAB programming command that
used to update the data is [Q1,R1] = qrupdate(Q,H,f,v);

and the command

used to down- date the data is [Q2,R2] = qrdelete(Q,H,j,row);. The

programming for upadating and downdating for QR method showed at appendix D.

18
3.7

Updating and down-date data using Cholesky method

In this thesis, other method that was used to update data and down-date data
is cholesky. The command that was used are Anew = cholupdate(H,a,+);
and Anews = cholupdate(H,b,-);. The programming for upadating and
downdating for Cholesky method showed at appendix C.

3.8

Measure computation Time for QR and Cholesky

In this thesis, the command that was used to measure the computation time
for QR and cholesky method is tic-toc command. This command was measure the
Computer Processing Unit (CPU) time. So time measured was display at command
window in unit second(s). This command applies at the bottom and top of the
MATLAB programming statement. This command functioning like a stop watch, the
command tic means the watch start counting and the command toc mean the
watch stop to counting the time. Below show how the tic toc command was used:

The watch start counting

time
Statement

The watch stop counting time

CHAPTER 4

RESULT AND DISCUSSION

4.1

Discussion

The time taken for QR and Cholesky method to compute the data have been
measured and recorded. The measured showed at Table 4.1, Table 4.2, Table 4.3,
Table 4.4, Table 4.5 and Table 4.6. The average time taken by QR to compute data to
solve the least square predictor is 0.492350 seconds and the time average for
Cholesky method to compute data was 0.466970 seconds. So the Cholesky method is
faster than QR method to compute the data. The time different between QR and
Cholesky was 25.38 mili second.
Table 4.1 shows the time taken for QR and Table 4.4 show the time taken for
Cholesky to compute the data. Next, Table 4.2 show the time taken for QR method to
updating the new data added to the plant. From the table, the average time for QR
updating data is 8.617 mili seconds. Table 4.5 shows that time taken for Cholesky to
update data. The average time is 8.436 mili second. Then time different between this
two method to update data is 0.181 mili second (t t(chol)). So the time taken for
(QR)

Cholesky method to update data is shorter than QR method.

20
Table 4.3 and Table 4.6 show that the time taken for QR and Cholesky to
down-dating the data. From the Table 4.3, the result shows that the average time for
QR to down-date the data is 2.291 mili second. Then from table 4.6, average time for
Cholesky to down-dating the data is 0.031 mili second. So, ime different between
this two method is 2.26ms. The time different for down-dating is more short than
time different for compute the data to find least square predictor and update data.
The Figure 4.1 shows that the performance of MIMO Aircraft system. From
the figure, the results show that the Aircraft system is stable. There have four graphs,
which show the performance of four transfer function of Aircraft system with two
inputs and two outputs. The Graph 1, shown the performance of the input rudder to
output Yaw rate. The second graph showed the performance of the input rudder to
bank angle. The third graph showed the performance of input aileron to output yaw
rate. The fourth graph showed the performance of input aileron to output bank angle.
The graphs show that all the performance of the system is stable.

21

4.2 Result

Table 4.1 Time taken by QR method to determine Lu and Lw

No. of measure

Time (s)

0.503669

0.460108

0.486851

0.571558

0.495226

0.462641

0.592830

0.442809

0.517873

10

0.390042

AVERAGE

0.492350

22

Table 4.2 Time taken by QR method to updating data

No. of measure

Time(ms)

8.889

7.659

12.09

8.036

8.274

8.188

8.960

8.594

7.428

10

8.048

AVERAGE

8.617

23

Table 4.3 Time taken by QR method to down-dating data

No. of measure

Time(ms)

2.081

1.916

2.177

2.793

2.827

2.184

2.665

2.174

1.958

10

2.137

AVERAGE

2.291

24

Table 4.4 Time taken by Cholesky method to determine Lu and Lw

No. of measure

Time(s)

0.500651

0.412089

0.453118

0.498101

0.457423

0.444088

0.435379

0.482314

0.473129

10

0.513413

AVERAGE

0.466970

25

Table 4.5 Time taken by Cholesky method to updating data

No. of measure

Time(ms)

8.815

8.888

9.113

7.396

7.792

8.505

8.929

8.466

7.894

10

8.570

AVERAGE

8.436

26

Table 4.6 Time taken by Cholesky method to down-dating data

No. of measure

Time(ms)

0.190

0.091

0.209

0.087

0.167

0.194

0.191

0.239

0.206

10

0.784

AVERAGE

0.031

27

Figure 4.1 The Performance of Aircraft system

CHAPTER 5

CONCLUSIONS AND FUTURE WORK

5.1 Conclusions
The main objective of this thesis is to compare the computation time between
Cholesky method and QR method. Chapter 4 have shows the result that Cholesky is
faster than QR method in order to compute data during solves the least square
predictor, updating and down-date the data. However, the time different between
these two methods in this thesis is quite small because the data that have been used is
in small amount. Beside that, the CPU of RAM speed may be can influence the time
computation; if the CPU has high speed so the time different between these two
methods is small. Then if the CPU has low speed so the time between these two
methods is large.
The Cholesky method faster than QR because the QR method is requires
more storage location than Cholesky. The QR method has produced more zero
elements in matrix than Cholesky method. In Cholesky method zero elements in the
matrix has been eliminated. So the size of matrix R after instruction chol is small
but for the QR method the size of R matrix become large because zero element
become more.
As a conclusion, for this thesis is the main objective of the thesis is
successful. As an expected outcome, the analysis of this thesis proves that the
Cholesky method is faster than QR method.

29
5.2 Future Work

As a recommendation, use more data input to the system, so that the time
different between QR and Cholesky method is larger. The matrix size that has been
use in this thesis is 32 times 32 so in the future work use larger matrix sizes like 64
times 64. So the CPU will execute the program in long time. Then the different time
between these two methods can measure in large value of time.

30

REFERENCES

[1] G.H GOLUB AND C.F. VAN LOAN, matrix computations, John Hopkins
University Press, Baltimore MD, third edition (1996).
[2] Bruce R. Woodley et.al, Subspace based direct adaptive control, Stanford
University, 2001.
[3] Katrien De Cock et.al, Recursive subspace identification for in-flight modal
analysis of airplane, K.U.Leuven Department of Electrical Engineering,
Belgium, 2006.
[4] Che Rahim Che The, MATLAB Programming Mathematical Approach, Jabatan
Matematik Fakulti Sains, Universiti Teknologi Malaysia,2008.
[5] Matlab, State space Jet Transport Aircraft,2008
[6] I.Geothals et.al, Recursive output only subspace identification for in-flight flutter
monitoring, Dearborn Michigan,2004.

31

APPENDIX A

GANTTS CHART FOR FINAL YEAR PROJECT 1

32

APPENDIX B

GANTTS CHART FOR FINAL YEAR PROJECT 1

33

APPENDIX C

MATLAB PROGRAMMING FOR CHOLESKY METHOD

34

35

36

APPENDIX D

MATLAB PROGRAMMING FOR QR METHOD

37

38

39