You are on page 1of 19

4.3.

The QR Reduction
Reading Trefethen and Bau (1997), Lecture 7 The QR factorization of a matrix A 2 <m n is

{ Q 2 <m m is an orthogonal matrix { R 2 <m n is upper triangular { Assume (for the moment) m n
Motivation: { An alternate to Gaussian elimination when solving Ax = b More expensive than Gaussian elimination { Determining bases for orthogonal subspaces { Determining numerical rank through the SVD { Solving least squares problems min jjAx ; bjj2 x

A = QR

(1)

{ Solving algebraic eigenvalue problems Ax = x

Householder QR Factorization
Use Householder re ections to reduce successive columns of A to zero below their main diagonals { Suppose m = 6, n = 4 2 3 a11 a12 a13 a14 6 a21 a22 a23 a24 7 6 6 a31 a32 a33 a34 7 7 6 A=6a a a a 7 6 41 42 43 44 7 7 6 7 4 5

a51 a52 a53 a54 a61 a62 a63 a64 r12 a(1) 22 a(1) 32 a(1) 42 a(1) 52 a(1) 62 r13 a(1) 23 a(1) 33 a(1) 43 a(1) 53 a(1) 63

{ Construct a re ection H1 such that


2 r11 6 0 6 6 6 0 H1 A = 6 0 6 6 6 0 4 0

r14 a(1) 7 24 7 (1) 7 a34 7 7 (1) 7 a44 7 a(1) 7 54 5 a(1) 64

Householder QR Factorization
Use a re ection to annihilate a(1), i = 3 : 6 i2 2 (1) 3 2 3 a 6 22 7 6 r22 7 6 a(1) 7 6 0 7 6 32 7 6 7 P2 6 a(1) 7 = 6 0 7 6 42 7 6 7 6 a(1) 7 4 0 5 4 52 5 0 a(1) 62

{ Apply the transformation


to get

1 0 H2 = 0 P 2

2 r11 6 0 6 6 6 H2H1A = 6 0 6 0 6 6 0 4 0

r12 r13 r22 r23 0 a(2) 33 0 a(2) 43 0 a(2) 53 0 a(2) a(2) 63 64

3 r14 r24 7 7 (2) 7 a34 7 7 (2) 7 a44 7 a(2) 7 54 5

Householder QR Factorization
Repeat the process on the third column 2 r11 r12 r13 6 0 r22 r23 6 6 0 0 r33 H3H2H1A = 6 0 0 0 6 6 6 4 0 0 0 0 0 0 One more should do it

r14 3 r24 7 7 r34 7 7 (2) 7 a44 7 7 a(2) 5 54 a(2) 64


3 r14 r24 7 7 r34 7 7 r44 7 7 7 0 5 0

2 r11 6 0 6 6 0 H4H3H2H1A = 6 0 6 6 6 0 4 0

r12 r13 r22 r23 0 r33


0 0 0 0 0 0

Householder QR Factorization
Algorithm for a Householder QR factorization function A = houseqr(A) % houseqr: Overwrite A 2 <m n with the product % QR where Q 2 <m m is orthogonal and % R 2 <m n is upper triangular. On return, % the essential part of Q is stored in the lower triangular % part of A and R is stored in the upper triangular part. for j = 1 : n v(j:m) = house(A(j:m,j)) A(j:m,j:n) = row.house(A(j:m,j:n), v(j:m)) if j < m A(j+1:m,j) = v(j+1:m) end end

Householder QR Factorization
As described in Example 2.4,

v(j)(v(j))T Hj = I ; 2 (v(j))T v(j) { v(j) is the Householder vector v used at step j { v(j) has the form ) ( v(j) = 0 0 ::: 0 1 vj(j+1 ::: vmj)]T

(2a)

(2b)

Normalize the Householder vector so that vj(j ) = 1 { This requires modi cation of the function house of Section 4.2. All components of v are divided by v(1) { The renormalization saves a storage location and allows v(j) to be stored in the lower part of A Upon completion of the algorithm 3 2 r11 r12 r13 r14 6 v(1) r22 r23 r24 7 7 6 2 6 (1) (2) 6 v3 v3 r33 r34 7 7 6 (1) (2) (3) (2c) A=6v v v r 7 7 44 6 4 4 4 (4) 7 6 v(1) v(2) v(3) v 7 4 5 5 5 5 5
(1) (2) (3) (4) v6 v6 v6 v6

Householder QR Factorization
QT = HnHn;1 H1 so Q = H1H2 Hn { Q rarely needs explicit construction
(3)

The factorization requires about 2n2(m ; n=3) FLOPs ( multiplications plus additions) { With m = n we require about 4n3=3 FLOPs { Gaussian elimination requires about 2n3=3 FLOPs With roundo , we compute ^ ^ QT (A + A) = R ^ { Q is exactly orthogonal ^ { R is upper triangular { The perturbation jj Ajj2 ujjAjj2 { This is acceptable

Givens QR Factorization
We can also use Givens rotations to construct the QR decomposition { The order of the reduction is { stands for a nonzero entry { The numbers indicate the rows that are rotated 2 3 2 3 2 3 6 7 7 26 7 6 7= 6 6 7= 6 7 A=6 7 36 7 360 7 4 5 4 5 4 5 4 0 0 2 3 2 3 2 3 1 7 7 260 7 260 6 7= 6 0 6 7= 6 A= 6 0 7 360 7 360 0 7 7 4 5 4 5 4 5 0 4 0 0 0 0 2 3 60 7 6 A= 3 6 0 0 7 7 4 5 4 0 0 0

Givens QR Factorization
Algorithm for the Givens QR factorization function A = givensqr(A) % givensqr: Reduce A 2 <m n to upper triangular % form by the Givens QR procedure where Q 2 <m m % is orthogonal and R 2 <m n is upper triangular. % On return, R is stored in the upper triangular part of A. for j = 1:n for i = m: -1: j + 1 c,s] = givens(A(i-1,j), A(i,j)) A(i-1:i,j:n) = row.rot(A(i-1:i,j:n), c, s) end end

{ The procedure does not save Q { The factorization requires about 3n2(m ; n=3) multiplications and additions Re ections require 2n2(m ; n=3) FLOPs Re ections are generally preferred { Let Gi = G(i i ; 1 ), then

Q = (GmGm{z1 G2} (Gm {z G3} )| ) ; |


1

st col:

nd col:

n th col:

(Gm} | {z )

Givens QR Factorization
Example 1. Obtain the QR factorization of 2 3 3 2 1 6 2 ;3 4 7 A = 6 5 1 ;1 7 6 7 4 5 7 4 2

{ Rotating the third and fourth rows using givens(5,7)


c = ;0:5812
2 3:0000 6 2:0000 A(1) = 6 ;8:6023 6 4 0

s = 0:8137
2:0000 ;3:0000 ;3:8362 ;1:5112

{ Applying row.rot to A(3 : 4 1 : 3) yields

3 1:0000 4:0000 7 7 7 ;1:0462 5 ;1:9762

{ The Givens matrix corresponding to this rotation is G(3 4 )T = GT 4


2 3 2 1 0 0 0 1 0 0 60 1 0 07 60 1 = 6 0 0 c ;s 7 = 6 0 0 ;0:0 6 7 6 5812 4 5 4 0 0 s c 0 0 0:8137 { We may check that GT A = A(1) 4

3 0 0 7 7 7 ;0:8137 5 ;0:5812

10

Givens QR Factorization
Rotate the second and third rows using givens(2.0000, -8.6023)

{ Applying row.rot to A(1)(2 : 3 1 : 3) yields


2 3:0000 6 8:8318 (2) A =6 6 0 4 0
2:0000 3:0571 ;3:7908 ;1:5112

c = 0:2265

s = 0:9740

3 1:0000 1:9249 7 7 3:6592 7 5 ;1:9762 3 0 07 7 07 5 1

{ The rotation matrix is

2 1 0 6 0 0:2265 GT = 6 0 0:9740 6 3 4 0 0

0 ;0:9740 0:2265 0

{ We may check that GT GT A = A(2) 3 4

11

Givens QR Factorization
Rotate the rst and second rows using givens(3.0000, 8.8318)

{ Applying row.rot to A(2)(1 : 2 1 : 3) yields


2 ;9:3274 6 0 (3) A =6 6 0 4 0
;3:5380

c = ;0:3216

s = 0:9469

0:9104 ;3:7908 ;1:5112

3 ;2:1442 0:3278 7 7 3:6592 7 5 ;1:9762 3 0 07 7 07 5 1

{ The Givens matrix

2 ;0:3216 6 0:9469 GT = 6 6 2 0 4 0

;0:9469 ;0:3216

0 0 0 1 0 0

{ The rst column has been reduced

12

Rotate the third and fourth rows using givens(-3.7908, -1.5112) c = 0:9289 s = ;0:3703 { Applying row.rot to A(3)(3 : 4 2 : 3) yields 3 2 ;9:3274 ;3:5380 ;2:1442 6 0 0:9104 0:3278 7 6 7 (4) A =6 0 ;4:0809 2:6672 7 4 5 0 0 ;3:1908 { The Givens matrix2 3 1 0 0 0 60 1 0 07 T =6 G4 6 0 0 0:9289 ;0:3703 7 7 4 5 0 0 0:3703 0:9289 Rotate the second and third rows using givens(0.9104, -4.0809) c = 0:2177 s = 0:9760 { Applying row.rot to A(4)(2 : 3 2 : 3) yields 3 2 ;9:3274 ;3:5380 ;2:1442 6 0 4:1812 ;2:5318 7 7 6 (5) A =6 0 0 0:9007 7 5 4 0 0 ;3:1908 { The Givens matrix2 3 1 0 0 0 6 0 0:2177 ;0:9760 0 7 GT = 6 0 0:9760 0:2177 0 7 6 7 3 4 5 0 0 0 1 { The second column is done
13

Givens QR Factorization

The Last Rotation


Rotate the third and fourth rows using givens(0.9007, -3.1908)

{ Applying row.rot to A(5)(3 : 4 3) yields


2 ;9:3274 6 0 (6) A =6 6 0 4 0 2 1 60 GT = 6 0 6 4 4 0
;3:5380

c = 0:2717

s = 0:9624
4:1812 0 0

3 ;2:1442 7 ;2:5318 7 3:3154 7 5 0

{ The Givens matrix

0 0 1 0 0 0:2717 0 0:9624

3 0 07 7 7 ;0:9624 5 0:2717

14

The Factored Form


The factored matrix 2 ;9:3274 6 0 R=6 6 0 4 0 The orthogonal matrix { We have
3

;3:5380

4:1812 0 0

3 ;2:1442 7 ;2:5318 7 3:3154 7 5 0

QT = (G4) (GT GT } (GT GT GT } 3 |{z} | 3{z 4 ) | 2 {z 4 )


rd col:
2

2 ;0:3216 0:2062 6 ;0:2144 ;0:8989 Q = 6 ;0:5361 ;0:2144 6 4 ;0:7505 0:3216 { Check that QR = A { Check that QT Q = I

nd col:

st col:

0:2511 0:3813 ;0:8121 0:3635

3 0:8894 0:0232 7 7 0:0851 7 5 ;0:4486

15

Properties of the QR Factorization


{ A = a1 a2 an] etc. { Let Qn = q1 q2 qn], Qm;n = qn+1 qn+2 { Then range(A) = range(Qn) range(A)? = range(Qm;n)

Theorem 1: Let A 2 <m n have rank n with m n. Let A = QR, then spanfa1 a2 ak g = spanfq1 q2 qk g k = 1 : n

(4a)

qm ]
(4b) (4c) (4d)

The ? denotes the orthogonal complement { In addition A = QnRn Rn = R(1 : n 1 : n) Remark: R = Rn Q = Qn Qm;n 0

{ Thus,

A = QR = Qn Qm;n
Proof: From (1)

Rn = QnRn 0 qk g qk g
(5)

ak =
Thus, spanfa1

k X j =1

rjk qj 2 spanfq1 q2

a2

ak g spanfq1 q2
16

Fat and Skinny QRs


But rank(A) = n thus, dim(spanfa1 a2 ak g) = k So spanfa1 a2 ak g = spanfq1 q2 qk g Thus, (4a) is satis ed The rest of the Theorem follows directly 2 We call (4c) the thin or reduced QR factorization of A Theorem 2: Let A 2 <m n have rank n, then the thin QR factorization (4c) is unique. { Rn is upper triangular with positive diagonal entries { Consider the Cholesky factorization of AT A = LLT (6a) L is lower triangular { Then Rn = LT (6b)

Proof: Consider

{ Comparing this with (6a) yields (6b) { The Cholesky factorization is unique { From (4d) Qn = AR;1 hence, Qn is unique 2 n
17

AT A = RT QT QnRn = RT Rn n n n

The Condition Number of a Rectangular Matrix


What is the condition number of an m n matrix? Theorem 3: Let A 2 <n n, then { The eigenvalues i, i = 1 : n, of AT A and the singular values i, i = 1 : n, satisfy
i

i=1:n

(7a)

{ The eigenvectors xi of AT A and the singular vectors vi of A satisfy xi = vi i = 1 : n (7b) { The Euclidean norm q jjAjj2 = (AT A) = 1 (7c) The spectral radius (AT A) is the largest eigenvalue T 1 of A A 1 is the largest singular value of A { The condition number of A in the Euclidean norm satis es (A) (7d) (A) = 1 2 (A)
n

18

The Condition Number of a Rectangular Matrix


Proof:

We can use this as a de nition of the condition number for a rectangular matrix { An O( ) perturbation in A introduces O( 2(A)) perturbations in Qn and Rn cf. Stewart (1993), On the perturbation of LU Cholesky and QR factorizations, SIAM J. Matrix. Anal., 14, 1141-1145
19

You might also like