Math 415 - Lecture 6

LU-decomposition

Friday September 2th 2016

Textbook: Chapter 1.5
Suggested Practice Exercise: Chapter 1.5 Exercise 4, 5, 11, 23, 29

Elementary matrices
Definition. The n n identity matrix In has all entries 0, except on the main diagonal where the
entries are 1.

1 0 0
Example 1. I3 = 0 1 0 .
0 0 1
Definition. An elementary matrix is one that is obtained by performing a single elementary row
operation on an identity matrix. A permutation matrix is one that is obtained by performing row
exchanges on an identity matrix.

1 0 0
1 0 0
1 0 0
Example 2. Let E1 = 0 2 0 , E2 = 0 0 1 , E3 = 0 1 0 . Then E1 , E2 , and E3
3 0 1
0 1 0
0 0 1
are elementary matrices. Why? Are there any permutation matrices?
Solution.

1
0
0

1
0
0

1
0
0

0
1
0
0
1
0
0
1
0

1 0
0
0 0 2
R22R2
0 0
1

0
1 0
0 0 0
R2R3
1
0 1

0
1
0
0

R3R3+3R1
1
0

0
0
1

0
1
0
0
0
1

0
1
0

Question. Let A be a 3 3-matrix. What happens to A if you multiply it by one of E1 , E2 and E3 ?

Solution.


a
Let A = d
g

b
e
h

c
f . Then
i

1
E1 A = 0
0

0
2
0

0
a
0 d
1
g

1 0 0
E2 A = 0 0 1
0 1 0

1 0 0
a b
E3 A = 0 1 0 d e
3 0 1
g h

b
e
h

c
a
b
f = 2d 2e
i
g h

c
2f
i

b c
a b c
e f = g h i
h i
d e f

c
a
b
c
f = d
e
f
i
g + 3a h + 3b i + 3c
a
d
g

Thus multiplying A by E1 is the same applying the row operation R2 2R2 to A. This is the same
operation that was used to create E1 from I3 ! Multiplying A by E2 is the same applying the row
operation R1 R2 to A. And finally, multiplying A by E3 is the same applying the row operation
R3 R3 + 3R1 to A.
Theorem 1. If an elementary row operation is performed on an mn- matrix A, the resulting matrix
can be written as EA, where the m m-matrix E is created by performing the same row operations
on Im .

Inverses of elementary matrices

Example 3. Consider the elementary row operation R3 R3 + 3R1. Is there an elementary row
operation that undoes this row operation (independently from the matrix this operation is applied
to)?
Solution.
Yes. The operation R3 R3 3R1 reverses the row operations R3 R3 + 3R1. For example,

1 0 0
1 0 0
1 0 0
0 1 0
0 1 0 .
0 1 0

R3R3+3R1
R3R33R1
0 0 1
0 0 1
3 0 1

Remark. Indeed, every row operation is reversible. We already saw how to reverse the replacement
operator. The scaling operatior R2 cR2 is reversed by the scaling operator R2 1c R2. Row
interchange R1 R2 is reversible by performing it twice.
Definition. An n n matrix A is said to be invertible if there is an n n matrix C satisfying

1
Example 4. What is the inverse of 0
3

CA = AC = In .

0 0
1 0?
0 1

Solution.

0 0 1
1 0 0
0 1 3

1
0
3
We write this inverse as

1
0
3

0
1
0

0
1
0

0
1
0 = 0
1
0

1
0
1
0 = 0
1
3

0
1
0

0
0
1

0 0
1 0
0 1

Theorem 2. Elementary matrices are invertible, because row operations are reversible. To determine
the inverse of an elementary matrix E, determine the elementary row operation needed to transform
E back into I and apply this operation to I to find the inverse.
More on inverses soon.

1
Example 5. Compute the inverses of A = 2
0

0
1
0

0
1
0, B = 0
1
0

0
2
0

0
1
0 and C = 0
1
0

0
0
1

0
1.
0

Solution.
The matrix A is obtained from I3 by the row operation R2 R2 + R1. The reverse of this row
operation is R2 R2 R1. Thus

1
2
0

0
1
0

1
1
0
0 = 2
0
1

0 0
1 0
0 1

The matrix B is obtained from I3 by the row operation R2 2R2. The reverse of this row operation
is R2 21 R2. Thus

1 0 0
1 0 0
0 2 0 = 0 1/2 0
0 0 1
0 0 1
The matrix C is obtained from I3 by the row operation R2 R3. The reverse of this row operation
is R2 R3. Thus

1 0 0
1 0 0
0 0 1 = 0 0 1
0 1 0
0 1 0

Triangular matrices

Definition. An n n matrix A is called upper triangular if it is of the form

0 0

.
0 0 0 . . . ..
0 0 0 0

An n n matrix B is called lower triangular

0 0
0

if it is of the form

0 0
0 0

0 0
.
..
..
. .

Example 6. Give a few examples of upper and lower triangular matrices!

Solution.
Upper triangular matrices:

1
0

 2
0
, 0
1
0

3
3
0

1
4
2

 2
0
, 0
1
1

0
3
2

Lower triangular matrices:


0
0

 
0
5
,
1
2

0
0
2

Neither upper nor lower triangular:


0
2


2
1

Definition. A matrix A has LU factorization if there is a lower triangular matrix L and a upper
triangular matrix U such that
A = LU.
In practice, L will have all 1s on the main diagonal. When is this possible?
Theorem 3. Let A be an n n-matrix. If A can be transformed into echelon form without the use
of row exchanges, then A has LU factorization.

2
1 1
Example 7. Let A = 4 6 0 . Can we transform it into Echelon form without row exchanges?
2 7 2

2
1 0 0
1 1
2
1
1
E1 A = 2 1 0 4 6 0 = 0 8 2 , `21 = 2
0 0 1 2 7 2
2 7
2

2
1
1
1 0 0
2 1
1
8 2 = 0 8 2 , `31 = 1
E2 (E1 A) = 0 1 0 0
1 0 1
0 8
3
2
7
2

1
1
1 0 0 2
2 1
1
E3 (E2 E1 A) = 0 1 0 0 8 2 = 0 8 2 , `32 = 1
0 1 1 0
0 0
1
8
3

We got an upper triangular matrix!

2
E3 E2 E1 A = 0
0

1
8
0
4

1
2 = U
1

(Always works - if an n n matrix is in echelon form, then it is upper triangular.) We need to reverse
these operations:
E3 E2 E1 A = U = A = E11 E21 E31 U

E11

E31

1
= 2
0

1
= 0
0

0
1
0

0
0
1

0
1
1

E11 E21 E31

E21

0
0
1

1
= 2
1

1
= 0
1

0
1
1

0 0
1 0
0 1

0
0 = L
1

This is a lower triangular matrix! Product of lower triangular matrices is lower triangular. So the
LU decomposition is

1
0 0
2 1
1
1 0 0 8 2
A = LU = 2
1 1 1
0 0
1
Summary:
If A can be brought in echelon form without row exchanges we have A = LU ,
U - Echelon form of A
L = E11 E21 E31 where E1 , E2 , E3 were elementary matrices that put A into Echelon form.
(No row exchanges!)

An easier way to compute L

1
0 0
In previous example L = I + `21 0 0 where `ij is the factor between pivot and the entry you
`31 `32 0
want to make zero in the elimination process: see the boxed numbers. This works in general if you
do the row operations in the right order. Here is what you have to do:

Step 1:

2
A= 4
2

1
6
7

1
0

2 R2R2 2 R1,R3R3+ 1

2
= 0
R1
0

1
8
8

Then set `21 = 2 and `31 = 1.

Step 2:

2 1
1
2 1
1
0 8 2

= 0 8 2 .
0 8
3 R3R3+ 1 R2
0 0
1

2 1
1
Set `32 = 1, U := 0 8 2 , and
0 0
1

1
0 0
1
0 0
1 0 = 2
1 0 .
L := `21
`31 `32 1
1 1 1
5

1
2
3


1
Example 8. Determine the LU -decomposition of 4
4

2
4
4

2
4.
8

Solution.

1
4
4

2
4
4

2
4

8 R2R2 4 R1,R3R3 4

Thus

1
U := 0
0

1
0
R1 0

2
4
4

2
4

0 R3R3 1

2
1
4 , L := 4
4
4

2
4
0

0
1
1

1
0
R2 0

2
4
0

2
4 .
4

0
0 .
1

Check that A = LU .
Remark. It is important that you do the row operations in the right order! Suppose you do the
following:

1 2 2
1 2 2
1 2
2
4 4 4
4 4 4
0 4 4

4 4 8 R3R3 1 R2 0 0 4 R2R2 4 R2 0 0
4
and you set

1
U := 0
0

2
4
0

1
2
4 , L := 4
0
4

0
1
1

0
0 .
1

This L is wrong!

1
LU = 4
0

2
4
4

2
4 6= A.
0

Row exchanges

Recall that a permutation matrix P is a square matrix obtained from the identity matrix by reordering
the rows.
Theorem 4. Let A be a n n-matrix that can be brought to echelon form. Then there is permutation
matrix P such that P A has LU factorization.
Why?
Solution.
If A can be brought to echelon form with the help of row exchanges, we can do those exchange
first. So there is a permutation matrix P such that P A can be brought to echelon form without row
exchanges.

0
Example 9. Let A = 1
2

0
1
1

1
0. Find P A that has a LU factorization.
0
6

Solution.

Lets rearrange the rows:

Move the 2nd row to the 1st row
Move the 3rd row to the 2nd row
Move the 1st row to the 3rd row
Do these moves to the identity matrix to get P :

0
P = 0
1

1
0
0

0
1
0

Doing these moves to A gives the same matrix as P A:

1 1 0
P A = 2 1 0
0 0 1
We can bring P A to echelon form without row exchanges (check

1 1
1 0 0
P A = LU = 2 1 0 0 1
0 0
0 0 1

this!) so P A = LU :

0
0
1