LINEAR SYSTEMS OF EQUATIONS

AND
MATRIX COMPUTATIONS

1. DIRECT METHODS FOR SOLVING LINEAR SYSTEMS OF EQUATIONS

1.1

SIMPLE GAUSSIAN ELIMINATION METHOD

Consider a system of n equations in n unknowns,

a11x1 + a12x2 + . + a1nxn = y1
a21x1 + a22x2 + . + a2nxn = y2

an1x1 + an2x2 + . + annxn = yn

We shall assume that this system has a unique solution and proceed to describe the
simple Gaussian Elimination Method, (from now on abbreviated as GEM),Page 2 of 11
for finding the solution. The method reduces the system to an upper triangular system
using elementary row operations (ERO).
Let A(1) denote the coefficient matrix A.

(1)
A =

a 11( 1 )
(1 )
a 21
.....
.....

a 12( 1 )
(1 )
a 22
......
......

.....
.....
.....
......

a 1( 1n )
a 2( 1n)
......
......

a n( 11 )

a n( 12)

......

(1 )
a nn

where aij(1) = aij

Let
y(1) =

y1(1)
(1)
y2
M

y (1)
n

where yi(1) = yi
We assume a11(1) 0
Then by ERO applied to A(1) , (that is, subtracting suitable multiples of the first
row from the remaining rows), reduce all entries below a11(1) to zero. Let the
resulting matrix be denoted by A(2).

(1)

Ri +mi(11)R1

A(2)
2

mi(11) =

where

ai(11)
;
a11(1)

i > 1.

Note A(2) is of the form

A(2)

... ... a1(n1)

... ... a2( 2n)

... ... a3( n2)
M M
M
( 2)
... ... ann

a11(1) a12(1)

( 2)
0 a22
0 a ( 2)
32

M
M

( 2)
0 an 2

Notice that the above row operations on A(1) can be effected by premultiplying
A(1) by M(1) where
1 0 0 L 0 0
(1)

m21

(1)
(1)

M = m31
I n 1

(1)

mn1

(In-1 being the n-1 n-1 identity matrix).

i.e.
M(1) A(1) = A(2)

Let

y(2) = M(1) y(1)

i.e.
R i + m i 1 R1
y (1)
y ( 2 )

Then the system Ax = y is equivalent to

A(2)x = y(2)
Next we assume
(2)
a22
0

and reduce all entries below this to zero by ERO

(2)

Ri +mi(22)

A(3) ;

( 2)
i2

ai(22)
= ( 2) ;
a22

i>3

Here

M ( 2)

1 0

0 1
0 m ( 2)
32
=
( 2)
0 m42

M
M
0 m ( 2)
n2

L
I n 2

and
M(2) A(2) = A(3) ;

M(2) y(2) = y(3) ;

and A(3) is of the form

A( 3 )

a11(1)

0
= 0

a12(1)

...

(2)
a22

( 2)
a23

( 3)
a33

an( 33)

... a1(n1)

... a2( 2n)

... a3(3n)
... M
(3)
... ann

(3)
We next assume a33
0 and proceed to make entries below this as zero. We
(1)
(2)
thus get M , M , . , M(r) where

M (r )

0
M

0
=
0

0
M

0 L

1 L

0 L

0 r ( n r )

0 L mr( +r )1r
0 L mr( +r )2 r
M M
M
0 L

mnr( r )

I n r

(r )

(r )

=A

( r +1)

a11(1)

M
=
M

M
0

...

...

...

...

( 2)
a22

...

...

...

arr( r )

...

...

ar( r++1r1+) 1 ...

...

...

anr( r ++11)

...

a1(n1)

a2( rn)

arn( r )
ar( +r 1+n1)

...
( r +1)
ann

M(r) y(r) = y(r+1)

At each stage we assume arr( r ) 0 .
Proceeding thus we get,
M(1), M(2), . , M(n-1) such that
M(n-1) M(n-2) . M(1) A(1) = A(n)

where

A( n )

a11(1)

M(n-1) M(n-2) . M(1) y(1) = y(n)

a12(1) L a1(n1)

( 2)
a22
L a2( 2n)

O
( n)
ann

which is an upper triangular matrix and the given system is equivalent to

A(n)x = y(n)
Since this is an upper triangular, this can be solved by back substitution; and
hence the system can be solved easily.
Note further that each M(r) is a lower triangular matrix with all diagonal entries as
1. Thus determinant of M(r) is 1 for every r. Now,
A(n) = M(n-1) . M(1) A(1)
Thus
det A(n) = det M(n-1) det M(n-2) . det M(1) det A(1)
det A(n) = det A(1) = det A

since A = A(1)

(2)
(n)
Now A(n) is an upper triangular matrix and hence its determinant is a11(1) a22
L ann
.
Thus det A is given by
(2)
( n)
det A = a11(1) a22
L ann

Thus the simple GEM can be used to solve the system Ax = y and also to
evaluate det A provided aii( i ) 0 for each i.
Further note that M(1), M(2), . , M(n-1) are lower triangular, and nonsingular as
their det = 1 and hence not zero. They are all therefore invertible and their
inverses are all lower triangular, i.e. if L = M(n-1) M(n-2) . M(1) then L is lower
triangular and nonsingular and L-1 is also lower triangular.
Now LA = LA(1) = M(n-1) M(n-2) . M(1) A(1) = A(n)
Therefore A = L-1 A(n)
Now L-1 is lower triangular which we denote by L and A(n) is upper triangular
which we denote by U, and we thus get the so called LU decomposition
A = LU
of a given matrix A as a product of a lower triangular matrix with an upper
triangular matrix. This is another application of the simple GEM. REMEMBER IF
AT ANY STAGE WE GET aii(1) = 0 WE CANNOT PROCEED FURTHER WITH
THE SIMPLE GEM.
EXAMPLE:
Consider the system
x1 + x2 + 2x3 = 4
2x1 - x2 + x3 = 2
x1 + 2x2
=3
Here
1 1 2

A = 2 1 1
1 2 0

4

y = 2
3

= 2
1

(1)

1
1
2

1
0

R 2 2 R1

R 3 R1

0
0

3 = A (2)
2

1
3
1

a11(1) = 1 0
(1)
m21
= 2
(1)
m31
= 1
(2)
= 3 0
a22

(1)

(2)

1 0 0

= 2 1 0
1 0 1

1
R3 + R 2
3

(2)
m31
=1

(2)

0
0

1
3
0

(1 )

4
4

= 2 6 = y (2) = M
3
1

3 = A ( 3)
3

(3)
a33
= 3

= 0

0
1 0

1
1
3

(3)

= M

(2)

(2)

= 6
3

Therefore the given system is equivalent to A(3)x = y(3) ,i.e.,

x1 + x2 + 2x3 = 4
-3x2 - 3x3 = -6
- 3x3 = -3

Back Substitution
x3 = 1
-3x2 - 3 = - 6 -3x2 = -3 x2 = 1

(1 )

y (1 )

x1 + 1 + 2 = 4 x1 = 1
Thus the solution of the given system is,
x1 1

x = x 2 = 1
x 1
3
The determinant of the given matrix A is
(2) (3)
a11(1) a22
a33 = (1)(3)(3) = 9.

Now

(M )
(1)

( 1)

(M ( ) )
2

( 1)

1 0 0

= 2 1 0
1 0 1

1
0
0

= 0
1
0

0 1
1
3

L = M(2) M(-1)
L-1= ( M (2) M (1) ) = ( M (1) )
1

(M )

(2) 1

0
0
1 0 0 1

= 2 1 0 0
1
0

1 0 1
1
0 1

1
0
0

L = L(-1) = 2
1
0

1 1
1
3

(n)

U=A

(3)

=A

2
1 1

= 0 3 3
0 0 3

Therefore A = LU

i.e.,

1
0
0
1 1 2

A= 2 1 1 = 2
1
0

1 2 0
1 1
1

2
1 1

0 3 3
0 0 3

is the LU decomposition of the given matrix A.

We observed that in order to apply simple GEM we need arr( r ) 0 for each
stage r. This may not be satisfied always. So we have to modify the simple
GEM in order to overcome this situation. Further, even if the condition arr( r ) 0 is
satisfied at each stage, simple GEM may not be a very accurate method to use.
What do we mean by this? Consider, as an example, the following system:
(0.000003) x1 + (0.213472) x2 + (0.332147) x3 = 0.235262
(0.215512) x1 + (0.375623) x2 + (0.476625) x3 = 0.127653
(0.173257) x1 + (0.663257) x2 + (0.625675) x3 = 0.285321
Let us do the computations to 6 significant digits.
Here,

0 .000003

A(1) = 0 .215512
0 . 173257

(1)

0 . 235262

= 0 . 127653
0 . 285321

0 .213472
0 .375623
0 . 663257

0 .332147

0 .476625
0 .625675

a11(1) = 0.000003 0

(1)
m21
=

(1)
a21
0.215512
=
= 71837.3
(1)
0.000003
a11

(1)
m31
=

(1)
a31
0.173257
=
= 57752.3
(1)
0.000003
a11

(1)

= 71837 .3
57752 . 3

0
1
0

0 ;
1

0.000003

0
A(2) = M(1) A(1) =

(2)

(1)

=M

(1)

0.235262

16900.5
13586.6

0.332147

15334 .9 23860 .0
12327 .8 19181 .7
0.213472

(2)
a22
= 15334.9 0

(2)
=
m32

(2)
a32
12327.8
=
= 0.803905
(2)
15334.9
a22

0
0
1

1
0
M(2) = 0
0 0.803905 1

(3)

(2)

=M

(2)

0 .235262

= 16900 . 5
0 . 20000

0.000003

0
A(3) = M(2) A(2) =

0.213472
15334 .9
0

0.332147

23860 .0
0.50000

Thus the given system is equivalent to the upper triangular system

A(3)x = y(3)
Back substitution yields,
x3 = 0.40 00 00
x2 = 0.47 97 23
x1 = -1.33 33 3

10

This compares poorly with the correct answers (to 10 digits) given by
x1 = 0.67 41 21 46 94
x2 = 0.05 32 03 93 39.1
x3 = -0.99 12 89 42 52
Thus we see that the simple Gaussian Elimination method needs modification in
order to handle the situations that may lead to arr( r ) = 0 for some r or situations as
arising in the above example. In order to do this we introduce the idea of Partial
Pivoting in the next section.

11