1

GAUSS ELIMINATION
2
An equation of the form ax+by+c=0 or equivalently
ax+by=-c is called a linear equation in x and y variables.
ax+by+cz=d is a linear equation in three variables, x, y,
and z.
Thus, a linear equation in n variables is
a
1
x
1
+a
2
x
2
+ +a
n
x
n
= b
A solution of such an equation consists of real numbers c
1
,
c
2
, c
3
, , c
n
. If you need to work more than one linear
equations, a system of linear equations must be solved
simultaneously.
3
There are many ways to solve a system of linear
equations:
Graphical method
Cramers rule
Method of elimination
Computer methods
For n 3
Solving Small Numbers of Equations
4
For two equations:
Solve both equations for x
2:
2 2 22 1 21
1 2 12 1 11
b x a x a
b x a x a
= +
= +
11 1
2 1 2 1
12 12
21 2
2 1
22 22
(slope) intercept
a b
x x x x
a a
a b
x x
a a
| |
= + = +
|
\
| |
= +
|
\
5
Plot x
2
vs. x
1
,
the
intersection of
the lines
present the
solution.
A system of linear equations (with n equations
and n unknowns) can generally be written as:
1 1 3 13 2 12 1 11
... b x a x a x a x a
n n
= + + + +
2 2 3 23 2 22 1 21
... b x a x a x a x a
n n
= + + + +
n n nn n n n
b x a x a x a x a = + + + + ...
3 3 2 2 1 1
. .
. .
. .
or
simply as: [ ][ ] [ ]
= A x b
7
The basic strategy is to successively solve one of
the equations of the set for one of the unknowns
and to eliminate that variable from the remaining
equations by substitution.
The elimination of unknowns can be extended to
systems with more than two or three equations;
however, the method becomes extremely tedious
to solve by hand.
8
Extension of method of elimination to large sets of
equations by developing a systematic scheme or
algorithm to eliminate unknowns and to back
substitute.
The method is called naive because it does not
avoid the division by zero.
As in the case of the solution of two equations, the
technique for n equations consists of two phases:
Forward elimination of unknowns
Back substitution
9
Fig. 9.3

(
(
(

1 12 144
1 8 64
1 5 25
The goal of Forward Elimination is to transform
the coefficient matrix into an Upper Triangular
Matrix

7 . 0 0 0
56 . 1 8 . 4 0
1 5 25
(
(
(

EXAMPLE:
Transform to an Upper Triangular Matrix
Step 1: Eliminate x
1
in 2
nd
equation using
equation 1 as the pivot equation
) (
21
11
1
a
a
Eqn

which will yield

1
11
21
1
11
21
2 12
11
21
1 21
... b
a
a
x a
a
a
x a
a
a
x a
n n
= + + +
Zeroing out the coefficient of x
1
in the 2
nd
equation.
Subtract this equation from 2
nd
equation
1
11
21
2 1
11
21
2 2 12
11
21
22
... b
a
a
b x a
a
a
a x a
a
a
a
n n n
=
|
|

\
|
+ +
|
|

\
|

'
2
'
2 2
'
22
... b x a x a
n n
= + +
n n n
a
a
a
a a
a
a
a
a a
1
11
21
2
'
2
12
11
21
22
'
22


=
=

or
where :
Repeat this procedure for the remaining
equations to reduce the set of equations as
1 1 3 13 2 12 1 11
... b x a x a x a x a
n n
= + + + +
'
2
'
2 3
'
23 2
'
22
... b x a x a x a
n n
= + + +
'
3
'
3 3
'
33 2
'
32
... b x a x a x a
n n
= + + +
' '
3
'
3 2
'
2
...
n n nn n n
b x a x a x a = + + +
. . .
. . .
. . .
) (
32
22
2
3
a
a
Eqn
Eqn
(

Step 2: Eliminate x
2
in the 3
rd
equation.
Equivalent to eliminating x
1
in the 2
nd
equation using equation 2 as the pivot equation.
This procedure is repeated for the remaining
equations to reduce the set of equations as
1 1 3 13 2 12 1 11
... b x a x a x a x a
n n
= + + + +
'
2
'
2 3
'
23 2
'
22
... b x a x a x a
n n
= + + +
"
3
"
3 3
"
33
... b x a x a
n n
= + +
" "
3
"
3
...
n n nn n
b x a x a = + +
. .
. .
. .
Continue this procedure by using the third equation
as the pivot equation and so on.
At the end of (n-1) Forward Elimination steps, the
system of equations will look like:
'
2
'
2 3
'
23 2
'
22
... b x a x a x a
n n
= + + +
"
3
"
3
"
33
... b x a x a
n n
= + +
( ) ( ) 1 1
=
n
n n
n
nn
b x a
. .
. .
. .
1 1 3 13 2 12 1 11
... b x a x a x a x a
n n
= + + + +
(
(
(
(
(
(

=
(
(
(
(
(
(

(
(
(
(
(
(

) - (n
n n
3
2
1
n
nn
n
n
n
b
b
b
b
x
x
x
x
a
a a
a a a
a a a a
1
"
3
'
2
1
) 1 (
"
3
"
33
'
2
'
23
'
22
1 13 12 11

At the end of the Forward Elimination steps

The goal of Back Substitution is to solve each of
the equations using the upper triangular matrix.
(
(
(

=
(
(
(

(
(
(

3
2
1
3
2
1
33
23 22
13 12 11
x
x
x

0 0
0
b
b
b
a
a a
a a a
Example of a system of 3 equations
Start with the last equation because it has only one
unknown
) 1 (
) 1 (

=
n
nn
n
n
n
a
b
x
Solve the second from last equation (n-1)
th
using
x
n
solved for previously.
This solves for x
n-1
.
Representing Back Substitution for all equations
by formula
( ) ( )
( ) 1
1
1 1

+ =


=
i
ii
n
i j
j
i
ij
i
i
i
a
x a b
x
for i=n-1, n-2,.,1
and
) 1 (
) 1 (

=
n
nn
n
n
n
a
b
x
Time, t Velocity, v
s m/s
5 106.8
8 177.2
12 279.2
The upward velocity of a rocket
is given at three different times
The velocity data is approximated by a polynomial as:
( ) 12. t 5 ,
3 2
2
1
+ + = a t a t a t v
Find: The Velocity at t = 6, 7.5, 9 and 11 seconds.
( ) 12. t 5 , a t a t a t v + + =
3 2
2
1
Assume
(
(
(

=
(
(
(

(
(
(

3
2
1
3
2
3
2
2
2
1
2
1
1
1
1
v
v
v
a
a
a

t t
t t
t t
3
2
1
Results in a matrix template of the form:
Using date from the time / velocity table, the matrix becomes:
(
(
(

=
(
(
(

(
(
(

2 . 279
2 . 177
8 . 106
1 12 144
1 8 64
1 5 25
3
2
1
a
a
a

Forward Elimination: Step 1
=
(

) 64 (
25
1
2
Row
Row
Yields
(
(
(

=
(
(
(

(
(
(

2 . 279
21 . 96
81 . 106
a
a
a

1 12 144
56 . 1 8 . 4 0
1 5 25
3
2
1
=
(

) 144 (
25
1
3
Row
Row
(
(
(

=
(
(
(

(
(
(

0 . 336
21 . 96
8 . 106
a
a
a

76 . 4 8 . 16 0
56 . 1 8 . 4 0
1 5 25
3
2
1
Yields
Forward Elimination: Step 1
Yields
=
(

) 8 . 16 (
8 . 4
2
3
Row
Row
(
(
(

=
(
(
(

(
(
(

735 . 0
21 . 96
8 . 106
a
a
a

7 . 0 0 0
56 . 1 8 . 4 0
1 5 25
3
2
1
This is now ready for Back Substitution
Forward Elimination: Step 2 Back Substitution: Solve for a
3
using the third equation
735 . 0 7 . 0
3
= a
7 0
735 0
.
.
a
3
=
050 1. a
3
=
Back Substitution: Solve for a
2
using the second equation
21 . 96 56 . 1 8 . 4
3 2
= a a
8 . 4
56 . 1 21 . 96
3
2

+
=
a
a
( )
8 4
050 1 56 1 21 96
. -
. . . -
a
2
+
=
70 19.
2
= a
Back Substitution: Solve for a
1
using the first equation
8 . 106 5 25
3 2 1
= + + a a a
25
5 8 . 106

3 2
1
a a
a

=
( )
25
050 . 1 70 . 19 5 8 . 106
1

= a
2900 . 0
1
= a
The solution vector is
(
(
(

=
(
(
(

050 . 1
70 . 19
2900 . 0
3
2
1
a
a
a
The polynomial that passes through the three data points is
then:
( )
3 2
2
1
a t a t a t v + + =
12 5 , 050 . 1 70 . 19 2900 . 0
2
+ + = t t t
Solution:
Substitute each value of t to find the corresponding velocity
( ) ( ) ( )
. s / m 1 . 165
050 . 1 5 . 7 70 . 19 5 . 7 2900 . 0 5 . 7 v
2
=
+ + =
( ) ( ) ( )
. s / m 8 . 201
050 . 1 9 70 . 19 9 2900 . 0 9 v
2
=
+ + =
( ) ( ) ( )
. s / m 8 . 252
050 . 1 11 70 . 19 11 2900 . 0 11 v
2
=
+ + =
( ) ( ) ( )
. / 69 . 129
050 . 1 6 70 . 19 6 2900 . 0 6
2
s m
v
=
+ + =
31
Division by zero. It is possible that during both
elimination and back-substitution phases a
division by zero can occur. Consider the set
of equations:
6 5 5
901 . 3 3 099 . 2 6
7 7 10
3 2 1
3 2 1
3 2
= +
= + +
=
x x x
x x x
x x
32
Round-off errors.
Ill-conditioned systems. Systems where small changes in
coefficients result in large changes in the solution.
Alternatively, it happens when two or more equations are
nearly identical, resulting a wide ranges of answers to
approximately satisfy the equations. Since round off errors
can induce small changes in the coefficients, these changes
can lead to large solution errors.
Singular systems. When two equations are identical, we
would loose one degree of freedom and be dealing with the
impossible case of n-1 equations for n unknowns. For large
sets of equations, it may not be obvious however. The fact
that the determinant of a singular system is zero can be used
and tested by computer algorithm after the elimination stage.
If a zero diagonal element is created, calculation is
terminated.
Increase the number of significant digits
Decreases round off error
Does not avoid division by zero
Gaussian Elimination with Partial Pivoting
Avoids division by zero
Reduces round off error
1. Explain what is Gaussian Elimination with
partial pivoting.
2. Give an example of a problem together
with its solution using Gaussian
Elimination with partial pivoting
technique.
Submit on 5
th
Dec 2011