Gauss-Siedel Method

Major: All Engineering Majors

Authors: Autar Kaw

http://numericalmethods.eng.usf.
edu

Transforming Numerical Methods Education for STEM

Undergraduates

03/21/15

http://numericalmethods.eng.usf.e
du

Gauss-Seidel Method

http://numericalmethods.eng.usf
.edu

Gauss-Seidel Method
An iterative method.
Basic Procedure:
-Algebraically solve each linear equation for xi
-Assume an initial guess solution array
-Solve for each xi and repeat
-Use absolute relative approximate error after each iteration
to check if error is within a pre-specified tolerance.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Why?
The Gauss-Seidel Method allows the user to control round-off error.

Elimination methods such as Gaussian Elimination and LU

Decomposition are prone to prone to round-off error.

Also: If the physics of the problem are understood, a close initial

guess can be made, decreasing the number of iterations needed.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
A set of n equations and n unknowns:

a11 x1 a12 x2 a13 x3 ... a1n xn b1

a21 x1 a22 x2 a23 x3 ... a2n xn b2
.
.
.

.
.
.

an1 x1 an 2 x2 an 3 x3 ... ann xn bn

If: the diagonal elements are

non-zero

Rewrite each equation solving

for the corresponding unknown

ex:
First equation, solve for x1
Second equation, solve for x2

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
Rewriting each equation

x1

c1 a12 x 2 a13 x3 a1n x n

a11

From Equation 1

c2 a21 x1 a23 x3 a2 n xn
a22

From equation 2

x2

xn 1
xn

cn 1 an 1,1 x1 an 1, 2 x2 an 1,n 2 xn 2 an 1,n xn

From equation n-1

an 1,n 1

cn an1 x1 an 2 x2 an ,n 1 xn 1
ann

From equation n

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
General Form of each equation
n

c1 a1 j x j
x1

j 1
j 1

a11

cn 1
xn 1

a 22

j 1
j n 1

n 1, j

xj

an 1,n 1

c n a nj x j

c2 a2 j x j
x2

a
n

j 1
j2

xn

j 1
j n

a nn
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Algorithm
General Form for any row i
n

ci aij x j
xi

j 1
j i

aii

, i 1,2, , n.

How or where can this equation be used?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X]

Important: Remember to use the

most recent value of xi. Which
means to apply values calculated to
the calculations remaining in the
current iteration.

x1
x
2

Use rewritten equations to solve for

each value of xi.

xn -1
xn

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error

a i

new
i

old
i

new
i

100

So when has the answer been found?

The iterations are stopped when the absolute relative
approximate error is less than a prespecified tolerance for all
unknowns.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 1
The upward velocity of a rocket
is given at three different times
Table 1 Velocity vs. Time data.
Time, t s Velocity v m/s
5

106.8

177.2

12

279.2

The velocity data is approximated by a polynomial as:

v t a1t 2 a2t a3 , 5 t 12.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 1
Using a Matrix template of the form

The system of equations becomes

Initial Guess: Assume an initial guess of

t12
2
t2
t32

25
64

t1 1 a1 v1

t 2 1 a2 v2
t3 1 a3 v3

1
1
144 12 1
5
8

a1
a
2
a3

a1 106.8
a 177.2
2

a3 279.2

1
2
5

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 1
Rewriting each equation

25
64

1
8 1
144 12 1
5

a1 106.8
a 177.2
2

a3 279.2

106.8 5a 2 a 3
a1
25
177.2 64a1 a 3
a2
8
279.2 144a1 12a 2
a3
1

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 1
Applying the initial guess and solving for ai
a1
a
2
a3

106.8 5(2) (5)

3.6720
25

a1

177.2 64 3.6720 5
a2
7.8510
8

Initial Guess

279.2 144 3.6720 12 7.8510

a3
155.36
1
When solving for a2, how many of the initial guess values were used?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 1
Finding the absolute relative approximate error

a i

xinew xiold

100
new
xi

a 1

3.6720 1.0000
x100 72.76%
3.6720

7.8510 2.0000

x100 125.47%
7.8510

a 3

At the end of the first iteration

a1
3.6720
a 7.8510
2

a 3
155.36
The maximum absolute
relative approximate error is
125.47%

155.36 5.0000
x100 103.22%
155.36
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 1
Using

a1 3.6720
a 7.8510
2

a3 155.36

Iteration #2
the values of ai are found:

106.8 5 7.8510 155.36

a1
12.056
25

from iteration #1

a2

177.2 6412.056 155.36

54.882
8

279.2 14412.056 12 54.882

a3
798.34
1

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 1
Finding the absolute relative approximate error
12.056 3.6720
At the end of the second iteration
a 1
x100 69.543%
12.056
a 12.056
1

a 2

a 3

54.882 7.8510
x100 85.695%
54.882

798.34 155.36
x100 80.540%
798.34

a 54.882
2

a3 798.54

The maximum absolute

relative approximate error is
85.695%

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 1
Repeating more iterations, the following values are obtained
Iteration

1
2
3
4
5
6

a1
3.6720
12.056
47.182
193.33
800.53
3322.6

a 1 %
72.767
69.543
74.447
75.595
75.850
75.906

a2

a 2 %

a3

7.8510
54.882
255.51
1093.4
4577.2
19049

125.47
85.695
78.521
76.632
76.112
75.972

155.36
798.34
3448.9
14440
60072
249580

a 3 %
103.22
80.540
76.852
76.116
75.963
75.931

Notice The relative errors are not decreasing at any significant rate
Also, the solution is not converging to the true solution of

a1 0.29048
a 19.690
2

a 3 1.0857

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gauss-Siedel method: not all
systems of equations will converge.

Is there a fix?
One class of system of equations always converges: One with a diagonally
dominant coefficient matrix.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:
n

aii aij
j 1
j i

for all i

and

aii aij

for at least one i

j 1
j i

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Pitfall
Diagonally dominant: The coefficient on the diagonal must be at least
equal to the sum of the other coefficients in that row and at least one row
with a diagonal coefficient greater than the sum of the other coefficients
in that row.
Which coefficient matrix is diagonally dominant?
2 5.81 34
A 45 43 1
123 16
1

124 34 56
[B] 23 53 5
96 34 129

Most physical systems do result in simultaneous linear equations that

have diagonally dominant coefficient matrices.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 2
Given the system of equations

12 x1 3 x2 - 5 x3 1

x1 5 x2 3x3 28
3 x1 7 x2 13 x3 76
With an initial guess of

x1
x
2
x3

1
0
1

The coefficient matrix is:

12 3 5
A 1 5 3
3 7 13
Will the solution converge using the
Gauss-Siedel method?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 2
Checking if the coefficient matrix is diagonally dominant
12 3 5
A 1 5 3
3 7 13

a11 12 12 a12 a13 3 5 8

a 22 5 5 a 21 a 23 1 3 4
a33 13 13 a31 a32 3 7 10

The inequalities are all true and at least one row is strictly greater than:
Therefore: The solution should converge using the Gauss-Siedel Method

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 2
Rewriting each equation
12 3 5
1 5 3

3 7 13

With an initial guess of

x1
1
x 0
2

x3
1

a1 1
a 28
2
a3 76

1 3 x 2 5 x3
x1
12
28 x1 3 x3
x2
5

76 3 x1 7 x 2
x3
13

x1

1 3 0 51
0.50000
12

x2

28 0.5 31
4.9000
5

76 3 0.50000 7 4.9000
x3
3.0923
13
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 2
The absolute relative approximate error

0.50000 1.0000
a 1
100 100.00%
0.50000

a
a

4.9000 0

100 100.00%
4.9000
3.0923 1.0000

100 67.662%
3.0923

The maximum absolute relative error after the first iteration is 100%
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 2
After Iteration #1
x1 0.5000
x 4.9000
2

x3 3.0923

Substituting the x values into the

equations

1 3 4.9000 5 3.0923
x1
0.14679
12
x2

28 0.14679 3 3.0923
3.7153
5

x3

76 3 0.14679 7 4.900
3.8118
13

After Iteration #2
x1 0.14679
x 3.7153
2

x3 3.8118

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 2
Iteration #2 absolute relative approximate error
a 1

0.14679 0.50000
100 240.61%
0.14679

3.7153 4.9000

100 31.889%
3.7153

3.8118 3.0923

100 18.874%
3.8118

The maximum absolute relative error after the first iteration is 240.61%

This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 2
Repeating more iterations, the following values are obtained
Iteration

a1

a 1 %

a2

a 2 %

a3

1
2
3
4
5
6

0.50000
0.14679
0.74275
0.94675
0.99177
0.99919

100.00
240.61
80.236
21.546
4.5391
0.74307

4.9000
3.7153
3.1644
3.0281
3.0034
3.0001

100.00
31.889
17.408
4.4996
0.82499
0.10856

3.0923
3.8118
3.9708
3.9971
4.0001
4.0001

a 3 %
67.662
18.876
4.0042
0.65772
0.074383
0.00101

x1 0.99919
The solution obtained x 3.0001 is close to the exact solution of
2

x3 4.0001

x1
x
2
x3

1
3 .
4

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 3
Given the system of equations

3x1 7 x2 13 x3 76

Rewriting the equations

x1 5 x2 3x3 28

76 7 x2 13x3
x1
3

12 x1 3x2 5 x3 1
With an initial guess of

x1
x
2
x3

1
0
1

28 x1 3 x3
x2
5

1 12 x1 3 x 2
x3
5
http://numericalmethods.eng.usf.edu

Gauss-Seidel Method:
Example 3
Conducting six iterations, the following values are obtained
Iteration

a1

1
2
3
4
5
6

21.000
196.15
1995.0
20149
2.0364105
2.0579105

a 1 %

A2

95.238
0.80000
110.71
14.421
109.83
116.02
109.90
1204.6
109.89
12140
109.89 1.2272105

a 2 %

a3

a 3 %

100.00
94.453
112.43
109.63
109.92
109.89

50.680
462.30
4718.1
47636
4.8144105
4.8653106

98.027
110.96
109.80
109.90
109.89
109.89

The values are not converging.

Does this mean that the Gauss-Seidel method cannot be used?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
The Gauss-Seidel Method can still be used
The coefficient matrix is not
diagonally dominant
But this is the same set of
equations used in example #2,
which did converge.

3 7 13
A 1 5 3
12 3 5
12 3 5
A 1 5 3
3 7 13

If a system of linear equations is not diagonally dominant, check to see if

rearranging the equations can form a diagonally dominant matrix.

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Not every system of equations can be rearranged to have a
diagonally dominant coefficient matrix.
Observe the set of equations

x1 x 2 x3 3
2 x1 3 x 2 4 x3 9

x1 7 x 2 x3 9
Which equation(s) prevents this set of equation from having a
diagonally dominant coefficient matrix?

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method
Summary
-Advantages of the Gauss-Seidel Method
-Algorithm for the Gauss-Seidel Method
-Pitfalls of the Gauss-Seidel Method

http://numericalmethods.eng.usf.edu

Gauss-Seidel Method

Questions?

http://numericalmethods.eng.usf.edu

Additional Resources
For all resources on this topic such as digital
audiovisual lectures, primers, textbook chapters,
multiple-choice tests, worksheets in MATLAB,
MATHEMATICA, MathCad and MAPLE, blogs,
related physical problems, please visit
http://numericalmethods.eng.usf.edu/topics/gaus
s_seidel.html

THE END
http://numericalmethods.eng.usf.edu