Professional Documents
Culture Documents
Kostas Kokkotas2
http://www.tat.physik.uni-tuebingen.de/ kokkotas
Kostas Kokkotas3
TOPICS
1. Solving nonlinear equations
2. Solving linear systems of equations
3. Interpolation, approximation & curve fitting
4. Numerical differentiation and numerical integration
5. Numerical Solution of ODEs
6. Boundary and Characteristic Value Problems
7. Mathematical introduction to PDEs
8. Numerical Solution of PDEs
9. Finite Elements Methods
TEXTBOOKS
1. Applied Numerical Analysis C.F. Gerald & P.O.Wheatley,
Addison-Wesley 7th Edition (2004)
2. Numerical Recipes. The Art of Scientific Computing W.H. Press,
S.A.Teukolsky, & W. T. Vetterling, Cambridge University Press
(2007)
Kostas Kokkotas4
Outline II
1. Solving nonlinear equations
I
I
I
I
Bisection method
Linear interpolation
xn+1 = g (xn )
Newtons method
Gaussian elimination
Iterative methods
Matrix inverse
Eigenvalues & Eigenvectors
Nonlinear systems
Interpolating polynomials
Spline Curves
Rational function approximations
Kostas Kokkotas5
Outline III
1. Numerical differentiation and integration
I
I
I
I
I
Numerical differentiation
Trapezoidal rule
Simpson rules
Gaussian quadrature
Multiple integrals
Eulers method
Runge-Kutta methods
Adams method
Prediction-Correction methods
Elliptic equations
Parabolic equations
Hyperbolic equations
Kostas Kokkotas6
(2)
Kostas Kokkotas7
(3)
...
xn = r1 0.4310378790
Bisection method II
If there exists a root in the interval [a0 , b0 ], then f (a0 ) f (b0 ) < 0. If
0 = (a0 + b0 )/2, then:
I
(III) either f (0 ) = 0 .
[0 , b0 ] if (II )
[a1 , b1 ] =
(4)
[a0 , 0 ] if (I )
Kostas Kokkotas9
REPEAT
SET x3 = (x1 + x2 )/2
IF f (x3 ) f (x1 ) < 0
SET x2 = x3
ELSE
SET x1 = x3
ENDIF
UNTIL (|x1 x2 | < E ) OR f (x3 ) = 0
Table: Procedure to find the root of a nonlinear equations using bisection
method
Kostas Kokkotas10
Bisection method IV
CRITISISM
I
Slow convergence
1
|an bn |
2
(5)
n
n1
0
= 2 = = n+1
2
2
2
(6)
(7)
Linear interpolation
Assume that the function is linear in the interval (x1 , x2 ) where f (x1 ) and
f (x2 ) are of opposite sign. Then there will be a straight line connecting
the points (x1 , f (x1 )) and (x2 , f (x2 )) described by the equation:
y (x) = f (x1 ) +
f (x1 ) f (x2 )
(x x1 )
x1 x2
(8)
which crosses the axis Ox let say in a point x3 which will be given by the
following relation
x3 =
x2 f (x1 ) x1 f (x2 )
f (x2 )
= x2
(x2 x1 )
f (x1 ) f (x2 )
f (x2 ) f (x1 )
(9)
Linear interpolation
A possible choice of the new points:
I
(III) If f (x3 ) = 0
Recurrence relation:
xn+2 = xn+1
f (xn+1 )
(xn+1 xn ) .
f (xn+1 ) f (xn )
(10)
Kostas Kokkotas13
Kostas Kokkotas14
The root is not included in the initial choice of the interval (x1 , x2 )
REPEAT
x1
SET x3 = x2 f (x2 ) f (xx22)f
(x1 )
IF |f (x1 )| < |f (x2 ) |
SET x2 = x3
ELSE
SET x1 = x3
ENDIF
UNTIL |f (x3 )| < E
Kostas Kokkotas15
(11)
2n 00
f (r )
2
(12)
f (xn+1 )
(xn+1 xn )
f (xn+1 ) f (xn )
(13)
becomes
r +n+2 = r +n+1
f 00 (r )
1 5
= 1.618 and k m = A = 0
m=
2
2f (r )
n+1 = k 1.618
n
Kostas Kokkotas17
(14)
(15)
Kostas Kokkotas18
Kostas Kokkotas19
xn
xn + 2e xn
.
3
Kostas Kokkotas20
Aitkens improvement
If a method is converging linearly (e.g. n+1 = g 0 (r )n ) then it can be
improved to produce even more accurate results without extra
iterations.
The error the method xn+1 = g 0 (xn ) after n iterations is:
r xn+1 g 0 (r )(r xn )
after n + 1 is:
r xn+2 g 0 (r )(r xn+1 )
and by division we get
r xn+1
g 0 (r )(r xn )
= 0
r xn+2
g (r )(r xn+1 )
Then by solving for r we get:
2
r = xn
(xn xn1 )
xn 2xn1 + xn2
(16)
Newtons method
If near the root r of an equation f (x) = 0 the 1st and 2nd derivatives of
f (x) are continuous we can develop root finding techniques which
converge faster than any method presented till now.
Lets assume that after n iterations we reached a
value xn+1 which is the root of the equation and xn is
a nearby value such as: xn+1 = xn + n . Then:
f (xn+1 )
= f (xn + n )
= f (xn ) + n f 0 (xn ) +
2n 00
f (xn ) +
2
f (xn )
f 0 (xn )
xn+1 = xn
f (xn )
f 0 (xn )
Kostas Kokkotas22
(17)
f (r ) + n f 0 (r ) + 12 2n f 00 (r )
f (r + n )
=
r
+
n
f 0 (r + n )
f 0 (r ) + n f 00 (r )
n
1 + f 00 (r )/f 0 (r )
f 0 (r )
we get the following relation:
n+1 =
f 00 (r ) 2
2f 0 (r ) n
(18)
Kostas Kokkotas23
f (x)f 00 (x)
f (x)
0
g
(x)
=
f 0 (x)
[f 0 (x)]2
(19)
g (r + n ) = g (r ) + g 0 (r )n +
g 00 () 2
2 n
(20)
00
g (r ) +
g () 2
,
2 n
[r , xn ]
(21)
g 00 () 2
2 n
Kostas Kokkotas24
xn+1 r n+1 =
g 00 () 2
2 n
(22)
and f 0 (x) = 2x
Then
xn+1
x2 a
= xn n
2xn
or in a better form
Kostas Kokkotas25
xn+1
1
=
2
a
xn +
. (23)
xn
2n 00
f (xn ) + . . .
2
h
i
n
f (xn ) + n f 0 (xn ) + f 00 (xn ) = 0
2
f (xn )
n = 0
f (xn ) + 2n f 00 (xn )
from the 1st order result we substitute
n
xn+1 = xn
f (xn )
f 0 (xn )
f 00 (xn )f (xn )
2f 0 (xn )
f (xn )
f 0 (xn )
= xn
2f
02
2f (xn ) f 0 (xn )
(24)
(xn ) f 00 (xn ) f (xn )
3n
n+1 =
6 f 0 ()
4 f 0 ()
(25)
Error
0 = 12
1 = 4.8
2 = 1.477
3 = 0.243
4 = 9.15103
Kostas Kokkotas27
xn3 + 3xn Q
3xn2 + Q
Halley
x0 =15
x1 =5.526
x2 =3.16024
x3 =3.00011
x4 =3.0000000...
(26)
Error
0 = 12
1 = 2.5
2 = 0.16
3 = 1.05 104
4 = 3.24 1014
(x r )q(x)
f (x)
=
0
f (x)
mq(x) + (x r )q 0 (x)
n
0 (xn )
{[f 0 (xn )]2 f (xn )f 00 (xn )} /[f 0 (xn )]2
0
f (xn ) f (xn )
= xn 0
(27)
[f (xn )]2 f (xn )f 00 (xn )
= xn
xn2 2
4xn
(A)
Modified Newton-Raphson
xn+1 = xn
x
x0
x1
x2
x3
(A)
1.5
1.458333333
1.436607143
1.425497619
(xn2 2)xn
xn2 + 2
Error (A)
8.6102
4.4102
2.2102
1.1102
Kostas Kokkotas29
(B).
(B)
1.5
1.411764706
1.414211439
1.414213562
Error (B)
8.6102
-2.4103
-2.1106
-1.61012
Convergence tests
We will compare the convergence rate of a method with linear and one
with quadratic convergence (Bisection and Newton)
For linear convergence we have::
lim
|n+1 |
= a |n | a|n1 | a2 |n2 | ... an |0 |
|n |
log10 |n | log10 |0 |
log10 |a|
(28)
n+1
n+1
|
n+1 |
= b |
n | b|
n1 |2 b 3 |
n2 |4 ... b 2 1 |
0 |2
|
n |2
Kostas Kokkotas30
log10 |
n | + log10 |b|
log10 |
0 | + log10 |b|
(29)
iterations
= e x 3y 1
(30)
g (x, y )
= x +y 4
Kostas Kokkotas32
Kostas Kokkotas33
g
f
y + g y
g f
f g
x y x y
and n =
g
x
g f
x y
f
g x
+f
f g
x y
(31)
= xn
yn+1
= yn
Kostas Kokkotas34
f gy g fy
fx gy gx fy
g fx f gx
fx gy gx fy
(32)
n
(33)
n
Kostas Kokkotas35
1
N
0
, ..., xn+1
)
= fN (xn+1
fN (xn1
(34)
xnN )
+
+
fN
fN
fN + 1 xn1 + ... + N xnN
x
x
Then the quantities xni will be found as solutions of the linear system
f1 f1
f1
1
f1
x
n
x 1
x 2
x N
..
..
..
..
.
.
(35)
.
= .
fN
fN
fN
fN
xnN
x
N
x 1
x 2
Kostas Kokkotas36
Thus if we start with N initial values (x01 , x02 , ..., x0N ) then from the
solution of the above system we will calculate the quantities xni which
lead to the new values (x11 , x12 , ..., x1N ) via the relations:
x11
= x01 + x01
.. ..
..
x1N
= x0N + x0N
(36)
This procedure will be repeated till the a given accuracy is reached i.e.
max |xni | < E .
Kostas Kokkotas37
fN (x1 , x2 , . . . , xN )
(37)
(38)
= FN (x1 , x2 , . . . , xN )
Kostas Kokkotas39
and e x 3y = 1
and yn+1 =
1 xn
(e 1)
3
for an initial choice (1, 0), we create the following sequence of solutions
n
x
y
0
-1
0
1
-2
-0.2107
2
-1.9884
-0.2882
3
-1.9791
-0.2877
4
-1.9792
-0.2873
5
-1.9793
-0.2873
Kostas Kokkotas40
0
-1
0
1
-2
-0.2882
2
-1.9791
-0.2873
3
-1.9793
-0.2873
i.e. the same accuracy has been achieved with only 3 iterations.
In order to find the 2nd solution of the system we modify the recurrence
relations as follows:
xn+1 =
4 yn2
and yn+1 =
1 xn
(e 1)
3
If we start with values close to the exact solution e.g. x0 = 1.5 and
y0 = 1 the results are :
n
x
y
0
1.5
1
1
1.7321
1.5507
2
1.2630
0.8453
3
1.8126
1.7087
4
1.0394
0.6092
5
1.9050
1.9064
we notice that we diverge from the solution and the reason is that the
criteria for convergence set earlier are not satisfied.
Kostas Kokkotas41
ln(1 + 3yn )
Kostas Kokkotas42
Selected problems I
1. Find the eigenvalues of the differential equation y 00 + 2 y = 0
with the following boundary conditions y (0) = 0 and y (1) = y 0 (1).
2. Find the inverse of a number without using division
Kostas Kokkotas43
and xy e x = 0
and f2 (x, y ) = xy 1 = 0 .
It is obvious that the solutions are (1, 1) and (1, 1), examine the
difficulties that might arise if we try to use Newtons method to find
the solution.
Kostas Kokkotas44