Computationalnotes Final

Lectures on Computational Physics
Badis Ydri
Adel Bouchareb Rak Chemam
Physics Department, Badji Mokhtar University, Annaba, Algeria
May 21, 2013
2 ydri et al, lectures on computational physics
Contents
1 Introduction and References 1
2 Euler Algorithm 3
2.1 Euler Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 First Example and Sample Code . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.1 Radioactive Decay . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.2 A Sample Fortran Code . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 More Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3.1 Air Resistance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3.2 Projectile Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4 Periodic Motions and Euler-Cromer and Verlet Algorithms . . . . . . . . . 11
2.4.1 Harmonic Oscillator . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4.2 Euler Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4.3 Euler-Cromer Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4.4 Verlet Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5 Lab Problem 1: Euler Algorithm- Air Resistance . . . . . . . . . . . . . . 14
2.6 Lab Problem 2: Euler Algorithm- Projectile Motion . . . . . . . . . . . . 15
2.7 Lab Problem 3: Euler, Euler-Cromer and Verlet Algorithms . . . . . . . . 16
3 Numerical Integration 19
3.1 Rectangular Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2 Trapezoidal Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3 Parabolic Approximation or Simpsons Rule . . . . . . . . . . . . . . . . . 20
3.4 Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.5 Lab Problem 4: Numerical Integration . . . . . . . . . . . . . . . . . . . . 23
4 Newton-Raphson Algorithms and Interpolation 25
4.1 Bisection Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2 Newton-Raphson Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.3 Hybrid Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.4 Lagrange Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.5 Cubic Spline Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.6 Lab Problem 5: Newton-Raphson Algorithm . . . . . . . . . . . . . . . . . 31
5 The Solar System-The Runge-Kutta Methods 33
5.1 The Solar System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.1.1 Newtons Second Law . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.1.2 Astronomical Units and Initial Conditions . . . . . . . . . . . . . . 34
5.1.3 Keplers Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.1.4 The inverse-Square Law and Stability of Orbits . . . . . . . . . . . 37
5.2 Euler-Cromer Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.3 The Runge-Kutta Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.3.1 The Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.3.2 Example 1: The Harmonic Oscillator . . . . . . . . . . . . . . . . . 40
5.3.3 Example 2: The Solar System . . . . . . . . . . . . . . . . . . . . . 40
5.4 Precession of the Perihelion of Mercury . . . . . . . . . . . . . . . . . . . . 42
5.5 Lab Problem 6: Runge-Kutta Algorithm- The Solar System . . . . . . . . 43
5.6 Lab Problem 7: Precession of the perihelion of Mercury . . . . . . . . . . 44
6 Chaos: Chaotic Pendulum 47
6.1 Equation of Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.2 Numerical Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
6.2.1 Euler-Cromer Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 50
6.2.2 Runge-Kutta Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 50
6.3 Elements of Chaos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
6.3.1 Buttery Eect: Sensitivity to Initial Conditions . . . . . . . . . . 51
6.3.2 Poincare Section and Attractors . . . . . . . . . . . . . . . . . . . . 52
6.3.3 Period-Doubling Bifurcations . . . . . . . . . . . . . . . . . . . . . 52
6.3.4 Feigenbaum Ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
6.3.5 Spontaneous Symmetry Breaking . . . . . . . . . . . . . . . . . . . 53
6.4 Lab Problem 8: The Buttery Eect . . . . . . . . . . . . . . . . . . . . . 54
6.5 Lab Problem 9: Poincar Sections . . . . . . . . . . . . . . . . . . . . . . . 55
6.6 Lab Problem 10: Period Doubling . . . . . . . . . . . . . . . . . . . . . . . 56
6.7 Lab Problem 11: Bifurcation Diagrams . . . . . . . . . . . . . . . . . . . . 57
7 Molecular Dynamics 59
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
7.2 The Lennard-Jones Potential . . . . . . . . . . . . . . . . . . . . . . . . . 60
7.3 Units, Boundary Conditions and Verlet Algorithm . . . . . . . . . . . . . 61
7.4 Some Physical Applications . . . . . . . . . . . . . . . . . . . . . . . . . . 63
7.4.1 Dilute Gas and Maxwell Distribution . . . . . . . . . . . . . . . . . 63
7.4.2 The Melting Transition . . . . . . . . . . . . . . . . . . . . . . . . 64
7.5 Lab Problem 12: Maxwell Distribution . . . . . . . . . . . . . . . . . . . . 64
7.6 Lab Problem 13: Melting Transition . . . . . . . . . . . . . . . . . . . . . 65
ydri et al, lectures computational physics 5
8 Pseudo Random Numbers and Random Walks 67
8.1 Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
8.1.1 Linear Congruent or Power Residue Method . . . . . . . . . . . . . 67
8.1.2 Statistical Tests of Randomness . . . . . . . . . . . . . . . . . . . . 68
8.2 Random Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
8.2.1 Random Walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
8.2.2 Diusion Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
8.3 The Random Number Generators RAN 0, 1, 2 . . . . . . . . . . . . . . . . 74
8.4 Lab Problem 14: Random Numbers . . . . . . . . . . . . . . . . . . . . . . 77
8.5 Lab Problem 15: Random Walks . . . . . . . . . . . . . . . . . . . . . . . 78
9 Monte Carlo Integration 79
9.1 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
9.1.1 Rectangular Approximation Revisted . . . . . . . . . . . . . . . . . 79
9.1.2 Midpoint Approximation of Multidimensional Integrals . . . . . . . 80
9.1.3 Spheres and Balls in d Dimensions . . . . . . . . . . . . . . . . . . 82
9.2 Monte Carlo Integration: Simple Sampling . . . . . . . . . . . . . . . . . . 83
9.2.1 Sampling (Hit or Miss) Method . . . . . . . . . . . . . . . . . . . . 83
9.2.2 Sample Mean Method . . . . . . . . . . . . . . . . . . . . . . . . . 84
9.2.3 Sample Mean Method in Higher Dimensions . . . . . . . . . . . . . 84
9.3 The Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 85
9.4 Monte Carlo Errors and Standard Deviation . . . . . . . . . . . . . . . . . 86
9.5 Nonuniform Probability Distributions . . . . . . . . . . . . . . . . . . . . . 89
9.5.1 The Inverse Transform Method . . . . . . . . . . . . . . . . . . . . 89
9.5.2 The Acceptance-Rejection Method . . . . . . . . . . . . . . . . . . 91
9.6 Lab Problem 16: Midpoint and Monte Carlo Approximations . . . . . . . 91
9.7 Lab Problem 17: Nonuniform Probability Distributions . . . . . . . . . . . 92
10 Monte Carlo Importance Sampling, Metropolis Algorithm and Ising
Model 95
10.1 The Canonical Ensemble . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
10.2 Importance Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
10.3 The Ising Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
10.4 The Metropolis Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
10.5 The Heat-Bath Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
10.6 The Mean Field Approximation . . . . . . . . . . . . . . . . . . . . . . . . 101
10.6.1 Phase Diagram and Critical Temperature . . . . . . . . . . . . . . 101
10.6.2 Critical Exponents . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
10.7 Simulation of The Ising Model and Numerical Results . . . . . . . . . . . 105
10.7.1 The Fortran Code . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
10.7.2 Some Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . 106
10.8 Lab Problem 18: The Metropolis Algorithm and The Ising Model . . . . . 109
10.9 Lab Problem 19: The Ferromagnetic Second Order Phase Transition . . . 110
10.10Lab Problem 20: The 2Point Correlator . . . . . . . . . . . . . . . . . . 111
10.11Lab Problem 21: Hysteresis and The First Order Phase Transition . . . . 111
Appendix 115
A Notes on Numerical Errors 115
A.1 Floating Point Representation . . . . . . . . . . . . . . . . . . . . . . . . . 115
A.2 Machine Precision and Roundo Errors . . . . . . . . . . . . . . . . . . . 117
A.3 Systematic (Algorithmic) Errors . . . . . . . . . . . . . . . . . . . . . . . . 118
B The Jackknife Method and The Method of Least Squares 121
C Exercises 123
1
Introduction and References
Computational physics is a subeld of computational science/scientic computing.
In computational physics we combine elements from physics (especially theoretical),
elements from mathematics (in particular applied mathematics such as numerical
analysis) and elements from computer science (programming) for the purpose of
solving a physics problem.
In physics there are traditionally two approaches. 1) The experimental approach
and 2) The theoretical approach. Nowadays we consider The computational ap-
proach as a third approach in physics. It can even be argued that the computa-
tional approach is independent from the rst two approaches and it is not just a
bridge between them.
The most important use of computers in physics is simulation. Simulations are
suited for nonlinear problems which can not generally solved by analytical methods.
The starting point of a simulation is an idealized model of a physical system of
interest. We want to check whether or not the behaviour of this model is consistent
with observation. We specify an algorithm for the implementation of the model on
a computer. The execution of this implementation is a simulation. Simulations are
therefore virtual experiments. The comparison between simulations and laboratory
experiments goes as follows:
Laboratory experiment Simulation
sample model
physical apparatus computer program (the
code)
calibration testing of code
measurement computation
data analysis data analysis
A crucial tool in computational physics is programming languages. In simulations
as used by the majority of research physicists codes are written in a high-level
compiled language such as Fortran and C/C++. In such simulations we also use
calls to routine libraries such as LAPACK. In this course we follow a dierent path
by writing all our codes in a high-level compiled language and not call any libraries.
The use of mathematical software packages such as MAPLE, MATHEMATICA
and MATLAB is only suited for relatively small calculations. These packages are
interpreted languages and thus the code they produce run generally far too slowly
compared to compiled languages.
As our programming language we will use Fortran 77 under the Linux operating
system. We adopt the ubuntu distribution of Linux. We will use the Fortran
compilers f77 and gfortran. As an editor we will use emacs and for graphics we will
use gnuplot.
The main references which we will follow in this course are:
N.J.Giordano, Computational Physics.
R.H.Landau, M.J.Paez, C.C.Bordeianu, Computational Physics.
H.Gould, J.Tobochnick, D.Christian, An Introduction To Computer Simula-
tion Methods.
R.Fitzpatrick, Computational Physics.
M. Hjorth-Jensen,Computational Physics.
Paul L.DeVries, A First Course in Computational Physics.
2
Euler Algorithm
2.1 Euler Algorithm
It is a well appreciated fact that rst order dierential equations are commonplace in
all branches of physics. They appear virtually everywhere and some of the most funda-
mental problems of nature obey simple rst order dierential equations or second order
dierential equations. It is so often possible to recast second order dierential equations
as rst order dierential equations with a doubled number of unknown. From the numer-
ical standpoint the problem of solving rst order dierential equations is a conceptually
simple one as we will now explain.
We consider the general rst order ordinary dierential equation
y
=
dy
dx
= f(x, y). (2.1)
We impose the general initial-value boundary condition is
y(x
0
) = y
0
. (2.2)
We solve for the function y = y(x) in the unit xinterval starting from x
0
. We make the
xinterval discretization
x
n
= x
0
+nx , n = 0, 1, ... (2.3)
The Euler algorithm is one of the oldest known numerical recipe. It consists in replacing
the function y(x) in the interval [x
n
, x
n+1
] by the straight line connecting the points
(x
n
, y
n
) and (x
n+1
, y
n+1
). This comes from the denition of the derivative at the point
x = x
n
given by
y
n+1
y
n
x
n+1
x
n
= f(x
n
, y
n
). (2.4)
This means that we replace the above rst order dierential equation by the nite dif-
ference equation
y
n+1
y
n
+ xf(x
n
, y
n
). (2.5)
This is only an approximation. The truncation error is given by the next term in the
Taylors expansion of the function y(x) which is given by
y
n+1
y
n
+ xf(x
n
, y
n
) +
1
2
x
2
df(x, y)
dx
|
x=xn
+.... (2.6)
The error then reads
1
2
(x)
2
df(x, y)
dx
|
x=xn
. (2.7)
The error per step is therefore proportional to (x)
2
. In a unit interval we will perform
N = 1/x steps. The total systematic error is therefore proportional to
N(x)
2
=
1
N
. (2.8)
2.2 First Example and Sample Code
2.2.1 Radioactive Decay
It is an experimental fact that radioactive decay obeys a very simple rst order dierential
equation. In a spontaneous radioactive decay a particle with no external inuence will
decay into other particles. A typical example is the nuclear isotope uranium 235. The
exact moment of decay of any one particle is random. This means that the number
dN(t) = N(t) N(t +dt) of nuclei which will decay during a time inetrval dt must be
proportional to dt and to the number N(t) of particles present at time t, i.e.
dN(t) N(t)dt. (2.9)
In other words the probability of decay per unit time given by (dN(t)/N(t))/dt is a
constant which we denote 1/. The minus sign is due to the fact that dN(t) is negative
since the number of particles decreases with time. We write
dN(t)
dt
=
N(t)
. (2.10)
The solution of this rst order dierential equation is given by a simple exponential
function, viz
N(t) = N
0
exp(t/). (2.11)
The number N
0
is the number of particles at time t = 0. The time is called the mean
lifetime. It is the average time for decay. For the uranium 235 the mean lifetime is
around 10
9
years.
The goal now is to obtain an approximate numerical solution to the problem of
radioactivity using the Euler algorithm. In this particular case we can compare to an
exact solution given by the exponential decay law (2.11). We start evidently from the
Taylors expansion
N(t + t) = N(t) + t
dN
dt
+
1
2
(t)
2
d
2
N
dt
2
+... (2.12)
We get in the limit t 0
dN
dt
= Lim
t0
N(t + t) N(t)
t
. (2.13)
We take t small but non zero. In this case we obtain the approximation
dN
dt

N(t + t) N(t)
t
. (2.14)
Equivalently
N(t + t) N(t) + t
dN
dt
. (2.15)
By using (2.10) we get
N(t + t) N(t) t
N(t)
. (2.16)
We will start from the number of particles at time t = 0 given by N(0) = N
0
which is
known. We substitute t = 0 in (2.16) to obtain N(t) = N(1) as a function of N(0).
Next the value N(1) can be used in equation (2.16) to get N(2t) = N(2), etc. We are
thus led to the time discretization
t t(i) = it , i = 0, ..., N. (2.17)
In other words
N(t) = N(i). (2.18)
The integer N determine the total time interval T = Nt. The numerical solution (2.16)
can be rewritten as
N(i + 1) = N(i) t
N(i)
, i = 0, ..., N. (2.19)
This is Euler algorithm for radioactive decay. For convenience we shift the integer i so
that the above equation takes the form
N(i) = N(i 1) t
N(i 1)
, i = 1, ..., N + 1. (2.20)
We introduce

N(i) = N(i 1), i.e

N(1) = N(0) = N
0
. We get
N(i + 1) =

N(i) t
N(i)
, i = 1, ..., N + 1. (2.21)
The corresponding times are
t(i + 1) = it , i = 1, ..., N + 1. (2.22)

The initial number of particles at time

t(1) = 0 is

N(1) = N
0
. This approximate solution
should be compared with the exact solution (2.11).
2.2.2 A Sample Fortran Code
The goal in this section is to provide a sample Fortran code which implements the above
algorithm (2.21). The reasons behind choosing Fortran were explained in the introduc-
tion. Any Fortran program, like any other programing language, must start with some
program statement and conclude with an end statement. The program statement allows
us to give a name to the program. The end statement may be preceded by a return
statement. This looks like
program radioactivity
c Here is the code
return
end
We have chosen the name radioactivity for our program. The c in the second line
indicates that the sentence here is the code is only a comment and not a part of the
code.
After the program statement come the declaration statements. We state the variables
and their types which are used in the program. In Fortran we have the integer type for
integer variables and the double precision type for real variables. In the case of (2.21) the
variables

N(i),

t(i), , t, N
0
are real numbers while the variables i and N are integer
numbers.
An array A of dimension K is an ordered list of K variables of a given type called
the elements of the array and denoted A(1), A(2),...,A(K). In our above example

N(i)
and

t(i) are real arrays of dimension N + 1. We declare that

N(i) and

t(i) are real for
all i = 1, ..., N + 1 by writing

N(1 : N + 1) and

t(1 : N + 1).
Since an array is declared at the begining of the program it must have a xed size. In
other words the upper limit must be a constant and not a variable. In Fortran a constant
is declared with a parameter statement. In our above case the upper limit is N + 1 and
hence N must be declared in parameter statement.
In the Fortran code we choose to use the notation A =

N, A0 =

N
0
, time =

t, = t
and tau = . By putting all declarations together we get the following preliminary lines
of code
integer i,N
parameter (N=100)
doubleprecision A(1:N+1),A0,time(1:N+1),Delta,tau
c Here is the code
return
end
The input of the computation in our case are obviously given by the parameters N
0
,
, t and N.
For the radioactivity problem the main part of the code consists of equations (2.21)
and (2.22). We start with the known quantities

N(1) = N
0
at

t(1) = 0 and generate via
the successive use of (2.21) and (2.22)

N(i) and

t(i) for all i > 1. This will be coded
using a do loop. It begins with a do statement and ends with an enddo statement. We
may also indicate a step size.
The output of the computation can be saved to a le using a write statement inside
the do loop. In our case the output is the number of particles

N(i) and the time

t(i).
The write statement reads explicitly
write(10, )

t(i),

N(i).
The data will then be saved to a le called fort.10.
By including the initialization, the do loop and the write statement we obtain the
complete code
integer i,N
parameter (N=100)
doubleprecision A(1:N+1),A0,time(1:N+1),Delta,tau
parameter (A0=1000,Delta=0.01d0,tau=1.0d0)
A(1)=A0
time(1)=0
do i=1,N+1,1
A(i+1)=A(i)-Delta*A(i)/tau
time(i+1)=i*Delta
write(10,*) time(i+1),A(i+1)
enddo
return
end
2.3 More Examples
2.3.1 Air Resistance
We consider an athlete riding a bicycle moving on a at terrain. The goal is to determine
the velocity. Newtons second law is given by
m
dv
dt
= F. (2.23)
F is the force exerted by the athlete on the bicycle. It is clearly very dicult to write down
a precise expression for F. Formulating the problem in terms of the power generated
by the athlete will avoid the use of an explicit formula for F. Multiplying the above
equation by v we obtain
dE
dt
= P. (2.24)
E is the kinetic energy and P is the power, viz
E =
1
2
mv
2
, P = Fv. (2.25)
Experimentaly we nd that the output of well trained athletes is around P = 400 watts
over periods of 1h. The above equation can also be rewritten as
dv
2
dt
=
2P
m
. (2.26)
For P constant we get the solution
v
2
=
2P
m
t +v
2
0
. (2.27)
We remark the unphysical eect that v as t . This is due to the absence of
the eect of friction and in particular air resistance.
The most important form of friction is air resistance. The force due to air resistance
(the drag force) is
F
drag
= B
1
v B
2
v
2
. (2.28)
At small velocities the rst term dominates whereas at large velocities it is the second
term that dominates. For very small velocities the dependence on v given by F
drag
=
B
1
v is known as Stockes law. For reasonable velocities the drag force is dominated by
the second term, i.e. it is given for most objects by
F
drag
= B
2
v
2
. (2.29)
The coecient B
2
can be calculated as follows. As the bicycle-rider combination moves
with velocity v it pushes in a time dt a mass of air given by dm
air
= Avdt where is
the air density and A is the frontal cross section. The corresponding kinetic energy is
dE
air
= dm
air
v
2
/2. (2.30)
This is equal to the work done by the drag force, i.e.
F
drag
vdt = dE
air
. (2.31)
From this we get
B
2
= CA. (2.32)
The drag coecient is C =
1
2
. The drag force becomes
F
drag
= CAv
2
. (2.33)
Taking into account the force due to air resistance we nd that Newtons law becomes
m
dv
dt
= F +F
drag
. (2.34)
Equivalently
dv
dt
=
P
mv

CAv
2
m
. (2.35)
It is not obvious that this equation can be solved exactly in any easy way. The Euler
algorithm gives the approximate solution
v(i + 1) = v(i) + t
dv
dt
(i). (2.36)
In other words
v(i + 1) = v(i) + t
_
P
mv(i)

CAv
2
(i)
m
_
, i = 0, ..., N. (2.37)
This can also be put in the form (with v(i) = v(i 1))
v(i + 1) = v(i) + t
_
P
m v(i)

CA v
2
(i)
m
_
, i = 1, ..., N + 1. (2.38)
The corresponding times are
t
t(i + 1) = it , i = 1, ..., N + 1. (2.39)

The initial velocity v(1) at time t(1) = 0 is known.
2.3.2 Projectile Motion
There are two forces acting on the projectile. The weight force and the drag force. The
drag force is opposite to the velocity. In this case Newtons law is given by
m
dv
dt
=

F +

F
drag
= mg B
2
v
2
v
v
= mg B
2
vv. (2.40)
The goal is to determine the position of the projectile and hence one must solve the two
equations
dx
dt
= v. (2.41)
m
dv
dt
= mg B
2
vv. (2.42)
In components (the horizontal axis is x and the vertical axis is y) we have 4 equations
of motion given by
dx
dt
= v
x
. (2.43)
m
dv
x
dt
= B
2
vv
x
. (2.44)
dy
dt
= v
y
. (2.45)
m
dv
y
dt
= mg B
2
vv
y
. (2.46)
We recall the constraint
v =
_
v
2
x
+v
2
y
. (2.47)
The numerical approach we will employ in order to solve the 4 equations of motion (2.43)-
(2.46) together with (2.47) consists in using Euler algorithm. This yields the approximate
solution given by the equations
x(i + 1) = x(i) + tv
x
(i). (2.48)
v
x
(i + 1) = v
x
(i) t
B
2
v(i)v
x
(i)
m
. (2.49)
y(i + 1) = y(i) + tv
y
(i). (2.50)
v
y
(i + 1) = v
y
(i) tg t
B
2
v(i)v
y
(i)
m
. (2.51)
The constraint is
v(i) =
_
v
x
(i)
2
+v
y
(i)
2
. (2.52)
In the above equations the index i is such that i = 0, ..., N. The initial position and
velocity are given, i.e. x(0), y(0), v
x
(0) and v
y
(0) are known.
2.4 Periodic Motions and Euler-Cromer and Verlet Algo-
rithms
As discussed above at each iteration using the Euler algorithm there is a systematic error
proportional to 1/N. Obviously this error will accumulate and may become so large that
it will alter the solution drastically at later times. In the particular case of periodic
motions, where the true nature of the motion can only become clear after few elapsed
periods, the large accumulated error can lead to diverging results. In this section we will
discuss simple variants of the Euler algorithm which perform much better than the plain
Euler algorithm for periodic motions.
2.4.1 Harmonic Oscillator
We consider a simple pendulum: a particle of mass m suspended by a massless string
from a rigid support. There are two forces acting on the particle. The weight and the
tension of the string. Newtons second law reads
m
d
2
s
dt
= mg +

T. (2.53)
The parallel (with respect to the string) projection reads
0 = mg cos +T. (2.54)
The perpendicular projection reads
m
d
2
s
dt
2
= mg sin . (2.55)
The is the angle that the string makes with the vertical. Clearly s = l. The force
mg sin is a restoring force which means that it is always directed toward the equilibrium
position (here = 0) opposite to the displacement and hence the minus sign in the above
equation. We get by using s = l the equation
d
2
dt
2
=
g
l
sin . (2.56)
For small we have sin . We obtain
d
2
dt
2
=
g
l
. (2.57)
The solution is a sinusoidal function of time with frequency =
_
g/l. It is given by
(t) =
0
sin(t +). (2.58)
The constants
0
and depend on the initial displacement and velocity of the pendulum.
The frequency is independent of the mass m and the amplitude of the motion and depends
only on the length l of the string.
2.4.2 Euler Algorithm
The numerical solution is based on Euler algorithm. It is found as follows. First we
replace the equation of motion (2.57) by the following two equations
d
dt
= . (2.59)
d
dt
=
g
l
. (2.60)
We use the denition of a derivative of a function, viz
df
dt
=
f(t + t) f(t)
t
, t 0. (2.61)
We get for small but non zero t the approximations
(t + t) (t) +(t)t
(t + t) (t)
g
l
(t)t. (2.62)
We consider the time discretization
t t(i) = it , i = 0, ..., N. (2.63)
In other words
(t) = (i) , (t) = (i). (2.64)
The integer N determine the total time interval T = Nt. The above numerical solution
can be rewritten as
(i + 1) = (i)
g
l
(i)t
(i + 1) = (i) +(i)t. (2.65)
We shift the integer i such that it takes values in the range [1, N + 1]. We obtain
(i) = (i 1)
g
l
(i 1)t
(i) = (i 1) +(i 1)t. (2.66)
We introduce (i) = (i 1) and

(i) = (i 1). We get with i = 1, ..., N + 1 the
equations
(i + 1) = (i)
g
l
(i)t
(i + 1) =

(i) + (i)t. (2.67)
By using the values of and at time i we calculate the corresponding values at time
i + 1. The initial angle and angular velocity

(1) = (0) and (1) = (0) are known.
This process will be repeated until the functions and are determined for all times.
2.4.3 Euler-Cromer Algorithm
As it turns out the above Euler algorithm does not conserve energy. In fact Eulers
method is not good for all oscillatory systems. A simple modication of Eulers algorithm
due to Cromer will solve this problem of energy non conservation. This goes as follows.
We use the values of the angle

(i) and the angular velocity (i) at time step i to calculate
the angular velocity (i +1) at time step i +1. This step is the same as before. However
we use

(i) and (i + 1) (and not (i)) to calculate

(i + 1) at time step i + 1. This
procedure as shown by Cromers will conserve energy in oscillatory problems. In other
words equations (2.67) become
(i + 1) = (i)
g
l
(i)t
(i + 1) =

(i) + (i + 1)t. (2.68)
The error can be computed as follows. From these two equations we get
(i + 1) =

(i) + (i)t
g
l
(i)t
2
=

(i) + (i)t +
d
2
dt
|
i
t
2
. (2.69)
In other words the error per step is still of the order of t
2
. However the Euler-Cromer
algorithm does better than Euler algorithm with periodic motion. Indeed at each step i
the energy conservation condition reads
E
i+1
= E
i
+
g
2l
(
2
i

g
l

2
i
)t
2
. (2.70)
The energy of the simple pendulum is of course by
E
i
=
1
2
2
i
+
g
2l
2
i
. (2.71)
The error at each step is still proportional to t
2
as in the Euler algorithm. However
the coecient is precisely equal to the dierence between the values of the kinetic energy
and the potential energy at the step i. Thus the accumulated error which is obtained by
summing over all steps vanishes since the average kinetic energy is equal to the average
potential energy. In the Euler algorithm the coecient is actually equal to the sum of
the kinetic and potential energies and as consequence no cancellation can occur.
2.4.4 Verlet Algorithm
Another method which is much more accurate and thus very suited to periodic motions
is due to Verlet. Let us consider the forward and backward Taylor expansions
(t
i
+ t) = (t
i
) + t
d
dt
|
t
i
+
1
2
(t)
2
d
2
dt
2
|
t
i
+
1
6
(t)
3
d
3
dt
3
|
t
i
+... (2.72)
(t
i
t) = (t
i
) t
d
dt
|
t
i
+
1
2
(t)
2
d
2
dt
2
|
t
i

1
6
(t)
3
d
3
dt
3
|
t
i
+... (2.73)
Adding these expressions we get
(t
i
+ t) = 2(t
i
) (t
i
t) + (t)
2
d
2
dt
2
|
t
i
+O(
4
). (2.74)
We write this as
i+1
= 2
i
i1
g
l
(t)
2
i
. (2.75)
This is the Verlet algorithm for the harmonic oscillator. First we remark that the error
is proportional to t
4
which is less than the errors in the Euler, Euler-Cromer (and even
less than the error in the second-order Runge-Kutta) methods so this method is much
more accurate. Secondly in this method we do not need to calculate the angular velocity
= d/dt. Thirdly this method is not self-starting. In other words given the initial
conditions
1
and
1
we need also to know
2
for the algorithm to start. We can for
example determine
2
using the Euler method, viz
2
=
1
+ t
1
.
2.5 Lab Problem 1: Euler Algorithm- Air Resistance
The equation of motion of a cyclist exerting a force on his bicycle corresponding to a
constant power P and moving against the force of air resistance is given by
dv
dt
=
P
mv

CAv
2
m
.
The numerical approximation of this rst order dierential equation which we will con-
sider in this problem is based on Euler algorithm.
(1) Calculate the speed v as a function of time in the case of zero air resistance and
then in the case of non-vanishing air resistance. What do you observe. We will
take P = 200 and C = 0.5. We also give the values
m = 70kg , A = 0.33m
2
, = 1.2kg/m
3
, t = 0.1s , T = 200s.
The initial speed is
v(1) = 4m/s ,

t(1) = 0.
(2) What do you observe if we change the drag coecient and/or the power. What do
you observe if we decrease the time step.
2.6 Lab Problem 2: Euler Algorithm- Projectile Motion
The numerical approximation based on the Euler algorithm of the equations of motion
of a projectile moving under the eect of the forces of gravity and air resistance is given
by the equations
v
x
(i + 1) = v
x
(i) t
B
2
v(i)v
x
(i)
m
.
v
y
(i + 1) = v
y
(i) tg t
B
2
v(i)v
y
(i)
m
.
v(i + 1) =
_
v
2
x
(i + 1) +v
2
y
(i + 1).
x(i + 1) = x(i) + t v
x
(i).
y(i + 1) = y(i) + t v
y
(i).
(1) Write a Fortran code which implements the above Euler algorithm.
(2) We take the values
B
2
m
= 0.00004m
1
, g = 9.8m/s
2
.
v(1) = 700m/s , = 30 degree.
v
x
(1) = v(1) cos , v
y
(1) = v(1) sin .
N = 10
5
, t = 0.01s.
Calculate the trajectory of the projectile with and without air resistance. What do
you observe.
(3) We can determine numerically the range of the projectile by means of the condi-
tional instruction if. This can be done by adding inside the do loop the following
condition
if (y(i + 1).le.0) exit
Determine the range of the projectile with and without air resistance.
(4) In the case where air resistance is absent we know that the range is maximal when
the initial angle is 45 degrees. Verify this fact numerically by considering several
angles. More precisely add a do loop over the initial angle in order to be able to
study the range as a function of the initial angle.
(5) In the case where air resistance is non zero calculate the angle for which the range
is maximal.
2.7 Lab Problem 3: Euler, Euler-Cromer and Verlet Algo-
rithms
We will consider the numerical solutions of the equation of motion of a simple harmonic
oscillator given by the Euler, Euler-Cromer and Verlet algorithms which take the form
i+1
=
i
g
l

i
t ,
i+1
=
i
+
i
t , Euler.
i+1
=
i
g
l
i
t ,
i+1
=
i
+
i+1
t , Euler Cromer.
i+1
= 2
i
i1
g
l
i
(t)
2
, Verlet.
(1) Write a Fortran code which implements the Euler, Euler-Cromer and Verlet algo-
rithms for the harmonic oscillator problem.
(2) Calculate the angle, the angular velocity and the energy of the harmonic oscillator
as functions of time. The energy of the harmonic oscillator is given by
E =
1
2
2
+
1
2
g
l

2
.
We take the values
g = 9.8m/s
2
, l = 1m .
We take the number of iterations N and the time step t to be
N = 10000 , t = 0.05s.
The initial angle and the angular velocity are given by
1
= 0.1 radian ,
1
= 0.
By using the conditional instruction if we can limit the total time of motion to be
equal to say 5 periods as follows
if (t(i + 1).ge.5 period) exit.
(3) Compare between the value of the energy calculated with the Euler method and
the value of the energy calculated with the Euler-Cromer method. What do you
observe and what do you conclude.
(4) Repeat the computation using the Verlet algorithm. Remark that this method can
not self-start from the initial values
1
and
1
only. We must also provide the angle
2
which can be calculated using for example Euler, viz
2
=
1
+
1
t.
We also remark that the Verlet algorithm does not require the calculation of the
angular velocity. However in order to calculate the energy we need to evaluate the
angular velocity which can be obtained from the expression
i
=

i+1
i1
2t
.
3
Numerical Integration
3.1 Rectangular Approximation
We consider a generic one dimensional integral of the form
F =
_
b
a
f(x)dx. (3.1)
In general this can not be done analytically. However this integral is straightforward to
do numerically. The starting point is Riemann denition of the integral F as the area
under the curve of the function f(x) from x = a to x = b. This is obtained as follows.
We discretize the xinterval so that we end up with N equal small intervals of lenght
x, viz
x
n
= x
0
+nx , x =
b a
N
(3.2)
Clearly x
0
= a and x
N
= b. Riemann denition is then given by the following limit
F = lim
_
x0 , N , ba=xed
_
_
x
N1
n=0
f(x
n
)
_
. (3.3)
The rst approximation which can be made is to drop the limit. We get the so-called
rectangular approximation given by
F
N
= x
N1
n=0
f(x
n
). (3.4)
General integration algorithms approximate the integral F by
F
N
=
N
n=0
f(x
n
)w
n
. (3.5)
In other words we evaluate the function f(x) at N + 1 points in the interval [a, b] then
we sum the values f(x
n
) with some corresponding weights w
n
. For example in the
rectangular approximation (3.4) the values f(x
n
) are summed with equal weights w
n
=
x, n = 0, N 1 and w
N
= 0. It is also clear that the estimation F
N
of the integral F
becomes exact only in the large N limit.
3.2 Trapezoidal Approximation
The trapezoid rule states that we can approximate the integral by a sum of trapezoids.
In the subinterval [x
n
, x
n+1
] we replace the function f(x) by a straight line connecting
the two points (x
n
, f(x
n
)) and (x
n+1
, f(x
n+1
)). The trapezoid has as vertical sides the
two straight lines x = x
n
and x = x
n+1
. The base is the interval x = x
n+1
x
n
. It is
not dicult to convince ourselves that the area of this trapezoid is
(f(x
n+1
) f(x
n
))x
2
+f(x
n
)x =
(f(x
n+1
) +f(x
n
))x
2
. (3.6)
The integral F computed using the trapezoid approximation is therefore given by sum-
ming the contributions from all the N subinterval, viz
T
N
=
N1
n=0
(f(x
n+1
) +f(x
n
))x
2
=
_
1
2
f(x
0
) +
N1
n=1
f(x
n
) +
1
2
f(x
N
)
_
x. (3.7)
We remark that the weights here are given by w
0
= x/2, w
n
= x, n = 1, ..., N 1
and w
N
= x/2.
3.3 Parabolic Approximation or Simpsons Rule
In this case we approximate the function in the subinterval [x
n
, x
n+1
] by a parabola given
by
f(x) = x
2
+x +. (3.8)
The area of the corresponding box is thus given by
_
x
n+1
xn
dx(x
2
+x +) =
_
x
3
3
+
x
2
2
+x
_
x
n+1
xn
. (3.9)
Let us go back and consider the integral
_
1
1
dx(x
2
+x +) =
2
3
+ 2. (3.10)
We remark that
f(1) = + , f(0) = , f(1) = + +. (3.11)
Equivalently
=
f(1) +f(1)
2
f(0) , =
f(1) f(1)
2
, = f(0). (3.12)
Thus
_
1
1
dx(x
2
+x +) =
f(1)
3
+
4f(0)
3
+
f(1)
3
. (3.13)
In other words we can express the integral of the function f(x) = x
2
+x + over the
interval [1, 1] in terms of the values of this function f(x) at x = 1, 0, 1. Similarly we
can express the integral of f(x) over the adjacent subintervals [x
n1
, x
n
] and [x
n
, x
n+1
]
in terms of the values of f(x) at x = x
n+1
, x
n
, x
n1
, viz
_
x
n+1
x
n1
dx f(x) =
_
x
n+1
x
n1
dx(x
2
+x +)
= x
_
f(x
n1
)
3
+
4f(x
n
)
3
+
f(x
n+1
)
3
_
. (3.14)
By adding the contributions from each pair of adjacent subintervals we get the full integral
S
N
= x
N2
2
p=0
_
f(x
2p
)
3
+
4f(x
2p+1
)
3
+
f(x
2p+2
)
3
_
. (3.15)
Clearly we must have N (the number of subintervals) even. We compute
S
N
=
x
3
_
f(x
0
) + 4f(x
1
) + 2f(x
2
) + 4f(x
3
) + 2f(x
4
) +... + 2f(x
N2
) + 4f(x
N1
) +f(x
N
)
_
.
(3.16)
It is trivial to read from this expression the weights in this approximation.
Let us now recall the trapezoidal approximation given by
T
N
=
_
f(x
0
) + 2
N1
n=1
f(x
n
) +f(x
N
)
_
x
2
. (3.17)
Let us also recall that Nx = b a is the length of the total interval which is always
kept xed. Thus by doubling the number of subintervals we halve the width, viz
4T
2N
=
_
2f( x
0
) + 4
2N1
n=1
f( x
n
) + 2f( x
2N
)
_
x
2
=
_
2f( x
0
) + 4
N1
n=1
f( x
2n
) + 4
N1
n=0
f( x
2n+1
) + 2f( x
2N
)
_
x
2
=
_
2f(x
0
) + 4
N1
n=1
f(x
n
) + 4
N1
n=0
f( x
2n+1
) + 2f(x
N
)
_
x
2
. (3.18)
In above we have used the identication x
2n
= x
n
, n = 0, 1, ..., N 1, N. Thus
4T
2N
T
N
=
_
f(x
0
) + 2
N1
n=1
f(x
n
) + 4
N1
n=0
f( x
2n+1
) +f(x
N
)
_
x
= 3S
N
. (3.19)
3.4 Errors
The error estimates for numerical integration are computed as follows. We start with the
Taylor expansion
f(x) = f(x
n
) + (x x
n
)f
(1)
(x
n
) +
1
2!
(x x
n
)
2
f
(2)
(x
n
) +... (3.20)
Thus
_
x
n+1
xn
dx f(x) = f(x
n
)x +
1
2!
f
(1)
(x
n
)(x)
2
+
1
3!
f
(2)
(x
n
)(x)
3
+... (3.21)
The error in the interval [x
n
, x
n+1
] in the rectangular approximation is
_
x
n+1
xn
dx f(x) f(x
n
)x =
1
2!
f
(1)
(x
n
)(x)
2
+
1
3!
f
(2)
(x
n
)(x)
3
+... (3.22)
This is of order 1/N
2
. But we have N subintervals. Thus the total error is of order 1/N.
n
, x
n+1
] in the trapezoidal approximation is
_
x
n+1
xn
dx f(x)
1
2
(f(x
n
) +f(x
n+1
))x =
_
x
n+1
xn
dx f(x)
1
2
(2f(x
n
) + xf
(1)
(x
n
) +
1
2!
(x)
2
f
(2)
(x
n
) +...)x
= (
1
3!

1
2
1
2!
)f
(2)
(x
n
)(x)
3
+... (3.23)
3
and thus the total error is of order 1/N
2
.
In order to compute the error in the interval [x
n1
, x
n+1
] in the parabolic approxi-
mation we compute
_
xn
x
n1
dx f(x) +
_
x
n+1
xn
dx f(x) = 2f(x
n
)x +
2
3!
(x)
3
f
(2)
(x
n
) +
2
5!
(x)
5
f
(4)
(x
n
) +...
(3.24)
Also we compute
x
3
(f(x
n+1
) +f(x
n1
) + 4f(x
n
)) = 2f(x
n
)x +
2
3!
(x)
3
f
(2)
(x
n
) +
2
3.4!
(x)
5
f
(4)
(x
n
) +...
(3.25)
Hence the error in the interval [x
n1
, x
n+1
] in the parabolic approximation is
_
x
n+1
x
n1
dx f(x)
x
3
(f(x
n+1
) +f(x
n1
) + 4f(x
n
)) = (
2
5!

2
3.4!
)(x)
5
f
(4)
(x
n
) +...
(3.26)
5
. The total error is therefore of order 1/N
4
.
3.5 Lab Problem 4: Numerical Integration
(1) We take the integral
I =
_
1
0
f(x)dx ; f(x) = 2x + 3x
2
+ 4x
3
.
Calculate the value of this integral using the rectangular approximation. Compare
with the exact result.
Hint: You can code the function using either "subroutine" or "function".
(2) Calculate the numerical error as a function of N. Compare with the theory.
(3) Repeat the computation using the trapezoid method and the Simpsons rule.
(4) Take now the integrals
I =
_
2
0
cos xdx , I =
_
e
1
1
x
dx , I =
_
+1
1
lim
0
_
1
x
2
+
2
_
dx.
4
Newton-Raphson Algorithms and Interpolation
4.1 Bisection Algorithm
Let f be some function. We are interested in the solutions (roots) of the equation
f(x) = 0. (4.1)
The bisection algorithm works as follows. We start with two values of x say x
+
and x
such that
f(x
) < 0 , f(x
+
) > 0. (4.2)
In other words the function changes sign in the interval between x
and x
+
and thus
there must exist a root between x
and x
+
. If the function changes from positive to
negative as we increase x we conclude that x
+
x
. We bisect the interval [x

+
, x
] at
x =
x
+
+x
2
. (4.3)
If f(x)f(x
+
) > 0 then x
+
will be changed to the point x otherwise x
will be changed
to the point x. We continue this process until the change in x becomes insignicant or
until the error becomes smaller than some tolerance. The relative error is dened by
error =
x
+
x
x
. (4.4)
Clearly the absolute error e = x
i
x
f
is halved at each iteration and thus the rate of
convergence of the bisection rule is linear. This is slow.
4.2 Newton-Raphson Algorithm
We start with a guess x
0
. The new guess x is written as x
0
plus some unknown correction
x, viz
x = x
0
+ x. (4.5)
Next we expand the function f(x) around x
0
, namely
f(x) = f(x
0
) + x
df
dx
|
x=x
0
. (4.6)
The correction x is determined by nding the intersection point of this linear approxi-
mation of f(x) with the x axis. Thus
f(x
0
) + x
df
dx
|
x=x
0
= 0 =x =
f(x
0
)
(df/dx)|
x=x
0
. (4.7)
The derivative of the function f is required in this calculation. In complicated problems
it is much simpler to evaluate the derivative numerically than analytically. In these cases
the derivative may be given by the forward-dierence approximation (with some x not
necessarily equal to x)
df
dx
|
x=x
0
=
f(x
0
+x) f(x
0
)
x
. (4.8)
In summary this method works by drawing the tangent to the function f(x) at the old
guess x
0
and then use the intercept with the x axis as the new hopefully better guess x.
The process is repeated until the change in x becomes insignicant.
Next we compute the rate of convergence of the Newton-Raphson algorithm. Starting
from x
i
the next guess is x
i+1
given by
x
i+1
= x
i
f(x
i
)
f
(x)
. (4.9)
The absolute error at step i is
i
= x x
i
while the absolute error at step i + 1 is
i+1
= x x
i+1
where x is the actual root. Then
i+1
=
i
+
f(x
i
)
f
(x)
. (4.10)
By using Taylor expansion we have
f(x) = 0 = f(x
i
) + (x x
i
)f
(x
i
) +
(x x
i
)
2
2!
f
(x
i
) +... (4.11)
In other words
f(x
i
) =
i
f
(x
i
)

2
i
2!
f
(x
i
) +... (4.12)
Therefore the error is given by
i+1
=
2
i
2
f
(x
i
)
f
(x
i
)
. (4.13)
This is quadratic convergence. This is faster than the bisection rule.
4.3 Hybrid Method
We can combine the certainty of the bisection rule in nding a root with the fast con-
vergence of the Newton-Raphson algorithm into a hybrid algorithm as follows. First we
must know that the root is bounded in some interval [a, c]. We can use for example a
graphical method. Next we start from some initial guess b. We take a Newton-Raphson
step
b
= b
f(b)
f
(b)
. (4.14)
We check whether or not this step is bounded in the interval [a, c]. In other words we
must check that
ab
f(b)
f
(b)
c (b c)f
(b) f(b)0(b a)f
(b) f(b). (4.15)

Therefore if
_
(b c)f
(b) f(b)
__
(b a)f
(b) f(b)
_
< 0 (4.16)
Then the Newton-Raphson step is accepted else we take instead a bisection step.
4.4 Lagrange Interpolation
Let us rst recall that taylor expansion allows us to approximate a function at a point
x if the function and its derivatives are known in some neighbouring point x
0
. The
lagrange interpolation tries to approximate a function at a point x if only the values of
the function in several other points are known. Thus this method does not require the
knowledge of the derivatives of the function. We start from taylor expansion
f(y) = f(x) + (y x)f
(x) +
1
2!
(y x)
2
f
(x) +.. (4.17)

Let us assume that the function is known at three points x
1
, x
2
and x
3
. In this case we
can approximate the function f(x) by some function p(x) and write
f(y) = p(x) + (y x)p
(x) +
1
2!
(y x)
2
p
(x). (4.18)
We have
f(x
1
) = p(x) + (x
1
x)p
(x) +
1
2!
(x
1
x)
2
p
(x)
f(x
2
) = p(x) + (x
2
x)p
(x) +
1
2!
(x
2
x)
2
p
(x)
f(x
3
) = p(x) + (x
3
x)p
(x) +
1
2!
(x
3
x)
2
p
(x). (4.19)
We can immediately nd
p(x) =
1
1 +a
2
+a
3
f(x
1
) +
a
2
1 +a
2
+a
3
f(x
2
) +
a
3
1 +a
2
+a
3
f(x
3
). (4.20)
The coecients a
2
and a
3
solve the equations
a
2
(x
2
x)
2
+a
3
(x
3
x)
2
= (x
1
x)
2
a
2
(x
2
x) +a
3
(x
3
x) = (x
1
x). (4.21)
We nd
a
2
=
(x
1
x)(x
3
x
1
)
(x
2
x)(x
2
x
3
)
, a
3
=
(x
1
x)(x
2
x
1
)
(x
3
x)(x
2
x
3
)
. (4.22)
Thus
1 +a
2
+a
3
=
(x
3
x
1
)(x
2
x
1
)
(x
2
x)(x
3
x)
. (4.23)
Therefore we get
p(x) =
(x x
2
)(x x
3
)
(x
1
x
2
)(x
1
x
3
)
f(x
1
) +
(x x
1
)(x x
3
)
(x
2
x
1
)(x
2
x
3
)
f(x
2
) +
(x x
1
)(x x
2
)
(x
3
x
1
)(x
3
x
2
)
f(x
3
).
(4.24)
This is a quadratic polynomial.
Let x be some independent variable with tabulated values x
i
, i = 1, 2, ..., n.. The
dependent variable is a function f(x) with tabulated values f
i
= f(x
i
). Let us then
assume that we can approximate f(x) by a polynomial of degree n 1 , viz
p(x) = a
0
+a
1
x +a
2
x
2
+... +a
n1
x
n1
. (4.25)
A polynomial which goes through the n points (x
i
, f
i
= f(x
i
)) was given by Lagrange.
This is given by
p(x) = f
1
1
(x) +f
2
2
(x) +... +f
n
n
(x). (4.26)
i
(x) =
n
j(=i)=1
x x
j
x
i
x
j
. (4.27)
We remark
i
(x
j
) =
ij
. (4.28)
n
i=1
i
(x) = 1. (4.29)
The Lagrange polynomial can be used to t the entire table with n equal the number of
points in the table. But it is preferable to use the Lagrange polynomial to to t only a
small region of the table with a small value of n. In other words use several polynomials
to cover the whole table and the t considered here is local and not global.
4.5 Cubic Spline Interpolation
We consider n points (x
1
, f(x
1
)),(x
2
, f(x
2
)),...,(x
n
, f(x
n
)) in the plane. In every interval
x
j
xx
j+1
we approximate the function f(x) with a cubic polynomial of the form
p(x) = a
j
(x x
j
)
3
+b
j
(x x
j
)
2
+c
j
(x x
j
) +d
j
. (4.30)
We assume that
p
j
= p(x
j
) = f(x
j
). (4.31)
In other words the p
j
for all j = 1, 2, ..., n 1 are known. From the above equation we
conclude that
d
j
= p
j
. (4.32)
We compute
p
(x) = 3a
j
(x x
j
)
2
+ 2b
j
(x x
j
) +c
j
. (4.33)
p
(x) = 6a
j
(x x
j
) + 2b
j
. (4.34)
Thus we get by substituting x = x
j
into p
(x) the result

b
j
=
p
j
2
. (4.35)
By substituting x = x
j+1
into p
(x) we get the result

a
j
=
p
j+1
p
j
6h
j
. (4.36)
By substituting x = x
j+1
into p(x) we get
p
j+1
= a
j
h
3
j
+b
j
h
2
j
+c
j
h
j
+p
j
. (4.37)
By using the values of a
j
and b
j
we obtain
c
j
=
p
j+1
p
j
h
j
h
j
6
(p
j+1
+ 2p
j
). (4.38)
Hence
p(x) =
p
j+1
p
j
6h
j
(x x
j
)
3
+
p
j
2
(x x
j
)
2
+
_
p
j+1
p
j
h
j
h
j
6
(p
j+1
+ 2p
j
)
_
(x x
j
) +p
j
.
(4.39)
In other words the polynomials are determined from p
j
and p
j
. The p
j
are known given
by p
j
= f(x
j
). It remains to determine p
j
. We take the derivative of the above equation
p
(x) =
p
j+1
p
j
2h
j
(x x
j
)
2
+p
j
(x x
j
) +
_
p
j+1
p
j
h
j
h
j
6
(p
j+1
+ 2p
j
)
_
. (4.40)
This is the derivative in the interval [x
j
, x
j+1
]. We compute
p
(x
j
) =
_
p
j+1
p
j
h
j
h
j
6
(p
j+1
+ 2p
j
)
_
. (4.41)
The derivative in the interval [x
j1
, x
j
] is
p
(x) =
p
j
p
j1
2h
j1
(x x
j1
)
2
+p
j1
(x x
j1
) +
_
p
j
p
j1
h
j1
h
j1
6
(p
j
+ 2p
j1
)
_
. (4.42)
We compute
p
(x
j
) =
p
j
p
j1
2
h
j1
+p
j1
h
j1
+
_
p
j
p
j1
h
j1
h
j1
6
(p
j
+ 2p
j1
)
_
. (4.43)
By matching the two expressions for p
(x
j
) we get
h
j1
p
j1
+ 2(h
j
+h
j1
)p
j
+h
j
p
j+1
= 6
_
p
j+1
p
j
h
j
p
j
p
j1
h
j1
_
. (4.44)
These are n 2 equations since j = 2, ..., n 1 for n unknown p
j
. We need two more
equations. These are obtained by computing the rst derivative p
(x) at x = x
1
and
x = x
n
. We obtain the two equations
h
1
(p
2
+ 2p
1
) =
6(p
2
p
1
)
h
1
6p
1
. (4.45)
h
n1
(p
n1
+ 2p
n
) =
6(p
n
p
n1
)
h
n1
+ 6p
n
. (4.46)
The n equations (4.44), (4.45) and (4.46) correspond to a tridiagonal linear system. In
general p
1
and p
n
are not known. In this case we may use natural spline in which the
second derivative vanishes at the end points and hence
p
2
p
1
h
1
p
1
=
p
n
p
n1
h
n1
p
n
= 0. (4.47)
4.6 Lab Problem 5: Newton-Raphson Algorithm
A particle of mass m moves inside a potential well of height V and length 2a centered
around 0. We are interested in the states of the system which have energies less than V ,
i.e. bound states. The states of the system can be even or odd. The energies associated
with the even wave functions are solutions of the transcendental equation
tan a = .
=
_
2mE
2
, =
_
2m(V E)
2
.
In the case of the innite potential well we nd the solutions
E
n
=
(n +
1
2
)
2
2
2ma
2
, n = 0, 1....
We choose (dropping units)
= 1 , a = 1 , 2m = 1.
In order to nd numerically the energies E
n
we will use the Newton-Raphson algorithm
which allows us to nd the roots of the equation f(x) = 0 as follows. From an initial
guess x
0
, the rst approximation x
1
to the solution is determined from the intersection
of the tangent to the function f(x) at x
0
with the xaxis. This is given by
x
1
= x
0
f(x
0
)
f
(x
0
)
.
Next by using x
1
we repeat the same step in order to nd the second approximation x
2
to the solution. In general the approximation x
i+1
to the desired solution in terms of the
approximation x
i
is given by the equation
x
i+1
= x
i
f(x
i
)
f
(x
i
)
.
(1) For V = 10, determine the solutions using the graphical method. Consider the two
functions
f() = tan a , g() =

=
_
V
2
1.
(2) Find using the method of Newton-Raphson the two solutions with a tolerance equal
10
8
. For the rst solution we take the initial guess = /a and for the second
solution we take the initial guess = 2/a.
(3) Repeat for V = 20.
(4) Find the 4 solutions for V = 100. Use the graphical method to determine the initial
step each time.
(5) Repeat the above questions using the bisection method.
5
The Solar System-The Runge-Kutta Methods
5.1 The Solar System
5.1.1 Newtons Second Law
We consider the motion of the Earth around the Sun. Let r be the distance and M
s
and
M
e
be the masses of the Sun and the Earth respectively. We neglect the eect of the
other planets and the motion of the Sun (i.e. we assume that M
s
>> M
e
). The goal
is to calculate the position of the Earth as a function of time. We start from Newtons
second law of motion
M
e
d
2
r
dt
2
=
GM
e
M
s
r
3
r
=
GM
e
M
s
r
3
(x
i +y
j). (5.1)
We get the two equations
d
2
x
dt
2
=
GM
s
r
3
x. (5.2)
d
2
y
dt
2
=
GM
s
r
3
y. (5.3)
We replace these two second-order dierential equations by the four rst-order dierential
equations
dx
dt
= v
x
. (5.4)
dv
x
dt
=
GM
s
r
3
x. (5.5)
dy
dt
= v
y
. (5.6)
dv
y
dt
=
GM
s
r
3
y. (5.7)
We recall
r =
_
x
2
+y
2
. (5.8)
5.1.2 Astronomical Units and Initial Conditions
The distance will be measured in astronomical units (AU) whereas time will be measured
in years. One astronomical unit of lenght (1 AU) is equal to the average distance between
the earth and the sun, viz 1AU = 1.5 10
11
m. The astronomical unit of mass can be
found as follows. Assuming a circular orbit we have
M
e
v
2
r
=
GM
s
M
e
r
2
. (5.9)
Equivalently
GM
s
= v
2
r. (5.10)
The radius is r = 1AU. The velocity of the earth is v = 2r/yr = 2AU/yr. Hence
GM
s
= 4
2
AU
3
/yr
2
. (5.11)
For the numerical simulations it is important to determine the correct initial conditions.
The orbit of Mercury is known to be an ellipse with eccentricity e = 0.206 and radius
(semimajor axis) a = 0.39 AU with the Sun at one of the foci. The distance between
the Sun and the center is ea. The rst initial condition is x
0
= r
1
, y
0
= 0 where r
1
is the maximum distance from Mercury to the Sun,i.e. r
1
= (1 + e)a = 0.47 AU. The
second initial condition is the velocity (0, v
1
) which can be computed using conservation
of energy and angular momentum. For example by comparing with the point (0, b) on
the orbit where b is the semiminor axis, i.e b = a
1 e
2
the velocity (v
2
, 0) there can
be obtained in terms of (0, v
1
) from conservation of angular momentum as follows
r
1
v
1
= bv
2
v
2
=
r
1
v
1
b
. (5.12)
Next conservation of energy yields
GM
s
M
m
r
1
+
1
2
M
m
v
2
1
=
GM
s
M
m
r
2
+
1
2
M
m
v
2
2
. (5.13)
In above r
2
=
e
2
a
2
+b
2
is the distance between the Sun and Mercury when at the
point (0, b). By substituting the value of v
2
we get an equation for v
1
. This is given by
v
1
=
_
GM
s
a
1 e
1 +e
= 8.2 AU/yr. (5.14)
5.1.3 Keplers Laws
Keplers laws are given by the following three statements:
The planets move in elliptical orbits around the sun. The sun resides at one focus.
The line joining the sun with any planet sweeps out equal areas in equal times.
Given an orbit with a period T and a semimajor axis a the ratio T
2
/a
3
is a constant.
The derivation of these three laws proceeds as follows. We work in polar coordinates.
Newtons second law reads
M
e
r =
GM
s
M
e
r
2
r. (5.15)
We use

r =

and

r to derive

r = r r + r
and

r = ( r r
2
) r + (r
+ 2 r
.
Newtons second law decomposes into the two equations
r
+ 2 r

= 0. (5.16)
r r
2
=
GM
s
r
2
. (5.17)
Let us recall that the angular momentum by unit mass is dened by
l = r
r = r
2

r
.
Thus l = r
2

. Equation (5.16) is precisely the requirement that angular momentum is
conserved. Indeed we compute
dl
dt
= r(r
+ 2 r
) = 0. (5.18)
Now we remark that the area swept by the vector r in a time interval dt is dA = (rrd)/2
where d is the angle traveled by r during dt. Clearly
dA
dt
=
1
2
l. (5.19)
In other words the planet sweeps equal areas in equal times since l is conserved. This is
Keplers second law.
The second equation (5.17) becomes now
r =
l
2
r
3

GM
s
r
2
(5.20)
By multiplying this equation with r we obtain
d
dt
E = 0 , E =
1
2
r
2
+
l
2
2r
2

GM
s
r
. (5.21)
This is precisely the statement of conservation of energy. E is the energy per unit mass.
Solving for dt in terms of dr we obtain
dt =
dr
2
_
E
l
2
2r
2
+
GMs
r
_
(5.22)
However dt = (r
2
d)/l. Thus
d =
ldr
r
2
2
_
E
l
2
2r
2
+
GMs
r
_
(5.23)
By integrating this equation we obtain (with u = 1/r)
=
_
ldr
r
2
2
_
E
l
2
2r
2
+
GMs
r
_
=
_
du
_
2E
l
2
+
2GMs
l
2
u u
2
. (5.24)
This integral can be done explicitly. We get
= arccos
_
u C
eC
_
+
, e =
1 +
2l
2
E
G
2
M
2
s
, C =
GM
s
l
2
. (5.25)
By inverting this equation we get an equation of ellipse with eccentricity e since E < 0,
viz
1
r
= C(1 +e cos(
)). (5.26)
This is Keplers rst law. The angle at which r is maximum is
= . This distance
is precisely (1 +e)a where a is the semi-major axis of the ellipse since ea is the distance
between the Sun which is at one of the two foci and the center of the ellipse. Hence we
obtain the relation
(1 e
2
)a =
1
C
=
l
2
GM
s
. (5.27)
From equation (5.19) we can derive Keplers third law. By integrating both sides of the
equation over a single period T and then taking the square we get
A
2
=
1
4
l
2
T
2
. (5.28)
A is the area of the ellipse, i.e. A = ab where the semi-minor axis b is related the
semi-major axis a by b = a
1 e
2
. Hence
2
a
4
(1 e
2
) =
1
4
l
2
T
2
. (5.29)
By using equation (5.27) we get the desired formula
T
2
a
3
=
4
2
GM
s
. (5.30)
5.1.4 The inverse-Square Law and Stability of Orbits
Any object with mass generates a gravitational eld and thus gravitational eld lines
will emanate from the object and radiate outward to innity. The number of eld lines
N is proportional to the mass. The density of eld lines crossing a sphere of radius r
surrounding this object is given by N/4r
2
. This is the origin of the inverse-square law.
Therefore any other object placed in this gravitational eld will experience a gravitational
force proportional to the number of eld lines which intersect it. If the distance between
this second object and the source is increased the force on it will become weaker because
the number of eld lines which intersect it will decrease as we are further away from the
source.
5.2 Euler-Cromer Algorithm
The time discretization is
t t(i) = it , i = 0, ..., N. (5.31)
The total time interval is T = Nt. We dene x(t) = x(i), v
x
(t) = v
x
(i), y(t) = y(i),
v
y
(t) = v
y
(i). Equations (5.4), (5.5), (5.6),(5.7) and (5.8) become (with i = 0, ..., N)
v
x
(i + 1) = v
x
(i)
GM
s
(r(i))
3
x(i)t. (5.32)
x(i + 1) = x(i) +v
x
(i)t. (5.33)
v
y
(i + 1) = v
y
(i)
GM
s
(r(i))
3
y(i)t. (5.34)
y(i + 1) = y(i) +v
y
(i)t. (5.35)
r(i) =
_
x(i)
2
+y(i)
2
. (5.36)
This is Euler algorithm. It can also be rewritten with x(i) = x(i 1), y(i) = y(i 1),
v
x
(i) = v
x
(i 1), v
y
(i) = v
y
(i 1), r(i) = r(i 1) and i = 1, ..., N + 1 as
v
x
(i + 1) = v
x
(i)
GM
s
( r(i))
3
x(i)t. (5.37)
x(i + 1) = x(i) + v
x
(i)t. (5.38)
v
y
(i + 1) = v
y
(i)
GM
s
( r(i))
3
y(i)t. (5.39)
y(i + 1) = y(i) + v
y
(i)t. (5.40)
r(i) =
_
x(i)
2
+ y(i)
2
. (5.41)
In order to maintain energy conservation we employ Euler-Cromer algorithm. We calcu-
late as in the Eulers algorithm the velocity at time step i + 1 by using the position and
velocity at time step i. However we compute the position at time step i +1 by using the
position at time step i and the velocity at time step i + 1, viz
v
x
(i + 1) = v
x
(i)
GM
s
( r(i))
3
x(i)t. (5.42)
x(i + 1) = x(i) + v
x
(i + 1)t. (5.43)
v
y
(i + 1) = v
y
(i)
GM
s
( r(i))
3
y(i)t. (5.44)
y(i + 1) = y(i) + v
y
(i + 1)t. (5.45)
5.3 The Runge-Kutta Algorithm
5.3.1 The Method
The problem is still trying to solve the rst order dierential equation
dy
dx
= f(x, y). (5.46)
In the Eulers method we approximate the function y = y(x) in each interval [x
n
, x
n+1
]
by the straight line
y
n+1
= y
n
+ xf(x
n
, y
n
). (5.47)
The slope f(x
n
, y
n
) of this line is exactly given by the slope of the function y = y(x) at
the begining of the inetrval [x
n
, x
n+1
].
Given the value y
n
at x
n
we evaluate the value y
n+1
at x
n+1
using the method of
Runge-Kutta as follows. First the middle of the interval [x
n
, x
n+1
] which is at the value
x
n
+
1
2
x corresponds to the y-value y
n+1
calculated using the Eulers method, viz
y
n+1
= y
n
+
1
2
k
1
where
k
1
= xf(x
n
, y
n
). (5.48)
Second the slope at this middle point (x
n
+
1
2
x, y
n
+
1
2
k
1
) which is given by
k
2
x
= f(x
n
+
1
2
x, y
n
+
1
2
k
1
) (5.49)
is the value of the slope which will be used to estimate the correct value of y
n+1
at x
n+1
using again Eulers method, namely
y
n+1
= y
n
+k
2
. (5.50)
In summary the Runge-Kutta algorithm is given by
k
1
= xf(x
n
, y
n
)
k
2
= xf(x
n
+
1
2
x, y
n
+
1
2
k
1
)
y
n+1
= y
n
+k
2
. (5.51)
The error in this method is proportional to x
3
. This can be shown as follows. We have
y(x + x) = y(x) + x
dy
dx
+
1
2
(x)
2
d
2
y
dx
2
+...
= y(x) + xf(x, y) +
1
2
(x)
2
d
dx
f(x, y) +...
= y(x) + x
_
f(x, y) +
1
2
x
f
x
+
1
2
xf(x, y)
f
y
_
+...
= y(x) + xf(x +
1
2
x, y +
1
2
xf(x, y)) +O(x
3
)
= y(x) + xf(x +
1
2
x, y +
1
2
k
1
) +O(x
3
)
= y(x) +k
2
+O(x
3
). (5.52)
Let us nally note that the above Runge-Kutta method is strictly speaking the second-
order Runge-Kutta method. The rst-order Runge-Kutta method is the Euler algorithm.
The higher-order Runge-Kutta methods will not be discussed here.
5.3.2 Example 1: The Harmonic Oscillator
Let us apply this method to the problem of the harmonic oscillator. We have the dier-
ential equations
d
dt
=
d
dt
=
g
l
. (5.53)
Eulers equations read
n+1
=
n
+ t
n
n+1
=
n
g
l

n
t. (5.54)
First we consider the function = (t). The middle point is (t
n
+
1
2
t,
n
+
1
2
k
1
) where
k
1
= t
n
. For the function = (t) the middle point is (t
n
+
1
2
t,
n
+
1
2
k
3
) where
k
3
=
g
l
t
n
. Therefore we have
k
1
= t
n
k
3
=
g
l
t
n
. (5.55)
The slope of the function (t) at its middle point is
k
2
t
=
n
+
1
2
k
3
. (5.56)
The slope of the function (t) at its middle point is
k
4
t
=
g
l
(
n
+
1
2
k
1
). (5.57)
The Runge-Kutta solution is then given by
n+1
=
n
+k
2
n+1
=
n
+k
4
. (5.58)
5.3.3 Example 2: The Solar System
Let us consider the equations
dx
dt
= v
x
. (5.59)
dv
x
dt
=
GM
s
r
3
x. (5.60)
dy
dt
= v
y
. (5.61)
dv
y
dt
=
GM
s
r
3
y. (5.62)
First we consider the function x = x(t). The middle point is (t
n
+
1
2
t, x
n
+
1
2
k
1
) where
k
1
= t v
xn
. For the function v
x
= v
x
(t) the middle point is (t
n
+
1
2
t, v
xn
+
1
2
k
3
) where
k
3
=
GMs
rn
t x
n
. Therefore we have
k
1
= t v
xn
k
3
=
GM
s
r
3
n
t x
n
. (5.63)
The slope of the function x(t) at the middle point is
k
2
t
= v
xn
+
1
2
k
3
. (5.64)
The slope of the function v
x
(t) at the middle point is
k
4
t
=
GM
s
R
3
n
(x
n
+
1
2
k
1
). (5.65)
Next we consider the function y = y(t). The middle point is (t
n
+
1
2
t, y
n
+
1
2
k
1
) where
k
1
= t v
yn
. For the function v
y
= v
y
(t) the middle point is (t
n
+
1
2
t, v
yn
+
1
2
k
3
) where
k
3
=
GMs
rn
t y
n
. Therefore we have
k
1
= t v
yn
k
3
=
GM
s
r
3
n
t y
n
. (5.66)
The slope of the function y(t) at the middle point is
k
2
t
= v
yn
+
1
2
k
3
. (5.67)
The slope of the function v
y
(t) at the middle point is
k
4
t
=
GM
s
R
3
n
(y
n
+
1
2
k
1
). (5.68)
In the above equations
R
n
=
_
(x
n
+
1
2
k
1
)
2
+ (y
n
+
1
2
k
1
)
2
. (5.69)
The Runge-Kutta solutions are then given by
x
n+1
= x
n
+k
2
v
x(n+1)
= v
xn
+k
4
y
n+1
= y
n
+k
2
v
y(n+1)
= v
yn
+k
4
. (5.70)
5.4 Precession of the Perihelion of Mercury
The orbit of Mercury is elliptic. The orientation of the axes of the ellipse rotate with
time. This is the precession of the perihelion (the point of the orbit nearest to the
Sun) of Mercury. Mercurys perihelion makes one revolution every 23000 years. This is
approximately 566 arcseconds per century. The gravitational forces of the other planets
(in particular Jupiter) lead to a precession of 523 arcseconds per century. The remaining
43 arcseconds per century are accounted for by general relativity.
For objects too close together (like the Sun and Mercury) the force of gravity predicted
by general relativity deviates from the inverse-square law. This force is given by
F =
GM
s
M
m
r
2
(1 +

r
2
) , = 1.1 10
8
AU
2
. (5.71)
We discuss here some of the numerical results obtained with the Runge-Kutta method
for dierent values of . We take the time step and the number of iterations to be
N = 20000 and dt = 0.0001. The angle of the line joining the Sun and Mercury with the
horizontal axis when mercury is at the perihelion is found to change linearly with time.
We get the following rates of precession
= 0.0008 ,
d
dt
= 8.414 0.019
= 0.001 ,
d
dt
= 10.585 0.018
= 0.002 ,
d
dt
= 21.658 0.019
= 0.004 ,
d
dt
= 45.369 0.017. (5.72)
Thus
d
dt
= a , = 11209.2 147.2 degrees/(yr.). (5.73)
By extrapolating to the value provided by general relativity, viz = 1.1 10
8
we get
d
dt
= 44.4 0.6 arcsec/century. (5.74)
5.5 Lab Problem 6: Runge-Kutta Algorithm- The Solar
System
Part I We consider a solar system consisting of a single planet moving around the Sun.
We suppose that the Sun is very heavy compared to the planet that we can safely assume
that it is not moving at the center of the system. Newtons second law gives the following
equations of motion
v
x
=
dx
dt
,
dv
x
dt
=
GM
s
r
3
x , v
y
=
dy
dt
,
dv
y
dt
=
GM
s
r
3
y.
We will use here the astronomical units dened by GM
s
= 4
2
AU
3
/yr
2
.
(1) Write a Fortran code in which we implement the Runge-Kutta algorithm for the
problem of solving the equations of motion of the the solar system.
(2) Compute the trajectory, the velocity and the energy as functions of time. What do
you observe for the energy.
(3) According to Keplers rst law the orbit of any planet is an ellipse with the Sun at
one of the two foci. In the following we will only consider planets which are known
to have circular orbits to a great accuracy. These planets are Venus, Earth, Mars,
Jupiter and Saturn. The radii in astronomical units are given by
a
venus
= 0.72 , a
earth
= 1 , a
mars
= 1.52 , a
jupiter
= 5.2 , a
saturn
= 9.54.
Verify that Keplers rst law indeed holds for these planets.
In order to answer questions 2 and 3 above we take the initial conditions
x(1) = a , y(1) = 0 , v
x
(1) = 0 , v
y
(1) = v.
The value chosen for the initial velocity is very important to get a correct orbit
and must be determined for example by assuming that the orbit is indeed circular
and as a consequence the centrifugal force is balanced by the force of gravitational
attraction. We get v =
_
GM
s
/a.
We take the step and the number of iterations t = 0.01 yr , N = 10
3
10
4
.
Part II
(1) According to Keplers third law the square of the period of a planet is directly
proportional to the cube of the semi-major axis of its orbit. For circular orbits the
proportionality factor is equal 1 exactly. Verify this fact for the planets mentioned
above. We can measure the period of a planet by monitoring when the planet
returns to its farthest point from the sun.
(2) By changing the initial velocity appropriately we can obtain an elliptical orbit.
Check this thing.
(3) The fundamental laws governing the motion of the solar system are Newtons law
of universal attraction and Newtons second law of motion. Newtons law of uni-
versal attraction states that the force between the Sun and a planet is inversely
proportioanl to the square of the distance between them and it is directed from
the planet to the Sun. We will assume in the following that this force is inversely
proportional to a dierent power of the distance. Modify the code accordingly and
calculate the new orbits for powers between 1 and 3. What do you observe and
what do you conclude.
5.6 Lab Problem 7: Precession of the perihelion of Mercury
According to Keplers rst law the orbits of all planets are ellipses with the Sun at one
of the two foci. This law can be obtained from applying Newtons second law to the
system consisting of the Sun and a single planet. The eect of the other planets on the
motion will lead to a change of orientation of the orbital ellipse within the orbital plane
of the planet. Thus the point of closest approach (the perihelion) will precess, i.e. rotate
around the sun. All planets suer from this eect but because they are all farther from
the sun and all have longer periods than Mercury the amount of precession observed for
them is smaller than that of Mercury.
However it was established earlier on that the precession of the perihelion of Mer-
cury due to Newtonian eects deviates from the observed precession by the amount
43 arcsecond/century. As it turns out this can only be explained within general rela-
tivity. The large mass of the Sun causes space and time around it to be curved which
is felt the most by Mercury because of its proximity. This spacetime curvature can be
approximated by the force law
F =
GM
s
M
m
r
2
(1 +

r
2
) , = 1.1.10
8
AU
2
.
(1) Include the above force in the code. The initial position and velocity of Mercury
are
x
0
= (1 +e)a , y
0
= 0.
v
x0
= 0 , v
y0
=
_
GM
s
a
1 e
1 +e
.
Thus initially Mercury is at its farthest point from the Sun since a is the semi-major
axis of Mercury (a = 0.39 AU) and e is its eccentricity (e = 0.206) and hence ea
is the distance between the Sun and the center of the ellipse. The semi-minor
axis is dened by b = a
1 e
2
. The initial velocity was calculated from applying
the principles of conservation of angular momentum and conservation of energy
between the above initial point and the point (0, b).
(2) The amount of precession of the perihelion of Mercury is very small because is
very small. In fact it can not be measured directly in any numerical simulation
with a limited amount of time. Therefore we will choose a larger value of for
example = 0.0008 AU
2
. We also work with N = 20000 , dt = 0.0001. Compute
the orbit for these values. Compute the angle made between the vector position
of Mercury and the horizontal axis as a function of time. Compute also the distance
between Mercury and the sun and its derivative with respect to time given by
dr
dt
=
xv
x
+yv
y
r
.
This derivative will vanish each time Mercury reaches its farthest point from the
sun or its closest point from the sun (the perihelion). Plot the angle
p
made
between the vector position of Mercury at its farthest point and the horizontal axis
as a function of time. What do you observe. Determine the slope d
p
/dt which is
precisely the amount of precession of the perihelion of Mercury for the above value
of .
(3) Repeat the above question for other values of say = 0.001, 0.002, 0.004. Each
time compute d
p
/dt. Plot d
p
/dt as a function of . Determine the slope.
Deduce the amount of precession of the perihelion of Mercury for the value of
= 1.1.10
8
AU
2
.
6
Chaos: Chaotic Pendulum
6.1 Equation of Motion
We start from a simple pendulum. The equation of motion is given by
ml
d
2
dt
2
= mg sin. (6.1)
We consider the eect of air resistance on the motion of the mass m. We will assume
that the force of air resistance is given by Stokes law. We get
ml
d
2
dt
2
= mg sin mlq
d
dt
. (6.2)
The air friction will drain all energy from the pendulum. In order to maintain the motion
against the damping eect of air resistance we will add a driving force. We will choose a
periodic force with amplitude mlF
D
and frequency
D
. This arise for example if we apply
a periodic electric eld with amplitude E
D
and frequency
D
on the mass m which is
assumed to have an electric charge q, i.e mlF
D
= qE
D
. It can also arise from the periodic
oscillations of the pendulums pivot point. By adding the driving force we get then the
equation of motion
ml
d
2
dt
2
= mg sin mlq
d
dt
+mlF
D
cos
D
t. (6.3)
The natural frequency of the oscillations is given by the frequency of the simple pendulum,
viz
0
=
_
g
l
. (6.4)
We will always take
0
= 1, i.e. l = g. The equation of motion becomes
d
2
dt
2
+
1
Q
d
dt
+ sin = F
D
cos
D
t. (6.5)
The coecient Q = 1/q is known as the quality factor. It measures how many oscillations
the pendulum without driving force will make before its energy is drained. We will write
the above second order dierential equation as two rst order dierential equations,
namely
d
dt
=
d
dt
=
1
Q
sin +F
D
cos
D
t. (6.6)
This system of dierential equations does not admit a simple analytic solution. The
linear approximation corresponds to small amplitude oscillations, viz
sin . (6.7)
The dierential equations become linear given by
d
dt
=
d
dt
=
1
Q
+F
D
cos
D
t. (6.8)
Or equivalently
d
2
dt
2
=
1
Q
d
dt
+F
D
cos
D
t. (6.9)
For F
D
= 0 the solution is given by
t0
=
_
(0) cos
t +
1
_
(0) +
(0)
2Q
_
sin
t
_
e
t
2Q
,
=
_
1
1
4Q
2
. (6.10)
For F
D
= 0 a particular solution is given by
= F
D
(a cos
D
t +b sin
D
t). (6.11)
We nd
a =
1
(1
2
D
)
2
+

2
D
Q
2
(1
2
D
) , b =
1
(1
2
D
)
2
+

2
D
Q
2
D
Q
. (6.12)
For F
D
= 0 the general solution is given by
=
+
t
. (6.13)
t
=
__
(0)
F
D
(1
2
D
)
(1
2
D
)
2
+

2
D
Q
2
_
cos
t +
1
_
(0) +
(0)
2Q

1
2Q
F
D
(1 3
2
D
)
(1
2
D
)
2
+

2
D
Q
2
_
sin
t
_
e
t
2Q
.
(6.14)
The last two terms depend on the initial conditions and will vanish exponentially at very
large times t , i.e. they are transients. The asymptotic motion is given by
.
Thus for t we get
=
= F
D
(a cos
D
t +b sin
D
t). (6.15)
Also for t we get
=
d
dt
= F
D
D
(a sin
D
t +b cos
D
t). (6.16)
We compute in the limit of large times t
2
+

2
2
D
=

F
2
D
= F
2
D
(a
2
+b
2
) =
F
2
D
(1
2
D
)
2
+

2
D
Q
2
. (6.17)
In other words the orbit of the system in phase space is an ellipse. The motion is periodic
with period equal to the period of the driving force. This ellipse is also called a periodic
attractor because regardless of the initial conditions the trajectory of the system will
tend at large times to this ellipse.
Let us also remark that the maximum angular displacement is

F
D
. The function
F
D
=

F
D
(
D
) exhibits resonant behavior as the driving frequency approaches the natural
frequency which is equivalent to the limit
D
1. In this limit

F
D
= QF
D
. The width
of the resonant window is proportional to 1/Q so for Q we observe that

F
D

when
D
1 while for Q 0 we observe that

F
D
0 when
D
1.
In general the time-asymptotic response of any linear system to a periodic drive
is periodic with the same period as the driving force. Furthermore when the driving
frequency approaches one of the natural frequencies the response will exhibits resonant
behavior.
The basic ingredient in deriving the above results is the linearity of the dynamical
system. As we will see shortly periodic motion is not the only possible time-asymptotic
response of a dynamical system to a periodic driving force.
6.2 Numerical Algorithms
The equations of motion are
d
dt
=
d
dt
=
1
Q
sin +F(t). (6.18)
The external force is periodic and it will be given by one of the following expressions
F(t) = F
D
cos
D
t. (6.19)
F(t) = F
D
sin
D
t. (6.20)
6.2.1 Euler-Cromer Algorithm
Numerically we can employ the Euler-Cromer algorithm in order to solve this system of
dierential equations. The solution goes as follows. First we choose the initial conditions.
For example
(1) = 0
(1) = 0
t(1) = 0. (6.21)
For i = 1, ..., N + 1 we use
(i + 1) = (i) + t
_
1
Q
(i) sin(i) +F(i)
_
(i + 1) = (i) + t (i + 1)
t(i + 1) = t i. (6.22)
F(i) F(t(i)) = F
D
cos
D
t(i 1). (6.23)
F(i) F(t(i)) = F
D
sin
D
t(i 1). (6.24)
6.2.2 Runge-Kutta Algorithm
In order to achieve better precision we employ the Runge-Kutta algorithm. For i =
1, ..., N + 1 we use
k
1
= t (i)
k
3
= t
_
1
Q
(i) sin (i) +F(i)
_
k
2
= t
_
(i) +
1
2
k
3
_
k
4
= t
_
1
Q
_
(i) +
1
2
k
3
_
sin
_
(i) +
1
2
k
1
_
+F(i +
1
2
)
_
(6.25)
(i + 1) = (i) +k
2
(i + 1) = (i) +k
4
t(i + 1) = t i. (6.26)
F(i) F(t(i)) = F
D
cos
D
t(i 1). (6.27)
F(i) F(t(i)) = F
D
sin
D
t(i 1). (6.28)
F(i +
1
2
) F(t(i) +
1
2
t) = F
D
cos
D
t(i
1
2
). (6.29)
F(i +
1
2
) F(t(i) +
1
2
t) = F
D
sin
D
t(i
1
2
). (6.30)
6.3 Elements of Chaos
6.3.1 Buttery Eect: Sensitivity to Initial Conditions
The solution in the linear regime (small amplitude) reads
=
+
t
. (6.31)
The transient is of the form
t
= f((0), (0))e
t/2Q
. (6.32)
This goes to zero at large times t. The time-asymptotic is thus given by
= F
D
(a cos
D
t +b sin
D
t). (6.33)
The motion in the phase space is periodic with period equal to the period of the driving
force. The orbit in phase space is precisley an ellipse of the form
+

2
2
D
= F
2
D
(a
2
+b
2
). (6.34)
Let us consider a perturbation of the initial conditions. We can imagine that we have
two pendulums A and B with slightly dierent initial conditions. Then the dierence
between the two trajectories is
= f((0), (0))e
t/2Q
. (6.35)
This goes to zero at large times. If we plot ln as a function of time we nd a straight line
with a negative slope. The time-asymptotic motion is not sensitive to initial conditions.
It converges at large times to
no matter what the initial conditions are. The curve
) is called a (periodic) attractor. This is because any perturbed trajectory

will decay exponentially in time to the attractor.
In order to see chaotic behavior we can for example increase Q keeping everything
else xed. We observe that the slope of the line ln = t starts to decrease until at some
value of Q it becomes positive. At this value the variation between the two pendulums
increases exponentially with time. This is the chaotic regime. The value = 0 is the
value where chaos happens. The coecient is called Lyapunov exponent.
The chaotic pendulum is a deterministic system (since it obeys ordinary dierential
equations) but it is not predictable in the sense that given two identical pendulums their
motions will diverge from each other in the chaotic regime if there is the slightest error in
determining their initial conditions. This high sensitivity to initial conditions is known
as the buttery eect and could be taken as the denition of chaos itself.
However we should stress here that the motion of the chaotic pendulum is not random.
This can be seen by inspecting Poincare sections.
6.3.2 Poincare Section and Attractors
The periodic motion of the linear system with period equal to the period of the driving
force is called a period-1 motion. In this motion the trajectory repeats itself exactly
every one single period of the external driving force. This is the only possible motion in
the low amplitude limit.
Generally a period-N motion corresponds to an orbit of the dynamical system which
repeats itself every N periods of the external driving force. These orbits exist in the
non-linear regime of the pendulum.
The Poincare section is dened as follows. We plot in the - phase space only one
point per period of the external driving force. We plot for example (, ) for
D
t = + 2n. (6.36)
The angle is called the Poincare phase and n is an integer. For period-1 motion the
Poincare section consists of one single point. For period-N motion the Poincare section
consists of N points.
Thus in the linear regime if we plot (, ) for
D
t = 2n we get a single point since
the motion is periodic with period equal to that of the driving force. The single point we
get as a Poincare section is also an attractor since all pendulums with almost the same
initial conditions will converge onto it.
In the chaotic regime the Poincare section is an attractor known as strange attractor.
It is a complicated curve which could have fractal structure and all pendulums with
almost the same initial conditions will converge onto it.
6.3.3 Period-Doubling Bifurcations
In the case of the chaotic pendulum we encounter between the linear regime and the
emergence of chaos the so-called period doubling phenomena. In the linear regime the
Poincare section is a point P which corresponds to a period-1 motion with period equal
T
D
= 2/
D
. The or coordinate of this point P will trace a line as we increase
Q while keeping everything xed. We will eventually reach a value Q
1
of Q where this
line bifurcates into two lines. By close inspection we see that at Q
1
the motion becomes
period-2 motion, i.e. the period becomes equal to 2T
D
.
In a motion where the period is T
D
(below Q
1
) we get the same value of each time
t = mT
D
and since we are plotting each time t = 2n/
D
= nT
D
we will get a single
point in the Poincare section. In a motion where the period is 2T
D
(at Q
2
) we get the
same value of each time t = 2mT
D
, i.e. the value of at times t = mT
D
is dierent
and hence we get two points in the Poincare section.
As we increase Q the motion becomes periodic with period equal 4T
D
, then with
period equal 8T
D
and so on. The motion with period 2
N
T
D
is called period-N motion.
The corresponding Poincare section consists of N distinct points.
The diagram of as a function of Q is called a bifurcation diagram. It has a fractal
structure. Let us point out here that normally in ordinary oscillations we get harmonics
with periods equal to the period of the driving force divided by 2
N
. In this case we
obtained in some sense subharmonics with periods equal to the period of the driving
force times 2
N
. This is very characteristic of chaos. In fact chaotic behavior corresponds
to the limit N . In other words chaos is period- (bounded) motion which could
be taken as another denition of chaos.
6.3.4 Feigenbaum Ratio
Let Q
N
be the critical value of Q above which the Nth bifurcation is triggered. In other
words Q
N
is the value where the transition to period-N motion happens. We dene the
Feigenbaum ratio by
F
N
=
Q
N1
Q
N2
Q
N
Q
N1
. (6.37)
It is shown that F
N
F = 4.669 as N . This is a universal ratio called the
Feigenbaum ratio and it characterizes many chaotic systems which suer a transition to
chaos via an innite series of period-doubling bifurcations. The above equation can be
then rewritten as
Q
N
= Q
1
+ (Q
2
Q
1
)
N2
j=0
1
F
j
(6.38)
Let us dene the accumulation point by Q
then
Q
= Q
1
+ (Q
2
Q
1
)
F
F 1
(6.39)
This is where chaos occur. In the bifurcation diagram the chaotic region is a solid black
region.
6.3.5 Spontaneous Symmetry Breaking
The bifurcation process is associated with a deep phenomenon known as spontaneous
symmetry breaking. The rst period-doubling bifurcation corresponds to the breaking of
the symmetry t t +T
D
. The linear regime respects this symmetry. However period-2
motion and in general period-N motions with N > 2 do not respect this symmetry.
There is another kind of spontaneous symmetry breaking which occurs in the chaotic
pendulum and which is associated with a bifurcation diagram. This happens in the region
of period-1 motion and it is the breaking of spatial symmetry or parity . Indeed
there exists solutions of the equations of motion that are either left-favoring or right-
favoring. In other words the pendulums in such solutions spend much of its time in the
regions to the left of the pendulums vertical ( < 0) or to the right of the pendulums
vertical ( > 0). This breaking of left-right symmetry can be achieved by a gradual
increase of Q. We will then reach either the left-favoring solution or the right-favoring
solution starting from a left-right symmetric solution depending on the initial conditions.
The symmetry is also spontaneously broken in period-N motions.
6.4 Lab Problem 8: The Buttery Eect
We consider a pendulum of a mass m and a length l moving under the inuence of the
force of gravity, the force of air resistance and a driving periodic force. Newtons second
law of motion reads
d
2
dt
2
=
g
l
sin q
d
dt
+F
D
sin 2
D
t.
We will always take the angular frequency
_
g/l associated with simple oscillations of
the pendulum equal 1, i.e. l = g. The numerical solution we will consider here is based
on the Euler-Cromer algorithm.
The most important property of a large class of solutions of this dierential equation
is hyper sensitivity to initial conditions known also as the buttery eect which is the
dening characteristic of chaos. For this reason the driven non-linear pendulum is also
known as the chaotic pendulum.
The chaotic pendulum can have two distinct behaviors. In the linear regime the
motion (neglecting the initial transients) is periodic with a period equal to the period of
the external driving force. In the chaotic regime the motion never repeats and any error
even innitesimal in determining the initial conditions will lead to a completely dierent
orbit in the phase space.
(1) Write a code which implements the Euler-Cromer algorithm for the chaotic pendu-
lum. The angle must always be taken between and which can be maintained
as follows
if(
i
.lt. )
i
=
i
2.
(2) We take the values and initial conditions
dt = 0.04s , 2
D
=
2
3
s
1
, q =
1
2
s
1
, N = 1000 2000.
1
= 0.2 radian ,
1
= 0 radian/s.
F
D
= 0 radian/s
2
, F
D
= 0.1 radian/s
2
, F
D
= 1.2 radian/s
2
.
Plot as a function of time. What do you observe for the rst value of F
D
. What
is the period of oscillation for small and large times for the second value of F
D
. Is
the motion periodic for the third value of F
D
.
6.5 Lab Problem 9: Poincar Sections
In the chaotic regime the motion of the pendulum although deterministic is not pre-
dictable. This however does not mean that the motion of the pendulum is random which
can clearly be seen from Poincare sections.
A Poincare section is a curve in the phase space obtained by plotting one point of the
orbit per period of the external drive. Explicitly we plot points (, ) which corresponds
to times t = n/
D
where n is an integer. In the linear regime of the pendulum Poincare
section consists of a single point. Poincare section in the chaotic regime is a curve which
does not depend on the initial conditions thus conrming that the motion is not random
and which may have a fractal structure. As a consequence this curve is called a strange
attractor.
(1) We consider two identical chaotic pendulums A and B with slightly dierent initial
conditions. For example we take
A
1
= 0.2 radian ,
B
1
= 0.201 radian.
The dierence between the two motions can be measured by
i
=
A
i

B
i
.
Compute ln as a function of time for
F
D
= 0.1 radian/s
2
, F
D
= 1.2 radian/s
2
.
What do you observe. Is the two motions identical. What happens for large times.
Is the motion of the pendulum predictable. For the second value of F
D
use
N = 10000 , dt = 0.01s.
(2) Compute the angular velocity as a function of for
F
D
= 0.5 radian/s
2
, F
D
= 1.2 radian/s
2
.
What is the orbit in the phase space for small times and what does it represent.
What is the orbit for large times. Compare between the two pendulums A and B.
Does the orbit for large times depend on the initial conditions.
(3) A Poincare section is obtained numerically by plotting the points (, ) of the orbit
at the times at which the function sin
D
t vanishes. These are the times at which
this function changes sign. This is implemented as follows
if(sin
D
t
i
sin
D
t
i+1
.lt.0)then
write(, )t
i
,
i
,
i
.
Verify that Poincare section in the linear regime is given by a single point in the
phase space. Take and use F
D
= 0.5 radian/s
2
, N = 10
4
10
7
, dt = 0.001s.
Verify that Poincare section in the chaotic regime is also an attractor. Take and use
F
D
= 1.2 radian/s
2
, N = 10
5
, dt = 0.04s. Compare between Poincare sections
of the pendulums A and B. What do you observe and what do you conclude.
6.6 Lab Problem 10: Period Doubling
Among the most important chaotic properties of the driven non-linear pendulum is the
phenomena of period doubling. The periodic orbit with period equal to the period of
the external driving force are called period-1 motion. There exist however other periodic
orbits with periods equal twice, four times and in general 2
N
times the period of the
external driving force. The orbit with period equal 2
N
times the period of the external
driving force is called period-N motion. The period doubling observed in the driven
non-linear pendulum is a new phenomena which belongs to the world of chaos. In the
standard phenomena of mixing the response of a non-linear system to a single frequency
external driving force will contain components with periods equal to the period of the
driving force divided by 2
N
. In other words we get "harmonics" as opposed to the
"subharmonics" we observe in the chaotic pendulum.
For period-N motion we expect that there are N dierent values of the angle for
every value of F
D
. The function = (F
D
) is called a bifurcation diagram. Formally the
transition to chaos occurs at N . In other words chaos is dened as period-innity
motion.
(1) We take the values and initial conditions
l = g , 2
D
=
2
3
s
1
, q =
1
2
s
1
, N = 3000 100000 , dt = 0.01s.
1
= 0.2 radian ,
1
= 0 radian/s.
Determine the period of the motion for
F
D
= 1.35 radian/s
2
, F
D
= 1.44 radian/s
2
, F
D
= 1.465 radian/s
2
.
What happens to the period when we increase F
D
. Does the two second values of
F
D
lie in the linear or chaotic regime of the chaotic pendulum.
(2) Compute the angle as a function of F
D
for the times t which satisfy the condition
2
D
t = 2n. We take F
D
in the interval
F
D
= (1.34 + 0.005k) radian/s
2
, k = 1, ..., 30.
Determine the interval of the external driving force in which the orbits are period-1,
period-2 and period-4 motions.
In this problem it is very important to remove the initial transients before we start
measuring the bifurcation diagram. This can be done as follows. We calculate the
motion for 2N steps but then only consider the last N steps in the computation of
the Poincare section for every value of F
D
.
6.7 Lab Problem 11: Bifurcation Diagrams
Part I The chaotic pendulum is given by the equation
d
2
dt
2
= sin
1
Q
d
dt
+F
D
cos 2
D
t.
In this simulation we take the values F
D
= 1.5 radian/s
2
and 2
D
=
2
3
s
1
. In order to
achieve a better numerical precision we use the second-order Runge-Kutta algorithm.
In the linear regime the orbits are periodic with period equal to the period T
D
of the
external driving force and are symmetric under . There exists other solutions
which are periodic with period equal T
D
but are not symmetric under . In these
solutions the pendulum spends the majority of its time in the region to the left of its
vertical ( < 0) or in the region to the right of its vertical ( > 0).
These symmetry breaking solutions can be described by a bifurcation diagram =
(Q). For every value of the quality factor Q we calculate the Poincare section. We
observe that the Poincare section will bifurcate at some value Q
of Q. Below this value

we get one line whereas above this value we get two lines corresponding to the two
symmetry breaking solutions in which the pendulum spends the majority of its time in
the regions ( > 0) and ( < 0).
(1) Rewrite the code for the chaotic pendulum using Runge-Kutta algorithm.
(2) We take two dierent sets of initial conditions
= 0.0 radian , = 0.0 radian/s.
= 0.0 radian , = 3.0 radian/s .
Study the nature of the orbit for the values Q = 0.5s, Q = 1.24s and Q = 1.3s.
What do you observe.
(3) Plot the bifurcation diagram = (Q) for values of Q in the interval [1.2, 1.3].
What is the value Q
at which the symmetry is spontaneously broken.

Part II As we have seen in the previous simulation period doubling can also be de-
scribed by a bifurcation diagram. This phenomena is also an example of a spontaneous
symmetry breaking. In this case the symmetry is t t +T
D
. Clearly only orbits with
period T
D
are symmetric under this transformation.
Let Q
N
be the value of Q at which the Nth bifurcation occurs. In other words this
is the value at which the orbit goes from being a period-(N 1) motion to a period-N
motion. The Feigenbaum ratio is dened by
F
N
=
Q
N1
Q
N2
Q
N
Q
N1
.
As we approach the chaotic regime, i.e. as N the ratio F
N
converges rapidly to the
constant value F = 4.669. This is a general result which holds for many chaotic systems.
Any dynamical system which can exhibit a transition to chaos via an innite series of
period-doubling bifurcations is characterized by a Feigenbaum ratio which approaches
4.669 as N .
(1) Calculate the orbit and Poincare section for Q = 1.36s. What is the period of the
motion. Is the orbit symmetric under t t + T
D
. Is the orbit symmetric under
.
(2) Plot the bifurcation diagram = (Q) for two dierent sets of initial conditions
for values of Q in the interval [1.3, 1.36]. What is the value Q at which the period
gets doubled. What is the value of Q at which the symmetry t t + T
D
is
spontaneously broken.
(3) In this question we use the initial conditions
= 0.0 radian , = 0.0 radian/s.
Calculate the orbit and Poincare section and plot the bifurcation diagram = (Q)
for values of Q in the interval [1.34, 1.38]. Determine from the bifurcation diagram
the values Q
N
for N = 1, 2, 3, 4, 5. Calculate the Feigenbaum ratio. Calculate the
accumulation point Q
at which the transition to chaos occurs.

7
Molecular Dynamics
7.1 Introduction
In the molecular dynamics approach we attempt to understand the behavior of a classical
many-particle system by simulating the trajectory of each particle in the system. In
practice this can be applied to systems containing 10
9
particles at most. The molecular
dynamics approach is complementary to the more powerful Monte Carlo method. The
Monte Carlo method deals with systems that are in thermal equilibrium with a heat
bath. The molecular dynamics approach on the other hand is useful in studying how fast
in real time a system moves from one microscopic state to another.
We consider a box containing a collection of atoms or molecules. We will use Newtons
second law to calculate the positions and velocities of all the molecules as functions of
time. Some of the questions we can answer with the molecular dynamics approach are:
The melting transition.
The rate of equilibration.
The rate of diusion.
As state above molecular dynamics allows us to understand classical systems. A classical
treatment can be justied as follows. We consider the case of liquid argon as an example.
The energy required to excite an argon atom is of the order of 10eV while the typical
kinetic energy of the center of mass of an argon atom is 0.1eV. Thus a collision between
two argon atoms will not change the electron conguration of either atoms. Hence for
all practical purposes we can ignore the internal structure of argon atoms. Furthermore
the wavelength of an argon atom which is of the order of 10
7
A is much smaller than the
spacing between argon atoms typically of the order of 1A which again justies a classical
treatment.
7.2 The Lennard-Jones Potential
We consider a box containing N argon atoms. For simplicity we will assume that our
argon atoms move in two dimensions. The equations of motion of the ith atom which is
located at the position (x
i
, y
i
) with velocity (v
i,x
, v
i,y
) read
dv
i,x
dt
= a
x,i
,
dx
i
dt
= v
i,x
. (7.1)
dv
i,y
dt
= a
y,i
,
dy
i
dt
= v
i,y
. (7.2)
Each argon atom experience a force from all other argon atoms. In order to calculate
this force we need to determine the interaction potential. We assume that the interaction
potential between any pair of argon atoms depend only on the distance between them.
Let r
ij
and u(r
ij
) be the distance and the interaction potential between atoms i and j.
The total potential is then given by
U =
N1
i=1
N
j=i+1
u(r
ij
). (7.3)
The precise form of u can be calculated from rst principles, i.e. from quantum me-
chanics. However this calculation is very complicated and in most circumstances a phe-
nomenological form of u will be sucient.
For large separations r
ij
the potential u(r
ij
) must be weakly attractive given by
the Van der Walls force which arises from electrostatic interaction between the electric
dipole moments of the two argon atoms. In other words u(r
ij
) for large r
ij
is attractive
due to the mutual polarization of the two atoms. The Van der Walls potential can be
computed from quantum mechanics where it is shown that it varies as 1/r
6
ij
. For small
separations r
ij
the potential u(r
ij
) must become strongly repulsive due to the overlap of
the electron clouds of the two argon atoms. This repulsion known also as core repulsion
is a consequence of Pauli exclusion principle. It is a common practice to choose the
repulsive part of the potential u to be proportional to 1/r
12
ij
. The total potential takes
the form
u(r) = 4
__
r
_
12
r
_
6
_
. (7.4)
This is the Lennard-Jones potential. The parameter is of dimension length while is
of dimension energy. We observe that at r = the potential is 0 identically while for
r > 2.5 the potential approaches zero rapidly. The minimum of the potential occurs at
r = 2
1/6
. The depth of the potential at the minimum is .
The force of atom k on atom i is
f
k,i
=
k,i
u(r
k,i
) =
24
r
ki
_
2
_

r
ki
_
12
_

r
ki
_
6
_
r
ki
. (7.5)
The acceleration of the ith atom is given by
a
x,i
=
1
m
k=i
f
k,i
cos
k,i
=
1
m
k=i
f
k,i
x
i
x
k
r
ki
=
24
m
k=i
x
i
x
k
r
2
ki
_
2
_

r
ki
_
12
_

r
ki
_
6
_
. (7.6)
a
y,i
=
1
m
k=i
f
k,i
sin
k,i
=
1
m
k=i
f
k,i
y
i
y
k
r
ki
=
24
m
k=i
y
i
y
k
r
2
ki
_
2
_

r
ki
_
12
_

r
ki
_
6
_
. (7.7)
7.3 Units, Boundary Conditions and Verlet Algorithm
Reduced Units We choose and as the units of distance and energy respectively.
We also choose the unit of mass to be the mass m of a single argon atom. Everything
else is measured in terms of , and m. For example velocity is measured in units of
(/m)
1/2
and time in units of (/m)
1/2
. The reduced units are given by
= = m = 1. (7.8)
For argon atoms we have the values
= 3.4 10
10
m , = 1.65 10
21
J = 120k
B
J , m = 6.69 10
26
kg. (7.9)
Thus
_
m
= 2.17 10
12
s. (7.10)
Hence a molecular dynamics simulation which runs for 2000 steps with a reduced time
step t = 0.01 corresponds to a total reduced time 2000 0.01 = 20 which is equivalent
to a real time 20(/m)
1/2
= 4.34 10
11
s.
Periodic Boundary Conditions The total number of atoms in a real physical system
is huge of the order of 10
23
. If the system is placed in a box the fraction of atoms of the
system near the walls of the box is negligible compared to the total number of atoms.
In typical simulations the total number of atoms is only of the order of 10
3
10
5
and in
this case the fraction of atoms near the walls is considerable and their eect can not be
neglected.
In order to reduce edge eects we use periodic boundary conditions. In other words
the box is eectively a torus and there are no edges. Let L
x
and L
y
be the lengths of the
box in the x and y directions respectively. If an atom crosses the walls of the box in a
particular direction we add or subtract the length of the box in that direction as follows
if (x > L
x
) then x = x L
x
if (x < 0) then x = x +L
x
. (7.11)
if (y > L
y
) then y = y L
y
if (y < 0) then y = y +L
y
. (7.12)
The maximum separation in the x direction between any two particles is only L
x
/2
whereas the maximum separation in the y direction between any two particles is only
L
y
/2. This can be implemented as follows
if (x
ij
> +L
x
/2) then x
ij
= x
ij
L
x
if (x
ij
< L
x
/2) then x
ij
= x
ij
+L
x
. (7.13)
if (y
ij
> +L
y
/2) then y
ij
= y
ij
L
y
if (y
ij
< L
y
/2) then y
ij
= y
ij
+L
y
. (7.14)
Verlet Algorithm The numerical algorithm we will use is Verlet algorithm. Let us
consider the forward and backward Taylor expansions of a function f given by
f(t
n
+ t) = f(t
n
) + t
df
dt
|
tn
+
1
2
(t)
2
d
2
f
dt
2
|
tn
+
1
6
(t)
3
d
3
f
dt
3
|
tn
+... (7.15)
f(t
n
t) = f(t
n
) t
df
dt
|
tn
+
1
2
(t)
2
d
2
f
dt
2
|
tn

1
6
(t)
3
d
3
f
dt
3
|
tn
+... (7.16)
Adding these expressions we get
f(t
n
+ t) = 2f(t
n
) f(t
n
t) + (t)
2
d
2
f
dt
2
|
tn
+O(t
4
). (7.17)
We remark that the error is proportional to t
4
which is less than the errors in the Euler,
Euler-Cromer and second-order Runge-Kutta methods so this method is more accurate.
We have therefore for the ith atom
x
i,n+1
= 2x
i,n
x
i,n1
+ (t)
2
a
x,i,n
. (7.18)
y
i,n+1
= 2y
i,n
y
i,n1
+ (t)
2
a
y,i,n
. (7.19)
The force and the acceleration are given by
f
k,i,n
=
24
r
ki,n
_
2
_

r
ki,n
_
12
_

r
ki,n
_
6
_
. (7.20)
a
x,i,n
=
1
m
k=i
f
k,i,n
x
i,n
x
k,n
r
ki,n
. (7.21)
a
y,i,n
=
1
m
k=i
f
k,i,n
y
i,n
y
k,n
r
ki,n
. (7.22)
The separation r
ki,n
between the two atoms k and i is given by
r
ki,n
=
_
(x
i,n
x
k,n
)
2
+ (y
i,n
y
k,n
). (7.23)
In the Verlet method it is not necessary to calculate the components dx
i,n
/dt and dy
i,n
/dt
of the velocity. However since the velocity will be needed for other purposes we will also
compute it using the equations
v
x,i,n
=
x
i,n+1
x
i,n1
2t
. (7.24)
v
y,i,n
=
y
i,n+1
y
i,n1
2t
. (7.25)
Let us remark that the Verlet method is not self starting. In other words given the initial
conditions x
i,1
, y
i,1
, v
x,i,1
and v
y,i,1
we need also to know x
i,2
, y
i,2
, v
x,i,2
and v
y,i,2
for the
algorithm to start which can be determined using the Euler method.
7.4 Some Physical Applications
7.4.1 Dilute Gas and Maxwell Distribution
A gas in thermal equilibrium is characterized by a temperature T. Molecular dynamics
allows us to study how a dilute gas approaches equilibrium. The temperature of the gas
can be computed using the molecular dynamics simulations as follows. According to the
equipartition theorem the average thermal energy of each quadratic degree of freedom in
a gas in thermal equilibrium is equal k
B
T/2. In other words
1
2
k
B
T =
1
d
<
1
2
mv
2
> . (7.26)
The average <> can be understood in two dierent but equivalent ways. We can follow
the motion of a single atom and take the time average of its kinetic energy. The same
result can be obtained by taking the average of the kinetic energy over the dierent
atoms. In this latter case we write
1
2
k
B
T =
1
dN
N
i=1
1
2
mv
2
i
. (7.27)
Another way of measuring the temperature T of a dilute gas is through a study of the
distribution of atom velocities. A classical gas in thermal equilibrium obeys Maxwell dis-
tribution. The speed and velocity distributions in two dimensions are given respectively
by
P(v) = C
v
k
B
T
e
mv
2
2k
B
T
. (7.28)
P(v
x
) = C
x
1
k
B
T
e
mv
2
x
2k
B
T
, P(v
y
) = C
y
1
k
B
T
e
mv
2
y
2k
B
T
. (7.29)
Recall that the probability per unit v of nding an atom with speed v is equal P(v)
whereas the probability per unit v
x,y
of nding an atom with velocity v
x,y
is equal P(v
x,y
).
The constants C and C
x,y
are determined from the normalization conditions. There are
peaks in the distributions P(v) and P(v
x,y
). Clearly the temperature is related to the
location of the peak which occurs in P(v). This is given by
k
B
T = mv
2
peak
. (7.30)
7.4.2 The Melting Transition
This is a very important subject which we will discuss at great length in the second lab
problem of this chapter.
7.5 Lab Problem 12: Maxwell Distribution
We consider the motion in two dimensions of N argon atoms in an L L box. The
interaction potential u between any two atoms in the gas separated by a distance r is
given by the Lennard-Jones potential. The numerical algorithm we will use is Verlet
algorithm.
In this problem we will always take L odd and N a perfect square. The lattice spacing
is dened by
a =
L
N
.
Clearly there are N cells of area a a. We choose L and N such that a > 2. For
simplicity we will use reduced units = = m = 1. In order to reduce edge eects we
use periodic boundary conditions. In other words the box is eectively a torus and there
are no edges. Thus the maximum separation in the x direction between any two particles
is only L/2 and similarly the maximum separation in the y direction between any two
particles is only L/2.
The initial positions of the atoms are xed as follows. The atom k =
N(i1)+j will
be placed at the center of the cell with corners (i, j), (i +1, j), (i, j +1) and (i +1, j +1).
Next we perturb in a random way these initial positions by adding random numbers in
the interval [a/4, +a/4] to the x and y coordinates of the atoms. The initial velocities
can be chosen in random directions with a speed equal v
0
for all atoms.
(1) Write a molecular dynamics code along the above lines. Take L = 15, N = 25,
t = 0.02, Time = 500 and v
0
= 1. As a rst test verify that the total energy is
conserved. Plot the trajectories of the atoms. What do you observe.
(2) As a second test we propose to measure the temperature by observing how the gas
approaches equilibrium. Use the equipartition theorem
k
B
T =
m
2N
N
i=1
(v
2
i,x
+v
2
i,y
).
Plot T as a function of time. Take Time = 1000 1500. What is the temperature
of the gas at equilibrium.
(3) Compute the speed distribution of the argon atoms by constructing an appropriate
histogram as follows. We take the value Time = 2000. We consider the speeds of
all particles at all times. There are Time N values of the speed in this sample.
Construct the histogram for this sample by 1) nding the maximum and minimum,
2) dividing the interval into bins, 3) determining the number of times a given value
of the speed falls in a bin and (4) properly normalizing the distribution. Compare
with the Mawell distribution
P
Maxwell
(v) = C
v
2
k
B
T
e
mv
2
2k
B
T
.
Deduce the temperature from the peak of the distribution given by k
B
T = mv
2
peak
.
Compare with the value of the temperature obtained from the equipartition theo-
rem. What happens if we increase the initial speed.
7.6 Lab Problem 13: Melting Transition
We would like to study the melting transition. First we need to establish the correct
conditions for a solid phase. Clearly the temperature must be suciently low and the
density must be suciently high. To make the temperature as low as possible we will
start with all particles at rest. In order to obatin maximum attraction between atoms we
choose a low density of approximately one particle per unit reduced area. In particular
we choose N = 16 and L = 4.
(1) Show that with these conditions you obtain a crystalline solid with a triangular
lattice.
(2) In order to observe melting we must heat up the system. This can be achieved by
increasing the kinetic energy of the atoms by hand. A convenient way of doing this
is to rescale the current and previous positions of the atoms periodically (say every
1000 steps) as follows
hh = int(n/1000)
if (hh 1000.eq.n) then
x(i, n) = x(i, n + 1) R(x(i, n + 1) x(i, n))
y(i, n) = y(i, n + 1) R(y(i, n + 1) y(i, n))
endif.
This procedure will rescale the velocity by the amount R. We choose R = 1.5.
Verify that we will indeed reach the melting transition by means of this method.
What happens to the energy and the temperature.
8
Pseudo Random Numbers and Random Walks
8.1 Random Numbers
A sequence of numbers r
1
, r
2
,... is called random if there are no correlations between
the numbers. The sequence is called uniform if all numbers have an equal probability
to occur. More precisely let the probability that a number r
i
in the sequence occurs
between r and r + dr be P(r)dr where P(r) is the probability distribution. A uniform
distribution corresponds P(r) = constant.
Most random number generators on computers generate uniform distributions be-
tween 0 and 1. These are sequences of pseudo random numbers since given r
i
and its
preceding elements we can compute r
i+1
. Therefore these sequences are not really ran-
dom and correlations among the numbers of the sequence exist. True random numbers
can be found in tables of random numbers determined during say radioactive decay or
other naturally occurring random physical phenomena.
8.1.1 Linear Congruent or Power Residue Method
In this method we generate a set of k random numbers r
1
,r
2
,...,r
k
in the interval [0, M1]
as follows. Given a random number r
i1
we generate the next random number r
i
by the
rule
r
i
= (ar
i1
+c) mod M = remainder
_
ar
i1
+c
M
_
. (8.1)
The notation y = z mod M means that we subtract M from z until 0yM 1. The
rst random number r
1
is supplied by the user and it is called the seed. Also supplied
are the multiplier a, the increment c and the modulus M. The remainder is a built-in
function in most computer languages. The largest possible integer number generated by
the above rule is M 1. Thus the maximum possible period is M, i.e kM. In general
the period k depends on a, c and M. To get a uniform sequence in the interval [0, 1] we
divide by M 1.
Let us take the following example a = 4,c = 1 and M = 9 with seed r
1
= 3. We get
a sequence of length 9 given by
3, 4, 8, 6, 7, 2, 0, 1, 5. (8.2)
After the last number 5 we get 3 and therefore the sequence will repeat. In this case the
period is M = 9.
It is clear that we need to choose the parameters a, c and M and the seed r
1
with
care so that we get the longest sequence of pseudo random numbers. The maximum
possible period depends on the size of the computer word. A 32bit machine may use
M = 2
31
= 2 10
9
. The numbers generated by (8.1) are random only in the sense that
they are evenly distributed over their range. Equation (8.1) is related to the logistic map
which is known to exhibit chaotic behaviour. Although chaos is deterministic it looks
random. In the same way although equation (8.1) is deterministic the numbers generated
by it look random. This is the reason why they are called pseudo random numbers.
8.1.2 Statistical Tests of Randomness
Period : The rst obvious test is to verify that the random number generator has a
suciently long period for a given problem. We can use the random number generator to
plot the position of a random walker. Clearly the plot will repeat itself when the period
is reached.
Uniformity : The kth moment of the random number distribution is
< x
k
i
>=
1
N
N
i=1
x
k
i
. (8.3)
Let P(x) be the probability distribution of the random numbers. Then
< x
k
i
>=
_
1
0
dx x
k
P(x) +O(
1
N
). (8.4)
For a uniform distribution P(x) = 1 we must have
< x
k
i
>=
1
k + 1
+O(
1
N
). (8.5)
In the words
N
_
1
N
N
i=1
x
k
i

1
k + 1
_
= O(1). (8.6)
This is a test of uniformity as well as of randomness. To be more precise if < x
k
i
>
is equal to 1/(k + 1) then we can infer that the distribution is uniform whereas if the
deviation varies as 1/
N then we can infer that the distribution is random.

A direct test of uniformity is to divide the unit interval into K equal subintevals
(bins) and place each random number in one of these bins. For a uniform distribution
we must obtain N/K numbers in each bin where N is the number of generated random
numbers.
Chi-Square Statistic : In the above test there will be statistical uctuations about the
ideal value N/K for each bin. The question is whether or not these uctuations are
consistent with the laws of statistics. The answer is based on the so-called chi-square
statistic dened by
2
m
=
K
i=1
(N
i
n
ideal
)
2
n
ideal
. (8.7)
In the above denition N
i
is the number of random numbers which fall into bin i and
n
ideal
is the expected number of random numbers in each bin.
The probability of nding any particular value
2
which is less than
2
m
is found to
be proportional to the incomplete gamma function (/2,
2
m
/2) where is the number
of degrees of freedom given by = K 1. We have
P(
2

2
m
) =
(/2,
2
m
/2)
(/2)
P(/2,
2
m
/2). (8.8)
The most likely value of
2
m
, for some xed number of degrees of freedom , corresponds
to the value P(/2,
2
m
/2) = 0.5. In other words in half of the measurements (bin tests),
for some xed number of degrees of freedom , the chi-square statistic predicts that we
must nd a value of
2
m
smaller than the maximum.
Randomness : Let r
1
, r
2
,...,r
N
be a sequence of random numbers. A very eective test
of randomness is to make a scatterplot of (x
i
= r
2i
, y
i
= r
2i+1
) for many i. There must
be no regularity in the plot otherwise the sequence is not random.
Short-Term Correlations : Let us dene the autocorrelation function
C(j) =
< x
i
x
i+j
> < x
i
>< x
i+j
>
< x
i
x
i
> < x
i
>
2
=
< x
i
x
i+j
> < x
i
>
2
< x
i
x
i
> < x
i
>
2
, j = 1, 2, ... (8.9)
In the above equation we have used the fact that < x
i+j
>=< x
i
> for a large sample,
i.e. the choice of the origin of the sequence is irrelevant in that case and
< x
i
x
i+j
>=
1
N j
Nj
i=1
x
i
x
i+j
. (8.10)
Again if x
i
and x
i+j
are independent random numbers which are distributed with the
joint probability distribution P(x
i
, x
i+j
) then
< x
i
x
i+j
>
_
1
0
dx
_
1
0
dyxyP(x, y). (8.11)
We have clearly assumed that N is large. For a uniform distribution, viz P(x, y) = 1 we
get
< x
i
x
i+j
>
1
4
. (8.12)
For a random distrubution the deviation from this result is of order 1/
N. Hence in the
case that the random numbers are not correlated we have
C(j) = 0. (8.13)
8.2 Random Systems
Both quantum and statistical physics deal with systems that are random or stochastic.
These are non deterministic systems as opposed to classical systems. The dynamics of
a deterministic system is given by a unique solution to the equations of motion which
describes the physics of the system at all times.
We take the case of the diusion of uid molecules. For example the motion of dust
particles in the atmosphere, the motion of perfume molecules in the air or the motion of
milk molecules in a coee. These are all cases of a Brownian motion.
In the case of a drop of milk in a coee the white mass of the drop of milk will
slowly spread until the coee takes on a uniform brown color. At the molecular level
each milk molecule collides with molecules in the coee. Clearly it will change direction
so frequently that its motion will appear random. This trajectory can be described by a
random walk. This is a system in which each milk molecule moves one step at a time in
any direction with equal probability.
The trajectory of a dust, perfume or milk molecule is not really random since it can
in principle be computed by solving Newtons equations of motion for all molecules which
then allows us to know the evolution of the system in time. Although this is possible
in principle it will not be feasible in practice. The random walk is thus eectively an
approximation. However the large number of molecules and collisions in the system
makes the random walk a very good approximation.
8.2.1 Random Walks
Let us consider a one dimensional random walk. It can take steps of lenght unity along
a line. It begins at s
0
= 0 and the rst step is chosen randomly to be either to the left
or to right with equal probabilities. In other words there is a 50 per cent chance that
the walker moves to the point s
1
= +1 and a 50 per cent chance that it moves to the
point s
1
= 1. Next the walker will again move either to the right or to the left from
the point s
1
to the point s
2
with equal probabilities. This process will be repeated N
times and we get the position of the walker x
N
as a function of the step number N. In
the motion of a molecule in a solution the time between steps is a constant and hence
the step number N is proportional to time. Therefore x
N
is the position of the walker
as a function of time.
In general a one-dimensional random walker can move to the right with probability p
and to the left with probability q = 1 p with steps of equal lenght a. The direction of
each step is independent of the previous one. The displacement or position of the walker
after N steps is
x
N
=
N
i=1
s
i
. (8.14)
The walker for p = q = 1/2 can be generated by ipping a coin N times. The position
is increased by a for heads and decreased by a for tails.
Averaging over many walks each consisting of N steps we get
< x
N
>=
N
i=1
< s
i
>= N < s > . (8.15)
In above we have used the fact that the average over every step is the same given by
< s
i
>=< s >= p(a) +q(a) = (p q)a. (8.16)
For p = q = 1/2 we get < x
N
>= 0. A better measure of the walk is given by
x
2
N
=
_
N
i=1
s
i
_
2
. (8.17)
The mean square net displacement x
2
is dened by
x
2
=< (x
N
< x
N
>)
2
>=< x
2
N
> < x
N
>
2
. (8.18)
We compute
x
2
=
N
i=1
N
j=1
< (s
i
< s >)(s
j
< s >) >
=
N
i=j=1
< (s
i
< s >)(s
j
< s >) > +
N
i=1
< (s
i
< s >)
2
> . (8.19)
In the rst term since i = j we have < (s
i
< s >)(s
j
< s >) >=< (s
i
< s >) ><
(s
j
< s >) >. But < (s
i
< s >) >= 0. Thus
x
2
=
N
i=1
< (s
i
< s >)
2
>
= N(< s
2
i
> < s >
2
>)
= N(a
2
(p q)
2
a
2
)
= 4Npqa
2
. (8.20)
For p = q = 1/2 and a = 1 we get
< x
2
N
> = N. (8.21)
The main point is that since N is proportional to time we have < x
2
N
> t. This is an
example of a diusive behaviour.
8.2.2 Diusion Equation
The random walk is successful in simulating many physical systems because it is related
to the solutions of the diusion equation. To see this we start from the probability
P(i, N) that the random walker is at site s
i
after N steps. This is given by
P(i, N) =
1
2
_
P(i + 1, N 1) +P(i 1, N 1)
_
. (8.22)
Let be the time between steps and a the lattice spacing. Then t = N and x = ia.
Also we dene P(x, t) = P(i, N)/a. We get
P(x, t) =
1
2
_
P(x +a, t ) +P(x a, t )
_
. (8.23)
Let us rewrite this equation as
1
_
P(x, t) P(x, t )
_
=
a
2
2
_
P(x +a, t ) 2P(x, t ) +P(x a, t )
_
1
a
2
.
(8.24)
In the limit a 0, 0 with the ratio D = a
2
/2 kept xed we obtain the equation
P(x, t)
t
= D
2
P(x, t)
x
2
. (8.25)
This is the diusion equation. Generalization to 3dimensions is
P(x, y, z, t)
t
= D
2
P(x, y, z, t). (8.26)
A particular solution of (8.25) is given by
P(x, t) =
1
x
2
2
2
, =
2Dt. (8.27)
In other words the spatial distribution of the diusing molecules is always a gaussian
with half-width increasing with time as
t.
The average of any function f of x is given by
< f(x, t) >=
_
f(x)P(x, t)dx. (8.28)
Let us multiply both sides of (8.25) by f(x) and then integrate over x, viz
_
f(x)
P(x, t)
t
dx = D
_
f(x)
2
P(x, t)
x
2
dx. (8.29)
Clearly
_
f(x)
P(x, t)
t
dx =
_

t
_
f(x)P(x, t)
_
dx =
d
dt
_
f(x)P(x, t)dx =
d
dt
< f(x) > . (8.30)
Thus
d
dt
< f(x) > = D
_
f(x)
2
P(x, t)
x
2
dx
= D
_
f(x)
P(x, t)
x
_
|
x=+
x=
D
_
f(x)
x
P(x, t)
x
dx. (8.31)
We have P(x = , t) = 0 and also all spatial derivatives are zero at x = . We then
get
d
dt
< f(x) > = D
_
f(x)
x
P(x, t)
x
dx. (8.32)
Let us choose f(x) = x. Then
d
dt
< x > = D
_
P(x, t)
x
dx = 0. (8.33)
In other words < x >= constant and since x = 0 at t = 0 we must have constant = 0.
Thus
< x >= 0. (8.34)
Let us next choose f(x) = x
2
. Then
d
dt
< x
2
> = 2D
_
x
P(x, t)
x
dx
= 2D. (8.35)
Hence
< x
2
> = 2Dt. (8.36)
This is the diusive behaviour we have observed in the random walk problem.
8.3 The Random Number Generators RAN 0, 1, 2
Linear congruential generators are of the form
r
i
= (ar
i1
+c) mod M. (8.37)
For c > 0 the linear congruential generators are called mixed. They are denoted by
LCG(a, c, M). The random numbers generated with LCG(a, c, M) are in the range
[0, M 1].
For c = 0 the linear congruential generators are called multiplicative. They are
denoted by MLCG(a, M). The random numbers generated with MLCG(a, M) are in the
range [1, M 1].
In the case that a is a primitive root modulo M and M is a prime the period of the
generator is M1. A number a is a primitive root modulo M means that for any integer
n such that gcd(n, M) = 1 there exists a k such that a
k
= n mod M.
An example of MLCG is RAN0 due to Park and Miller which is used extensively on
IBM computers. In this case
a = 16807 = 7
5
, M = 2
31
1. (8.38)
The period of this generator is not very long given by
period = 2
31
2 2.15 10
9
. (8.39)
This generator can not be implemented directly in a high level language because of
integer overow. Indeed the product of a and M 1 exceeds the maximum value for
a 32bit integer. Assemply language implementation using 64bit product register is
straightforward but not portable.
A better solution is given by Schrages algorithm. This algorithm allows the multipli-
cation of two 32bit integers without using any intermediate numbers which are larger
than 32 bits. To see how this works explicitly we factor M as
M = aq +r. (8.40)
r = M mod a , q = [
M
r
]. (8.41)
In the above equation [ ] denotes integer part. Remark that
r = M mod a = M [
M
a
]a. (8.42)
Thus by denition r < a. We will also demand that r < q and hence
r
qa
<< 1. (8.43)
We have also
X
i+1
= aX
i
mod M = aX
i
[
aX
i
M
]M
= aX
i
[
aX
i
aq +r
]M. (8.44)
We compute
aX
i
aq +r
=
X
i
q +
r
a
=
X
i
q
1
1 +
r
qa
=
X
i
q
(1
r
qa
)
=
X
i
q

X
i
aq
r
q
. (8.45)
Clearly
X
i
aq
=
X
i
M r

X
i
M
< 1. (8.46)
Hence
[
aX
i
M
] = [
X
i
q
], (8.47)
if neglecting = (rX
i
)/(aq
2
) does not aect the integer part of aX
i
/M and
[
aX
i
M
] = [
X
i
q
] 1, (8.48)
if neglecting does aect the integer part of aX
i
/M. Therefore we get
X
i+1
= aX
i
[
aX
i
M
](aq +r)
= a(X
i
[
aX
i
M
]q) [
aX
i
M
]r (8.49)
= a(X
i
[
X
i
q
]q) [
X
i
q
]r (8.50)
= a(X
i
mod q) [
X
i
q
]r, (8.51)
if
a(X
i
mod q) [
X
i
q
]r 0. (8.52)
Also
X
i+1
= aX
i
[
aX
i
M
](aq +r)
= a(X
i
[
aX
i
M
]q) [
aX
i
M
]r (8.53)
= a(X
i
[
X
i
q
]q +q) [
X
i
q
]r +r (8.54)
= a(X
i
mod q) [
X
i
q
]r +M, (8.55)
if
a(X
i
mod q) [
X
i
q
]r < 0. (8.56)
The generator RAN0 contains serial correlations. For example Ddimensional vectors
(x
1
, ..., x
D
), (x
D+1
, ..., x
2D
),...which are obtained by successive calls of RAN0 will lie on
a small number of parallel (D 1)dimensional hyperplanes. Roughly there will be
M
1/D
such hyperplanes. In particular successive points (x
i
, x
i+1
) when binned into a
2dimensional plane for i = 1, ..., N will result in a distribution which fails the
2
test
for N 10
7
which is much less than the period M 1.
The RAN1 is devised so that the correlations found in RAN0 is removed using the
Bays-Durham algorithm. The Bays-Durham algorithm shues the sequence to remove
low-order serial correlations. In other words it changes the order of the numbers so
that the sequence is not dependent on order and a given number is not correlated with
previous numbers. More precisely the jth random number is output not on the jth call
but on a randomized later call which is on average the j + 32th call on .
The RAN2 is an improvement over RAN1 and RAN0 due to LEcuyer. It uses two
sequences with dierent periods so as to obtain a new sequence with a larger period
equal to the least common multiple of the two periods. In this algorithm we add the two
sequences modulo the modulus M of one of them. In order to avoid overow we subtract
rather than add and if the result is negative we add M1 so as to wrap around into the
inetrval [0, M 1]. LEcuyer uses the two sequences
M
1
= 2147483563 , a
1
= 40014 , q
1
= 53668 , r
1
= 12211. (8.57)
M
2
= 2147483399 , a
2
= 40692 , q
2
= 52774 , r
2
= 3791. (8.58)
The period is 2.3 10
18
. Let us also point out that RAN2 uses Bays-Durham algorithm
in order to implement an additional shue.
We conclude this section by discussing another generator based on the linear congru-
ential method which is the famous random number generator RAND given by
RAND = LCG(69069, 1, 2
32
). (8.59)
The period of this generator is 2
32
and lattice structure is present for higher dimensions
D 6.
8.4 Lab Problem 14: Random Numbers
Part I We consider a linear congruential pseudo-random number generator given by
r
i+1
= remainder
_
ar
i
+c
M
_
.
We take the values
a = 899, c = 0, M = 32768, r
1
= 12 good
a = 57, c = 1, M = 256, r
1
= 10 , bad.
The function remainder is implemented in Fortran by
remainder
a
b
= mod(a, b).
(1) Compute the sequence of the random numbers r
i
obtained using the above param-
eters. Plot r
i
as a function of i. Construct a scatterplot (x
i
= r
2i
, y
i
= r
2i+1
).
(2) Compute the average of the random numbers. What do you observe.
(3) Let N be the number of generated random numbers. Compute the correlation
functions dened by
sum
1
(k) =
1
N k
Nk
i=1
x
i
x
i+k
.
sum
2
=
sum
1
(k) < x
i
>
2
sum
1
(0) < x
i
>
2
.
What is the behavior of these functions as a function of k.
(4) Compute the period of the above generators.
Part II We take N random numbers in the interval [0, 1] which we divide into K bins
of length = 1/K. Let N
i
be the number of random numbers which fall in the ith bin.
For a uniform sequence of random numbers the number of random numbers in each bin
is n
ideal
= N/K.
(1) Verify this result for the generator rand found in the standard Fortran library
with seed given by seed = 32768. We take K = 10 and N = 1000. Plot N
i
as a
function of the position x
i
of the ith bin.
(2) The number of degrees of freedom is = K 1. The most probable value of the
chi-square statistics
2
is . Verify this result for a total number of bin tests equal
L = 1000 and K = 11. Each time calculate the number of times L
i
in the L = 1000
bin tests we get a specic value of
2
. Plot L
i
as a function of
2
. What do you
observe.
8.5 Lab Problem 15: Random Walks
Part I We consider the motion of a random walker in one dimension. The walker can
move with a step s
i
= a to the right with a probability p or with a step s
i
= a to the
left with a probability q = 1 p. After N steps the position of the walker is x
N
=
i
s
i
.
We take
p = q =
1
2
, a = 1.
In order to simulate the motion of a random walker we need a generator of random
numbers. In this problem we work with the generator rand found in the standard
Fortran library. We call this generator as follows
call srand(seed)
rand()
The motion of the random walker is implemented with the code
if (rand() < p) then
x
N
= x
N
+a
else
x
N
= x
N
a
endif.
(1) Compute the positions x
i
of three dierent random walkers as functions of the step
number i. We take i = 1, 100. Plot the three trajectories.
(2) We consider now the motion of K = 500 random walkers. Compute the averages
< x
N
>=
1
K
K
i=1
x
(i)
N
, < x
2
N
>=
1
K
K
i=1
(x
(i)
N
)
2
.
In the above equations x
(i)
N
is the position of the ith random walker after N steps.
Study the behavior of these averages as a function of N. Compare with the theo-
retical predictions.
Part II (optional) We consider next a random walker in two dimensions on an innite
lattice of points. From any point (i, j) on the lattice the walker can reach one of the 4
possible nearest neighbor sites (i+1, j), (i1, j), (i, j +1) and (i, j 1) with probabilities
p
x
, q
x
, p
y
and q
y
respectively such that p
x
+ q
x
+ p
y
+ q
y
= 1. For simplicity we will
assume that p
x
= q
x
= p
y
= q
y
= 0.25.
(1) Compute the averages < r
N
> and < r
2
N
> as function of the number of steps
N for a collection of L = 500 two dimensional random walkers. We consider the
values N = 10, ..., 1000.
9
Monte Carlo Integration
9.1 Numerical Integration
9.1.1 Rectangular Approximation Revisted
As usual let us start with something simple. The approximation of one-dimensional
integrals by means of the rectangular approximation. This is a topic we have already
discussed before.
Let us then begin by recalling how the rectangular approximation of one dimensional
integrals works. We consider the integral
F =
_
b
a
f(x)dx. (9.1)
We discretize the xinterval so that we end up with N equal small intervals of lenght
x, viz
x
n
= x
0
+nx , x =
b a
N
(9.2)
Clearly x
0
= a and x
N
= b. Riemann denition of the integral is given by the following
limit
F = lim x
N1
n=0
f(x
n
) , x 0 , N , b a = xed. (9.3)
The rst approximation which can be made is to simply drop the limit. We get the
so-called rectangular approximation given by
F
N
= x
N1
n=0
f(x
n
). (9.4)
The error can be computed as follows. We start with the Taylor expansion
f(x) = f(x
n
) + (x x
n
)f
(1)
(x
n
) +
1
2!
(x x
n
)
2
f
(2)
(x
n
) +... (9.5)
Thus
_
x
n+1
xn
dx f(x) = f(x
n
)x +
1
2!
f
(1)
(x
n
)(x)
2
+
1
3!
f
(2)
(x
n
)(x)
3
+... (9.6)
n
, x
n+1
] is
_
x
n+1
xn
dx f(x) f(x
n
)x =
1
2!
f
(1)
(x
n
)(x)
2
+
1
3!
f
(2)
(x
n
)(x)
3
+... (9.7)
2
. But we have N subintervals. Thus the total error is of order 1/N.
9.1.2 Midpoint Approximation of Multidimensional Integrals
Let us start with the two dimensional integral
F =
_
R
dx dy f(x, y). (9.8)
R is the domain of integration. In order to give the midpoint approximation of this
integral we imagine a rectangle of sides x
b
x
a
and y
b
y
a
which encloses the region R
and we divide it into squares of lenght h. The points in the x/y direction are
x
i
= x
a
+ (i
1
2
)h , i = 1, ..., n
x
. (9.9)
y
i
= y
a
+ (i
1
2
)h , i = 1, ..., n
y
. (9.10)
The number of points in the x/y direction are
n
x
=
x
b
x
a
h
, n
y
=
y
b
y
a
h
. (9.11)
The number of cells is therefore
n = n
x
n
y
=
(x
b
x
a
)(y
b
y
a
)
h
2
. (9.12)
The integral is then approximated by
F = h
2
nx
i=1
ny
j=1
f(x
i
, y
j
)H(x
i
, y
j
). (9.13)
The Heaviside function is dened by
H(x
i
, y
j
) = 1 if (x
i
, y
j
) R otherwise H(x
i
, y
j
) = 0. (9.14)
The generalization to many dimensions is straightforward. We get
F = h
d
n
1
i
1
=1
...
n
d
i
d
=1
f(x
i
1
1
, ..., x
i
d
d
)H(x
i
1
1
, ..., x
i
d
d
). (9.15)
The meaning of the dierent symbols is obvious.
The midpoint approximation is an improvement over the rectangular approximation.
To see this let us consider a one dimensional integral
F =
_
R
dx f(x). (9.16)
The midpoint approximation reads in this case as follows
F = h
nx
i=1
f(x
i
)H(x
i
) = h
nx
i=1
f(x
i
). (9.17)
Let us say that we have n
x
intervals [x
i
, x
i+1
] with x
0
= a and x
i
= x
a
+ (i 0.5)h,
i = 1, ..., n
x
1. The term hf(x
i+1
) is associated with the interval [x
i
, x
i+1
]. It is clear
that we can write this approximation as
F = h
nx1
i=0
f(
x
i
+x
i+1
2
) , x
i
= x
a
+ih. (9.18)
i
, x
i+1
] is given by
_
x
i+1
x
i
f(x) dx f(
x
i
+x
i+1
2
)x =
1
24
f
(x
i
)(x)
3
+... (9.19)
The total error is thereore 1/n
2
x
as opposed to the 1/n
x
of the rectangular approximation.
Let us do this in two dimensions. We write the error as
_
x
i+1
x
i
_
y
j+1
y
j
f(x, y) dx dy f(
x
i
+x
i+1
2
,
y
j
+y
j+1
2
)xy (9.20)
As usual we use Taylor series in the form
f(x, y) = f(x
i
, y
j
) +f
x
(x
i
, y
j
)(x x
i
) +f
y
(x
i
, y
j
)(y y
j
) +
1
2
f
x
(x
i
, y
j
)(x x
i
)
2
+
1
2
f
y
(x
i
, y
j
)(y y
j
)
2
+f
xy
(x
i
, y
j
)(x x
i
)(y y
j
) +... (9.21)
We nd
_
x
i+1
x
i
_
y
j+1
y
j
f(x, y) dx dy f(
x
i
+x
i+1
2
,
y
j
+y
j+1
2
)xy =
1
24
f
x
(x
i
, y
j
)(x)
3
y +
1
24
f
y
(x
i
, y
j
)x(y)
3
+ ... (9.22)
Since x = y = h. The individual error is proportional to h
4
. The total error is
nh
4
where n = n
x
n
y
. Since n is proportional to 1/h
2
, the total error in dimension
two is proportional to h
2
or equivalently to 1/n. As we have already seen the same
method led to an error proportional to 1/n
2
in dimension one. Thus as we increase the
number of dimensions the error becomes worse. If in one dimension the error behaves
as 1/n
a
then in dimension d it will behave as 1/n
a
d
. In other words classical numerical
integration methods become impractical at suciently higher dimensions (which is the
case of quantum mechanics and statistical mechanics).
9.1.3 Spheres and Balls in d Dimensions
The volume of a ball of radius R in d dimensions is given by
V
d
=
_
x
2
1
+...+x
2
d
R
2
dx
1
...dx
d
=
_
x
2
1
+...+x
2
d
R
2
r
d1
dr d
d1
=
R
d
d
_
d
d1
=
R
d
d
2
d
2
(
d
2
)
. (9.23)
The surface of a sphere of radius R in d dimensions is similarly given by
S
d1
=
_
x
2
1
+...+x
2
d
=R
2
dx
1
...dx
d
= R
d1
2
d
2
(
d
2
)
. (9.24)
Here are some properties of the gamma function
(1) = 1 , (
1
2
) =
, (n + 1) = n(n). (9.25)
In order to compute numerically the volume of the ball in any dimension d we need a
recursion formula which relates the volume of the ball in d dimensions to the volume of
the ball in d 1 dimensions. The derivation goes as follows
V
d
=
_
+R
R
dx
d
_
x
2
1
+...+x
2
d1
R
2
x
2
d
dx
1
...dx
d1
=
_
+R
R
dx
d
_
R
2
x
2
d
0
r
d2
dr
_
d
d2
=
V
d1
R
d1
_
+R
R
dx
d
(R
2
x
2
d
)
d1
2
. (9.26)
At each dimension d we are thus required to compute only the remaining integral over
x
d
using, for instance, the midpoint approximation while the volume V
d1
is determined
in the previous recursion step. The starting point of the recursion process, for example
the volume in d = 2, can be determined also using the midpoint approximation. As we
will see in the lab problems this numerical calculation is very demanding with signicant
errors compared with the Monte Carlo method.
9.2 Monte Carlo Integration: Simple Sampling
Let us start with the one dimensional integral
F =
_
b
a
dx f(x). (9.27)
A Monte Carlo method is any procedure which uses (pseudo) random numbers to com-
pute or estimate the above integral. In the following we will describe two very simple
Monte Carlo methods based on simple sampling which give an approximate value for
this integral. As we progress we will be able to give more sophisticated Monte Carlo
methods. First we start with the sampling (hit or miss) method then we go on to the
sample mean method.
9.2.1 Sampling (Hit or Miss) Method
This method consists of the following three main steps:
We imagine a rectangle of width b a and height h such that h is greater than the
maximum value of f(x), i.e the function is within the boundaries of the rectangle.
To estimate the value F of the integral we choose n pairs of uniform random
numbers (x
i
, y
i
) where a x
i
b and 0 y
i
h.
Then we evaluate the function f at the points x
i
. Let n
in
be the number of random
points (x
i
, y
i
) such that y
i
f(x
i
). The value F of the integral is given by
F = A
n
in
n
, A = h(b a). (9.28)
9.2.2 Sample Mean Method
We start from the mean-value theorem of calculus, viz
F =
_
b
a
dx f(x) = (b a) < f > . (9.29)
< f > is the average value of the function f(x) in the range a x b. The sample
mean method estimates the average < f > as follows:
We choose n random points x
i
from the interval [a, b] which are distributed uni-
formly.
We compute the values of the function f(x) at these point.
We take their average. In other words
F = (b a)
1
n
n
i=1
f(x
i
). (9.30)
This is formally the same as the rectangular approximation. The only dierence is that
here the points x
i
are chosen randomly from the interval [a, b] whereas the points in
the rectangular approximation are chosen with equal spacing. For lower dimensional
integrals the rectangular approximation is more accurate whereas for higher dimensional
integrals the sample mean method becomes more accurate.
9.2.3 Sample Mean Method in Higher Dimensions
We start with the two dimensional integral
F =
_
R
dx dy f(x, y). (9.31)
Again we consider a rectangle of sides y
b
y
a
and x
b
x
a
which encloses the region R.
The Monte carlo sample mean method yields the approximation
F = A
1
n
n
i=1
f(x
i
, y
i
)H(x
i
, y
i
). (9.32)
The points x
i
are random and uniformly distributed in the interval [x
a
, x
b
] whereas the
points y
i
are random and uniformly distributed in the interval [y
a
, y
b
]. A is the areas of
the rectangle, i.e A = (x
b
x
a
)(y
b
y
a
). The Heaviside function is dened by
H(x
i
, y
i
) = 1 if (x
i
, y
i
) R otherwise H(x
i
, y
i
) = 0. (9.33)
Generalization to higher dimensions is obvious. For example in three dimensions we
would have
F =
_
R
dx dy dz f(x, y, z) F = V
1
n
n
i=1
f(x
i
, y
i
, z
i
)H(x
i
, y
i
, z
i
). (9.34)
V is the volume of the parallelepiped which encloses the three dimensional region R.
9.3 The Central Limit Theorem
Let p(x) be a probability distribution function. We generate (or measure) n values x
i
of
a certain variable x according to the probability distribution function p(x). The average
y
1
=< x
i
> is given by
y
1
=< x
i
>=
1
n
n
i=1
x
i
p(x
i
). (9.35)
We repeat this measurement N times thus obtaining N averages y
1
, y
2
,...,y
N
. The mean
z of the averages y
i
is
z =
1
N
N
i=1
y
i
. (9.36)
The question we want to answer is: what is the probability distribution function of z.
Clearly the probability of obtaining a particular value z is the product of the prob-
abilities of obtaining the individual averages y
i
(which are assumed to be independent)
with the constraint that the average of y
i
is z.
Let p(y) be the probability distribution function of the average y and let P(z) be the
probability distribution of the average z of the averages. We can then write P(z) as
P(z) =
_
dy
1
...
_
dy
N
p(y
1
)... p(y
N
)(z
y
1
+... +y
N
N
). (9.37)
The delta function expresses the constraint that z is the average of y
i
. The delta function
can be written as
(z
y
1
+... +y
N
N
) =
1
2
_
dqe
iq(z
y
1
+...+y
N
N
)
. (9.38)
Let be the actual average of y
i
, i.e.
=< y
i
>=
_
dy p(y)y. (9.39)
We write
P(z) =
1
2
_
dqe
iq(z)
_
dy
1
p(y
1
)e
iq
N
(y
1
)
...
_
dy
N
p(y
N
)e
iq
N
(y
N
)
=
1
2
_
dqe
iq(z)
_ _
dy p(y)e
iq
N
(y)
_
N
. (9.40)
But
_
dy p(y)e
iq
N
(y)
=
_
dy p(y)
_
1 +
iq
N
( y)
q
2
( y)
2
2N
2
+...
_
= 1
q
2
2
2N
2
+... (9.41)
We have used
_
dy p(y)( y)
2
=< y
2
> < y >
2
=
2
. (9.42)
Hence
P(z) =
1
2
_
dqe
iq(z)
e
q
2
2
2N
=
1
2
e
N
2
2
(z)
2
_
dqe
2
2N
(q
iN
(z))
2
=
1
2
e
(z)
2
2
2
N
N
. (9.43)
N
=

N
. (9.44)
This is the normal distribution. Clearly the result does not depend on the original
probability distribution functions p(x) and p(y).
The average z of N random numbers y
i
corresponding to a probability distribution
function p(y) is distributed according to the normal probability distribution function
with average equal to the average value of p(y) and variance equal to the variance of p(y)
divided by
N.
9.4 Monte Carlo Errors and Standard Deviation
In any Monte Carlo approximation method the error goes as 1/
N where N is the num-

ber of samples. This behaviour is independent of the integrand and is independent of
the number of dimensions. In contrast if the error in a classical numerical approximation
method goes as 1/N
a
in one dimension (where N is now the number of intervals) then
the error in the same approximation method will go as 1/N
a
d
in d dimensions. Thus as
we increase the number of dimensions the error becomes worse. In other words classi-
cal numerical integration methods become impractical at suciently higher dimensions.
This is the fundamental appeal of Monte Carlo methods in physics (quantum mechanics
and statistical mechanics) where we usually and so often encounter integrals of innite
dimensionality.
Let us again consider for simplicity the one dimensional integral as an example. We
take
F =
_
b
a
dx f(x). (9.45)
The Monte Carlo sample mean method gives the approximation
F
N
= (b a) < f > , < f >=
1
N
N
i=1
f(x
i
). (9.46)
The error is by denition given by
= F F
N
. (9.47)
However in general we do not know the exact result F. The best we can do is to calculate
the probability that the approximate result F
N
is within a certain range centered around
the exact result F.
The starting point is the central limit theorem. This states that the average z of
N random numbers y
corresponding to a probability distribution function p(y) is dis-

tributed according to the normal probability distribution function. Here the variable y
is (we assume for simplicity that b a = 1)
y =
1
N
N
i=1
f(x
i
). (9.48)
We make M measurements y
of y. We write
y
=
1
N
N
i=1
f(x
i,
). (9.49)
The mean z of the averages is given by
z =
1
M
M
=1
y
. (9.50)
According to the central limit theorem the mean z is distributed according to the normal
probability distribution function with average equal to the average value < y > of y
and variance equal to the variance of y
divided by
M, viz
M
2
2
M
exp
_
M
(z < y >)
2
2
2
M
_
. (9.51)
The
M
is the standard deviation of the mean given by the square root of the variance

2
M
=
1
M 1
M
=1
(y
< y >)
2
. (9.52)
The use of M 1 instead of M is known as Bessels correction. The reason for this
correction is the fact that the computation of the mean < y > reduces the number of
independent data points y
by one. For very large M we can replace

M
with
M
dened
by

2
M

2
M
=
1
M
M
=1
(y
< y >)
2
=< y
2
> < y >
2
. (9.53)
The standard deviation of the sample (one single measurement with N data points) is
given by the square root of the variance

2
=
1
N 1
N
i=1
(f(x
i
) < f >)
2
. (9.54)
Again since N is large we can replace with dened by
2
=
1
N
N
i=1
(f(x
i
) < f >)
2
=< f
2
> < f >
2
. (9.55)
< f >=
1
N
N
i=1
f(x
i
) , < f
2
>=
1
N
N
i=1
f(x
i
)
2
. (9.56)
The standard deviation of the mean
M

M
is given in terms of the standard deviation
of the sample by the equation
M
=

N
. (9.57)
The proof goes as follows. We generalize equations (9.55) and (9.56) to the case of M
measurements each with N samples. The total number of samples is MN. We have
2
=
1
NM
M
=1
N
i=1
(f(x
i,
) < f >)
2
=< f
2
> < f >
2
. (9.58)
< f >=
1
NM
M
=1
N
i=1
f(x
i,
) , < f
2
>=
1
NM
M
=1
N
i=1
f(x
i,
)
2
. (9.59)
M

M
is given by
2
M
=
1
M
M
=1
(y
< y >)
2
=
1
M
M
=1
_
1
N
N
i=1
f(x
i,
) < f >
_
2
=
1
N
2
M
M
=1
N
i=1
N
j=1
_
f(x
i,
) < f >
__
f(x
i,
) < f >
_
. (9.60)
In above we have used the fact that < y >=< f >. For every set the sum over i and
j splits into two pieces. The rst is the sum over the diagonal elements with i = j and
the second is the sum over the o diagonal elements with i = j. Clearly f(x
i,
) < f >
and f(x
j,
) < f > are on the average equally positive and negative and hence for large
numbers M and N the o diagonal terms will cancel and we end up with
2
M
=
1
N
2
M
M
=1
N
i=1
_
f(x
i,
) < f >
_
2
=

2
N
. (9.61)
M
can therefore be interpreted as the probable error
in the original N measurements since if we make M sets of measurements each with N
samples the standard deviation of the mean
M
will estimate how much an average over
N measurements will deviate from the exact mean.
This means in particular that the original measurement F
N
of the integral F has a
68 per cent chance of being within one standard deviation
M
of the true mean and a 95
per cent chance of being within 2
M
and a 99.7 per cent chance of being within 3
M
. In
general the proportion of data values within
M
standard deviations of the true mean
is dened by the error function
_
<y>+
M
<y>
M
1
_
2
2
M
exp
_
(z < y >)
2
2
2
M
_
dz =
2
2
0
exp
_
x
2
_
dx = erf(

2
).
(9.62)
9.5 Nonuniform Probability Distributions
9.5.1 The Inverse Transform Method
We consider two discrete events 1 and 2 which occur with probabilities p
1
and p
2
respec-
tively such that p
1
+ p
2
= 1. The question is how can we choose the two events with
the correct probabilities using only a uniform probability distribution. The answer is as
follows. Let r be a uniform random number between 0 and 1. We choose the event 1 if
r < p
1
else we choose the event 2.
Let us now consider three discrete events 1, 2 and 3 with probabilities p
1
, p
2
and p
3
respectively such that p
1
+p
2
+p
3
= 1. Again we choose a random number r between 0
and 1. If r < p
1
then we choose event 1, if p
1
< r < p
1
+ p
2
we choose event 2 else we
choose event 3.
We consider now n discrete events with probabilities p
i
such that
n
i=1
p
i
= 1. Again
we choose a random number r between 0 and 1. We choose the event i if the random
number r satises the inequality
i1
j=1
p
j
r
i
j=1
p
j
. (9.63)
In the continuum limit we replace the probability p
i
with p(x)dx which is the probability
that the event x is found between x and x +dx. The condition
n
i=1
p
i
= 1 becomes
_
+
p(x) dx = 1. (9.64)
The inequality (9.63) becomes the identity
P(x)
_
x
p(x
) dx
= r (9.65)
Thus r is equal to the cumulative probability distribution P(x), i.e the probability of
choosing a value less than or equal to x. This equation leads to the inverse transform
method which allows us to generate a nonuniform probability distribution p(x) from a
uniform probability distribution r. Clearly we must be able to 1) perform the integral
analytically to nd P(x) then 2) invert the relation P(x) = r for x.
As a rst example we consider the Poisson distribution
p(x) =
1
, 0 x . (9.66)
We nd
P(x) = 1 e
= r. (9.67)
Hence
x = ln(1 r). (9.68)
Thus given the uniform random numbers r we can compute directly using the above for-
mula the random numbers x which are distributed according to the Poisson distribution
p(x) =
1
.
The next example is the Gaussian distribution in two dimensions
p(x, y) =
1
2
2
e
x
2
+y
2
2
2
. (9.69)
We can immediately compute that
1
2
2
_
+
dx
_
+
dy e
x
2
+y
2
2
2
=
_
1
0
dw
_
1
0
dv. (9.70)
x = r cos , y = r sin . (9.71)
r
2
= 2
2
ln v , = 2w. (9.72)
The random numbers v and w are clearly uniformly distributed between 0 and 1. The
random numbers x (or y) are distributed according to the Gaussian distribution in one
dimension. This method is known as the Box-Muller method.
9.5.2 The Acceptance-Rejection Method
This was proposed by Von Neumann. The goal is to generate a sequence of random
numbers distributed according to some normalized probability density y = p(x). This
method consists of the following steps:
We start by generating a uniform random number r
x
in the range of interest x
min

r
x
x
max
where [x
min
, x
max
] is the interval in which y = p(x) does not vanish.
We evaluate p(r
x
).
Then we generate another uniform random number r
y
in the range [0, y
max
] where
y
max
is the maximum value of the distribution y = p(x).
If r
y
< p(r
x
) then we accept the random number r
x
else we reject it.
We repeat this process a sucient number of times.
It is not dicult to convince ourselves that the accepted random numbers r
x
will be
distributed according to y = p(x).
9.6 Lab Problem 16: Midpoint and Monte Carlo Approxi-
mations
Part I The volume of a ball of radius R in d dimensions is given by
V
d
=
_
x
2
1
+...+x
2
d
R
2
dx
1
...dx
d
= 2
_
dx
1
...dx
d1
_
R
2
x
2
1
... x
2
d1
=
R
d
d
2
d
2
(
d
2
)
.
(1) Write a program that computes the three dimensional integral using the midpoint
approximation. We take the stepsize h = 2R/N, the radius R = 1 and the number
of steps in each direction to be N = N
x
= N
y
= 2
p
where p = 1, 15.
(2) Show that the error goes as 1/N. Plot the logarithm of the absolute value of the
absolute error versus the logarithm of N.
(3) Try out the two dimensional integral. Work in the positive quadrant and again
take the stepsize h = R/N where R = 1 and N = 2
p
, p = 1, 15. We know that
generically the theoretical error goes at least as 1/N
2
. What do you actually nd?
Why do you nd a discrepancy?
Hint: the second derivative of the integrand is singular at x = R which changes
the dependence from 1/N
2
to 1/N
1.5
.
Part II In order to compute numerically the volume of the ball in any dimension d we
use the recursion formula
V
d
=
V
d1
R
d1
_
+R
R
dx
d
(R
2
x
2
d
)
d1
2
.
(1) Find the volumes in d = 4, 5, 6, 7, 8, 9, 10, 11 dimensions. Compare with the exact
result given above.
Part III
(1) Use the Monte Carlo sampling (hit or miss) method to nd the integrals in d =
2, 3, 4 and d = 10 dimensions. Is the Monte Carlo method easier to apply than the
midpoint approximation?
(2) Use the Monte Carlo sample mean value method to nd the integrals in d = 2, 3, 4
and d = 10 dimensions. For every d we perform M measurements each with N
samples. We consider M = 1, 10, 100, 150 and N = 2
p
, p = 10, 19. Verify that the
exact error in this case goes like 1/
N.
Hint: Compare the exact error which is known in this case with the standard
deviation of the mean
M
and with /
N where is the standard deviation of

the sample, i.e. of a single measurement. These three quantities must be identical.
Part IV
(1) The value of can be given by the integral
=
_
x
2
+y
2
R
2
dx dy.
Use the Monte Carlo sampling (hit or miss) method to give an approximate value
of .
(2) The above integral can also be put in the form
= 2
_
+1
1
dx
_
1 x
2
.
Use the Monte Carlo sample mean value method to give another approximate value
of .
9.7 Lab Problem 17: Nonuniform Probability Distributions
Part I The Gaussian distribution is given by
P(x) =
1
2
2
exp
(x )
2
2
.
The parameter is the mean and is the variance, i.e the square root of the standard
deviation. We choose = 0 and = 1.
(1) Write a program that computes a sequence of random numbers x distributed ac-
cording to P(x) using the inverse transform method (Box-Muller algorithm) given
by the equations
x = r cos .
r
2
= 2
2
ln v , = 2w.
The v and w are uniform random numbers in the interval [0, 1].
(2) Draw a histogram of the random numbers obtained in the previous question. The
steps are as follows:
a- Determine the range of the points x.
b- We divide the interval into u bins. The lenght of each bin is h = interval/u.
We take for example u = 100.
c- We determine the location of every point x among the bins. We increase the
counter of the corresponding bin by a unit.
d- We plot the fraction of points as a function of x. The fraction of point is equal
to the number of random numbers in a given bin divided by hN where N is
the total number of random numbers. We take N = 10000.
(3) Draw the data on a logarithmic scale, i.e plot log(fraction) versus x
2
. Find the t
and compare with theory.
Part II
(1) Apply the acceptance-rejection method to the above problem.
(2) Apply the Fernandez-Criado algorithm to the above problem. The procedure is as
follows
a- Start with N points x
i
such that x
i
= .
b- Choose at random a pair (x
i
, x
j
) from the sequence and make the following
change
x
i

x
i
+x
j
2
x
j
x
i
+
2x
j
.
c- Repeat step 2 until we reach equilibrium. For example try it M times where
M = 10, 100, ....
10
Monte Carlo Importance Sampling, Metropolis
Algorithm and Ising Model
10.1 The Canonical Ensemble
We consider physical systems which are in thermal contact with an environment. The
environment is usually much larger than the physical system of interest and as a conse-
quence energy exchange between the two of them will not change the temperature of the
environement. The environement is called heat bath or heat reservoir. When the system
reaches equilibrium with the heat bath its temperature will be given by the temperature
of the heat bath.
A system in equilibrium with a heat bath is described statistically by the canonical
ensemble in which the temperature is xed. In contrast an isolated system is described
statistically by the microcanonical ensemble in which the energy is xed. Most systems
in nature are not isolated but are in thermal contact with the environment. It is a
fundamental result of statistical mechanics that the probability of nding a system in
equilibrium with a heat bath at temperature T in a microstate s with energy E
s
is given
by the Boltzmann distribution
P
s
=
1
Z
e
Es
, =
1
k
B
T
. (10.1)
The normalization connstant Z is the partition function. It is dened by
Z =
s
e
Es
. (10.2)
The sum is over all the microstates of the system with a xed N and V . The Helmholtz
free energy F of a system is given by
F = k
B
T lnZ. (10.3)
In equilibrium the free energy is minimum. All other thermodynamical quantities can
be given by various derivatives of F. For example the internal energy U of the system
which is the expectation value of the energy can be expressed in terms of F as follows
U =< E >=
s
E
s
P
s
=
1
Z
s
E
s
e
Es
=

ln Z =

(F). (10.4)
The specic heat is given by
C
v
=

T
U. (10.5)
In the denition of the partition function (10.2) we have implicitly assumed that we
are dealing with a physical system with congurations (microstates) which have discrete
energies. This is certainly true for many quantum systems. However for many other
systems especially classical ones the energies are not discrete. For example the partition
function of a gas of N distinguishable classical particles is given by
Z =
_
N
i=1
d
3
p
i
d
3
q
i
h
3
e
H( p
i
,q
i
)
. (10.6)
For quantum dynamical eld systems (in Euclidean spacetimes) which are of fundamental
importance to elementary particles and their interactions the partition function is given
by the so-called path integral which is essentially of the same form as the previous
equation with the replacement of the Hamiltonian H( p
i
, q
i
) by the action S[] where
stands for the eld variables and the replacement of the measure
N
i=1
(d
3
p
i
d
3
q
i
)/h
3
by
the relevant (innite dimensional) measure D on the space of eld congurations. We
obtain therefore
Z =
_
D e
S[]
. (10.7)
Similarly to what happens in statistical mechanics where all observables can be derived
from the partition function the observables of a quantum eld theory can all be derived
from the path integral. The fundamental problem therefore is how to calculate the
partition function or the path integral for a given physical system. Normally an analytic
solution will be ideal. However nding such a solution is seldom possible and as a
consequence only the numerical approach remains available to us. The partition function
and the path integral are essentially given by multidimensional integrals and thus one
should seek numerical approaches to the problem of integration.
10.2 Importance Sampling
In any Monte Carlo integration the numerical error is proportional to the standard de-
viation of the integrand and is inversely proportional to the number of samples. Thus
in order to reduce the error we should either reduce the variance or increase the number
of samples. The rst option is preferable since it does not require any extra computer
time. Importance sampling allows us to reduce the standard deviation of the integrand
and hence the error by sampling more often the important regions of the integral where
the integrand is largest. Importance sampling uses also in a crucial way nonuniform
probability distributions.
Let us again consider the one dimensional integral
F =
_
b
a
dx f(x). (10.8)
We introduce the probability distribution p(x) such that
1 =
_
b
a
dx p(x). (10.9)
We write the integral as
F =
_
b
a
dx p(x)
f(x)
p(x)
. (10.10)
We evaluate this integral by sampling according to the probability distribution p(x). In
other words we nd a set of N random numbers x
i
which are distributed according to
p(x) and then approximate the integral by the sum
F
N
=
1
N
N
i=1
f(x
i
)
p(x
i
)
. (10.11)
The probability distribution p(x) is chosen such that the function f(x)/p(x) is slowly
varying which reduces the corresponding standard deviation.
10.3 The Ising Model
We consider a ddimensional periodic lattice with n points in every direction so that
there are N = n
d
points in total in this lattice. In every point (lattice site) we put a spin
variable s
i
(i = 1, ..., N) which can take either the value +1 or 1. A conguration of
this system of N spins is therefore specied by a set of numbers {s
i
}. In the Ising model
the energy of this system of N spins in the conguration {s
i
} is given by
E
I
{s
i
} =
<ij>
ij
s
i
s
j
H
N
i=1
s
i
. (10.12)
The parameter H is the external magnetic eld. The symbol < ij > stands for nearest
neighbor spins. The sum over < ij > extends over N/2 terms where is the number of
nearest neighbors. In 2, 3, 4 dimensions = 4, 6, 8. The parameter
ij
is the interaction
energy between the spins i and j. For isotropic interactions
ij
= . For > 0 we obtain
ferromagnetism while for < 0 we obtain antiferromagnetism. We consider only > 0.
The energy becomes with these simplications given by
E
I
{s
i
} =
<ij>
s
i
s
j
H
N
i=1
s
i
. (10.13)
The partition function is given by
Z =
s
1
s
2
...
s
N
e
E
I
{s
i
}
. (10.14)
There are 2
N
terms in the sum and = 1/k
B
T.
In d = 2 we have N = n
2
spins in the square lattice. The conguration {s
i
} can
be viewed as an n n matrix. We impose periodic boundary condition as follows. We
consider (n + 1) (n + 1) matrix where the (n + 1)th row is identied with the rst
row and the (n + 1)th column is identied with the rst column. The square lattice is
therefore a torus.
10.4 The Metropolis Algorithm
The internal energy U =< E > can be put into the form
< E >=
s
E
s
e
Es
s
e
Es
. (10.15)
Generally given any physical quantity A its expectation value < A > can be computed
using a similar expression, viz
< A >=
s
A
s
e
Es
s
e
Es
. (10.16)
The number A
s
is the value of A in the microstate s. In general the number of microstates
N is very large. In any Monte Carlo simulation we can only generate a very small number
n of the total number N of the microstates. In other words < E > and < A > will be
approximated with
< E > < E >
n
=
n
s=1
E
s
e
Es
n
s=1
e
Es
. (10.17)
< A > < A >
n
=
n
s=1
A
s
e
Es
n
s=1
e
Es
. (10.18)
The calculation of < E >
n
and < A >
n
proceeds therefore by 1) choosing at random
a microstate s, 2) computing E
s
, A
s
and e
Es
then 3) evaluating the contribution of
this microstate to the expectation values < E >
n
and < A >
n
. This general Monte
Carlo procedure is however highly inecient since the microstate s is very improbable
and therefore its contribution to the expectation values is negligible. We need to use
importance sampling. To this end we introduce a probability distribution p
s
and rewrite
the expectation value < A > as
< A >=
s
As
ps
e
Es
p
s
s
1
ps
e
Es
p
s
. (10.19)
Now we generate the microstates s with probabilities p
s
and approximate < A > with
< A >
n
given by
< A >
n
=
n
s=1
As
ps
e
Es
n
s=1
1
ps
e
Es
. (10.20)
This is importantce sampling. The Metropolis algorithm is importance sampling with p
s
given by the Boltzmann distribution, i.e.
p
s
=
e
Es
n
s=1
e
Es
. (10.21)
We get then the arithmetic average
< A >
n
=
1
n
n
s=1
A
s
. (10.22)
The Metropolis algorithm in the case of spin systems such as the Ising model can be
summarized as follows:
(1) Choose an initial microstate.
(2) Choose a spin at random and ip it.
(3) Compute E = E
trial
E
old
. This is the change in the energy of the system due
to the trial ip.
(4) Check if E 0. In this case the trial microstate is accepted.
(5) Check if E > 0. In this case compute the ratio of probabilities w = e
E
.
(6) Choose a uniform random number r in the inetrval [0, 1].
(7) Verify if r w. In this case the trial microstate is accepted, otherwise it is rejected.
(8) Repeat steps 2) through 7) until all spins of the system are tested. This sweep
counts as one unit of Monte Carlo time.
(9) Repeat setps 2) through 8) a sucient number of times until thermalization, i.e.
equilibrium is reached.
(10) Compute the physical quantities of interest in n thermalized microstates. This can
be done periodically in order to reduce correlation between the data points.
(11) Compute averages.
The proof that this algorithm leads indeed to a sequence of states which are distributed
according to the Boltzmann distribution goes as follows.
It is clear that the steps 2) through 7) corresponds to a transition probability between
the microstates {s
i
} and {s
j
} given by
W(i j) = min(1, e
E
) , E = E
j
E
i
. (10.23)
Since only the ratio of probabilities w = e
E
is needed it is not necessary to normalize
the Boltzmann probability distribution. It is clear that this probability function satises
the detailed balance condition
W(i j) e
E
i
= W(j i) e
E
j
. (10.24)
Any other probability function W which satises this condition will generate a sequence
of states which are distributed according to the Boltzmann distribution. This can be
shown by summing over the index j in the above equation and using
j
W(i j) = 1.
We get
e
E
i
=
j
W(j i) e
E
j
. (10.25)
The Boltzmann distribution is an eigenvector of W. In other words W leaves the equilib-
rium ensemble in equilibrium. As it turns out this equation is also a sucient condition
for any ensemble to approach equilibrium.
10.5 The Heat-Bath Algorithm
The heat-bath algorithm is generally a less ecient algorithm than the Metropolis algo-
rithm. The acceptance probability is given by
W(i j) = min(1,
1
1 +e
E
) , E = E
j
E
i
. (10.26)
This acceptance probability satises also detailed balance for the Boltzmann probability
distribution. In other words the detailed balance condition which is sucient but not
necessary for an ensemble to reach equilibrium does not have a unique solution.
10.6 The Mean Field Approximation
10.6.1 Phase Diagram and Critical Temperature
We consider N = L
2
spins on a square lattice where L is the number of lattice sites
in each direction. Each spin can take only two possible values s
i
= +1 (spin up) and
s
i
= 1 (spin down). Each spin interacts only with its 4 neigbhors and also with a
magnetic eld H. The Ising model in 2 dimensions is given by the energy
E{s} = J
<ij>
s
i
s
j
H
i
s
i
. (10.27)
The system is assumed to be in equilibrium with a heat bath with temperature T. Ther-
mal equilibrium of the Ising model is described by the canonical ensemble. The prob-
ability of nding the Ising model in a conguration {s
1
, ..., s
2
N } is given by Boltzmann
distribution
P{s} =
e
E{s}
Z
. (10.28)
The partition function is given by
Z =
{s}
e
E{s}
=
s
1
...
s
2
N
e
E{s}
. (10.29)
The magnetization M in a conguration {s
1
, ..., s
2
N } is the order parameter of the system.
It is dened by
M =
i
s
i
. (10.30)
The average of M is given by
< M >=
i
< s
i
>= N < s > . (10.31)
In above < s
i
>=< s > since all spins are equivalent. We have
< M >=
1
log Z
H
=
F
H
. (10.32)
In order to compute < M > we need to compute Z. In this section we use the mean
eld approximation. First we rewrite the energy E{s} in the form
E{s} = (J
<ij>
s
j
)s
i
H
i
s
i
=
i
H
i
e
s
i
H
i
s
i
. (10.33)
The eective magnetic eld H
i
e
is given by
H
i
e
= J
j(i)
s
j(i)
. (10.34)
The index j(i) runs over the four nearest neighbors of the spin i. In the mean eld
approximation we replace the spins s
j(i)
by their thermal average < s >. We obtain
H
i
e
= J < s > , = 4. (10.35)
In other words
E{s} = (H +J < s >)
i
s
i
= H
e
i
s
i
(10.36)
The partition function becomes
Z =
_
s
1
e
H
e
s
i
_
N
=
_
e
H
e
+e
H
e
_
N
(10.37)
=
_
2 cosh H
e
_
N
. (10.38)
The free energy and magnetization are then given by
F = kT ln Z = kTN ln
_
2 cosh H
e
_
. (10.39)
< M >= N < s >= N tanh H
e
. (10.40)
Thus for zero magnetic eld we get from the second equation the constraint
< s >= tanh J < s > . (10.41)
Clearly < s >= 0 is always a solution. This is the high temperature paramagnetic phase.
For small temperature we have also a solution < s >= 0. This is the ferromagnetic phase.
There must exist a critical temperature T
c
which separates the two phases. We expect
< s > to approach < s >= 0 as T goes to T
c
from below. In other words near T
c
we can
treat < s > as small and as a consequence we can use the expansion tanh x = x
1
3
x
3
.
We obtain
< s >= J < s >
1
3
_
J < s >
_
3
. (10.42)
Equivalently
< s >
_
< s >
2
3
T
1
(J)
3
_
J
k
B
T
_
_
= 0. (10.43)
We get the two solutions
< s >= 0 , paramagnetic phase
< s >=
3
T
1
(J)
3
(T
c
T)
, ferromagnetic phase. (10.44)

The critical temperature T
c
and the critical exponent are given by
T
c
=
J
k
B
, =
1
2
. (10.45)
The ferromagnetic solution can only exist for T < T
c
.
10.6.2 Critical Exponents
The free energy for zero magnetic eld is
F = kTN ln
_
2 cosh J < s >
_
. (10.46)
We see that for T < T
c
the ferromagnetic solution has a lower free energy than the
paramagnetic solution < s >= 0. The phase T < T
c
is indeed ferromagnetic. The
transition at T = T
c
is second order. The free energy is continuous at T = T
c
, i.e.
there is no latent heat while the specic heat is logarithmically divergent. The mean
eld theory yields the correct value 0 for the critical exponent although it does not
reproduce the logarithmic divergence. The susceptibility diverges at T = T
c
with critical
exponent = 1. These latter statements can be seen as follows.
The specic heat is given by
C
v
=

T
_
k
B
T
2

T
(F)
_
= 2k
B
T

T
(F) k
B
T
2

2
T
2
(F). (10.47)
Next we use the expression F = N ln(e
x
+e
x
) where x = J < s >. We nd
C
v
N
= 2k
B
T tanh x
x
T
+k
B
T
2
tanh
2
x
2
x
T
2
+k
B
T
2
1
cosh
2
x
(
x
T
)
2
. (10.48)
We compute
x =
3k
B
J
(T
c
T)
1
2
,
x
T
=
1
2
3k
B
J
(T
c
T)
1
2
,

2
x
T
2
=
1
4
3k
B
J
(T
c
T)
3
2
.
(10.49)
It is not dicult to show that the divergent terms cancel and as a consequence
C
v
N
(T
c
T)
, = 0. (10.50)
The susceptibility is given by
=

H
< M > . (10.51)
To compute the behavior of near T = T
c
we consider the equation
< s >= tanh(J < s > +H). (10.52)
For small magnetic eld we can still assume that J < s > +H is small near T = T
c
and as a consequence we can expand the above equation as
< s >= (J < s > +H)
1
3
(J < s > +H)
3
. (10.53)
Taking the derivative with respect to H of both sides of this equation we obtain
= (J +) (J +)(J < s > +H)
2
. (10.54)
=

H
< s > . (10.55)
Setting the magnetic eld to zero we get
= (J +) (J +)(J < s >)
2
. (10.56)
In other words
_
1 J +J(J < s >)
2
_
= (J < s >)
2
. (10.57)
2
T
c
T
T
=
1
k
B
T
(1 (J < s >)
2
). (10.58)
Hence
=
1
2k
B
(T
c
T)
, = 1. (10.59)
10.7 Simulation of The Ising Model and Numerical Results
10.7.1 The Fortran Code
We choose to write our code in Fortran. The reason is simplicity and straightforwardness.
A person who is not well versed in programming languages, who has a strong background
in physics and maths, and who wants to get up and running quickly with the coding so
that she starts doing physics (almost) immediately the choice of Fortran for her is ideal
and thus it is only natural. The potential superior features which may be found in C are
peripheral to our purposes here.
The spin found in the intersection point of the ith row and jth column of the lattice
will be represented with the matrix element (i, j). The energy will then read (with
N = n
2
and n L)
E =
n
i,j=1
_
J
2
(i, j)
_
(i + 1, j) +(i 1, j) +(i, j + 1) +(i, j 1)
_
+H(i, j)
_
.
(10.60)
We impose periodic boundary condition in order to reduce edge and boundary eects.
This can be done as follows. We consider (n + 1) (n + 1) matrix where the (n + 1)th
row is identied with the rst row and the (n + 1)th column is identied with the rst
column. The square lattice is therefore a torus. The toroidal boundary condition will
read explicitly as follows
(0, j) = (n, j) , (n + 1, j) = (1, j) , (i, 0) = (i, n) , (i, n + 1) = (i, 1).
The variation of the energy due to the ipping of the spin (i, j) is an essential ingredient
in the Metropolis algorithm. This variation is explicitly given by
E = 2J(i, j)
_
(i + 1, j) +(i 1, j) +(i, j + 1) +(i, j 1)
_
+ 2H(i, j).(10.61)
The Fortran code contains the following pieces:
A subroutine which generates pseudo random numbers. We prefer to work with
well established suboutines such as the RAN 2 or the RANLUX.
A subroutine which implements the Metropolis algorithm for the Ising model. This
main part will read (with some change of notation such as J = exch)
do i=1,L
ip(i)=i+1
im(i)=i-1
enddo
ip(L)=1
im(1)=L
do i=1,L
do j=1,L
deltaE=2.0d0*exch*phi(i,j)*(phi(ip(i),j)+phi(im(i),j)+phi(i,ip(j))+phi(i,im(j)))
deltaE=deltaE + 2.0d0*H*phi(i,j)
if (deltaE.ge.0.0d0)then
probability=dexp(-beta*deltaE)
call ranlux(rvec,len)
r=rvec(1)
if (r.le.probability)then
phi(i,j)=-phi(i,j)
endif
else
phi(i,j)=-phi(i,j)
endif
enddo
enddo
We compute the energy < E > and the magnetization < M > of the Ising model
in a separate subroutine.
We compute the errors using for example the Jackknife method in a separate sub-
routine.
We x the parameters of the model such as L, J, = 1/T and H.
We choose an initial conguration. We consider both cold and hot starts which are
given respectively by
(i, j) = +1. (10.62)
(i, j) = random signs. (10.63)
We run the Metropolis algorithm for a given thermalization time and study the
history of the energy and the magnetization for dierent values of the temperature.
We add a Monte Carlo evolution with a reasonably large number of steps and
compute the averages of E and M.
We compute the specic heat and the susceptibility of the system.
10.7.2 Some Numerical Results
Energy: The energy is continuous through the transition point and as a consequence
there is no latent heat. This indicates a second order behavior.
Specic Heat: The critical exponent associated with the specic heat is given by
= 0. However the specic heat diverges logarithmically at T = T
c
. This translates into
the fact that the peak grows with n logarithmically, namely
C
v
n
2
log n. (10.64)
Magnetization: The magnetization near but below the critical temperature in the
two-dimensional Ising model scales as
< M >
n
2
(T
c
T)
, = 1/8. (10.65)
Susceptibility: The susceptibility near the critical temperature in the two-dimensional
Ising model scales as
n
2
|T T
c
|
, = 7/4. (10.66)
Critical Temperature: From the behavior of the above observable we can measure
the critical temperature, which marks the point where the second order ferromagnetic
phase transition occurs, to be given approximately by
k
B
T
c
=
2J
ln(
2 + 1)
. (10.67)
Critical Exponents and 2Point Correlation Function: The 2point correlation
function of the two-dimensional Ising model is dened by the expression
f(x) = < s
0
s
x
>
= <
1
4n
2
i,j
(i, j)
_
(i +x, j) +(i x, j) +(i, j +x) +(i, j x)
_
> .
(10.68)
We can verify numerically the following statements:
At T = T
c
the behaviour of f(x) is given by
f(x)
1
x
, = 1/4. (10.69)
At T less than T
c
the behavior of f(x) is given by
f(x) =< M >
2
. (10.70)
At T larger than T
c
the behaviour of f(x) is given by
f(x) a
1
x
. (10.71)
Near T
c
the correlation lenght diverges as

1
|T T
c
|
, = 1. (10.72)
Note that near-neighbor lattice sites which are a distance x away in a given direc-
tion from a given index i are given by
do x=1,nn
if (i+x .le. n) then
ipn(i,x)=i+x
else
ipn(i,x)=(i+x)-n
endif
if ((i-x).ge.1)then
imn(i,x)=i-x
else
imn(i,x)=i-x+n
endif
enddo
For simplicity we consider only odd lattices, viz n = 2nn + 1. Clearly because
of the toroidal boundary conditions the possible values of the distance x are x =
1, 2, ..., nn.
First Order Transition and Hysteresis: We can also consider the eect of a mag-
netic eld H on the physics of the Ising model. We observe a rst order phase transition
at H = 0 or H near 0 and a phenomena of hysteresis. We observe the following:
For T < T
c
we can observe a rst order phase transition. Indeed we observe a
discontinuity in the energy and the magnetization which happens at a non-zero
value of H due to hysteresis. The jumps in the energy and the magnetization are
typical signal for a rst order phase transition.
For T > T
c
the magnetization becomes a smooth function of H near H = 0 which
means that above T
c
there is no distinction between the ferromagnetic states with
M 0 and M 0.
We recompute the magnetization as a function of H for a range of H back and
fourth. We observe the following:
A hysteresis loop.
The hysteresis window shrinks with increasing temperature or accumulating
more Monte Carlo time.
The hysteresis eect is independent of the size of the lattice.
The phenomena of hysteresis indicates that the behaviour of the system depends
on its initial state and history. Equivalently we say that the system is trapped in
a metastable state.
10.8 Lab Problem 18: The Metropolis Algorithm and The
Ising Model
Part I We consider N = L
2
spins on a square lattice where L is the number of lattice
sites in each direction. Each spin can take only two possible values s
i
= +1 (spin up)
and s
i
= 1 (spin down). Each spin interacts only with its 4 neigbhors and also with a
magnetic eld H. The Ising model in 2 dimensions is given by the energy
E = J
<ij>
s
i
s
j
H
i
s
i
.
We will impose toroidal boundary condition. The system is assumed to be in equilib-
rium with a heat bath with temperature T. Thermal uctuations of the system will be
simulated using the Metropolis algorithm.
(1) Write a subroutine that computes the energy E and the magnetization M of the
Ising model in a conguration . The magnetization is the order parameter of the
system. It is dened by
M =
i
s
i
. (10.73)
(2) Write a subroutine that implements the Metropolis algorithm for this system. You
will need for this the variation of the energy due to ipping the spin (i, j).
(3) We choose L = 10, H = 0, J = 1, = 1/T. We consider both a cold start and a
hot start.
Run the Metropolis algorithm for a thermalization time TTH = 2
6
and study the
history of the energy and the magnetization for dierent values of the temperature.
The energy and magnetization should approach the values E = 0 and M = 0 when
T and the values E = 2JN and M = +1 when T 0.
(4) Add a Monte Carlo evolution with TTM = 2
10
and compute the averages of E and
M.
(5) Compute the specic heat and the susceptibility of the system. These are dened
by
C
v
=

< E >=

T
(< E
2
> < E >
2
) , =

H
< M >= (< M
2
> < M >
2
).
(6) Determine the critical point. Compare with the theoretical exact result
k
B
T
c
=
2J
ln(
2 + 1)
.
Part II Add to the code a separate subroutine which implements the Jackknife method
for any set of data points. Compute the errors in the energy, magnetization, specic heat
and susceptibility of the Ising model using the Jackknife method.
10.9 Lab Problem 19: The Ferromagnetic Second Order
Phase Transition
Part I The critical exponent associated with the specic heat is given by = 0, viz
C
v
L
2
(T
c
T)
, = 0.
However the specic heat diverges logarithmically at T = T
c
. This translates into the
fact that the peak grows with L logarithmically, namely
C
v
L
2
log L.
Verify this behaviour numerically. To this end we take lattices between L = 1030 with
TTH = 2
10
, TMC = 2
13
. The temperature is taken in the range
T = T
c
10
2
step , step = 50, 50.
Plot the maximum of C
v
/L
2
versus ln L.
Part II The magnetization near but below the critical temperature in 2D Ising model
scales as
< M >
L
2
(T
c
T)
, =
1
8
.
We propose to study the magnetization near T
c
in order to determine the value of
numerically. Towards this end we plot | < M > | versus T
c
T where T is taken in the
the range
T = T
c
10
4
step , step = 0, 5000.
We take large lattices say L = 30 50 with TTH = TMC = 2
10
.
Part III The susceptibility near the critical temperature in 2D Ising model scales as
L
2
|T T
c
|
, =
7
4
.
Determine numerically. Use TTH = 2
10
, TMC = 2
13
, L = 50 with the two ranges
T = T
c
5 10
4
step , step = 0, 100.
T = T
c
0.05 4.5 10
3
step , step = 0, 100.
10.10 Lab Problem 20: The 2Point Correlator
In this exercise we will continue our study of the ferromagnetic second order phase
transition. In particular we will calculate the 2point correlator dened by the expression
f(n) =< s
0
s
n
>=<
1
4L
2
i,j
(i, j)
_
(i +n, j) +(i n, j) +(i, j +n) +(i, j n)
_
> .
(1) Verify that at T = T
c
the behaviour of f(n) is given by
f(n)
1
n
, =
1
4
.
(2) Verify that at T less than T
c
f(n) =< M >
2
.
(3) Verify that at T larger than T
c
f(n) a
1
n
.
In all the above questions we take odd lattices say L = 2LL+1 with LL = 2050.
We also consider the parameters TTH = 2
10
, TTC = 2
13
.
(4) Near T
c
the correlation lenght diverges as

1
|T T
c
|
, = 1.
In the above question we take LL = 20. We also consider the parameters TTH =
2
10
, TTC = 2
15
and the temperatures
T = T
c
+ 0.1 step , step = 0, 10.
10.11 Lab Problem 21: Hysteresis and The First Order Phase
Transition
In this exercise we consider the eect of the magnetic eld on the physics of the Ising
model. We will observe a rst order phase transition at H = 0 or H near 0 and a
phenomena of hysteresis .
(1) We will compute the magnetization and the energy as functions of H for a range of
temperatures T. The initialization will be done once for all H. The thermalization
will be performed once for the rst value of the magnetic eld H say H = 5.
After we compute the magnetization for H = 5, we start slowly (adiabatically)
changing the magnetic eld with small steps so we do not loose the thermalization
of the Ising system of spins. We try out the range H = 5, 5 with step equal 0.25.
a- For T < T
c
say T = 0.5 and 1.5 determine the rst order transition point from
the discontinuity in the energy and the magnetization. The transition should
happen at a non-zero value of H due to hysteresis. The jump in the energy
is associated with a non-zero latent heat. The jumps in the energy and the
magnetization are the typical signal for a rst order phase transition.
b- For T > T
c
say T = 3 and 5 the magnetization becomes a smooth function of
H near H = 0 which means that above T
c
there is no distinction between the
ferromagnetic states with M 0 and M 0.
(2) We recompute the magnetization as a function of H for a range of H from 5 to
5 and back. You should observe a hysteresis loop.
a- Verify that the hysteresis window shrinks with increasing temperature or ac-
cumulating more Monte Carlo time.
b- Verify what happens if we increase the size of the lattice.
The phenomena of hysteresis indicates that the behaviour of the system depends
on its initial state and history or equivalently the system is trapped in metastable
states.
Appendix
A
Notes on Numerical Errors
A.1 Floating Point Representation
Any real number x can be put in the binary form
x = m2
ebias
, 1m < 2 , m = b
0
.b
1
b
2
b
3
... (A.1)
We consider a 32bit computer. Since 1m < 2 we must have b
0
= 1. This binary
expansion is called normalized. For single precision oating-point numbers (singles or
oats) we use a 32bit word with one bit for the sign, 8 bits for the exponent e and
23 bits for the signicand m. Since only 8 bits are used to store the exponent we must
have e in the range 0e255. The bias is chosen bias = 127 so that the actual exponent
is in the range 127e bias128. This way we can have very small numbers while
the stored exponent is always positive. Since the rst bit of the signicand is 1 the
stored bits of the signicand are only b
1
b
2
...b
23
. If b
24
, b
25
, .. are not all zero the oating
point representation is not exact. Strictly speaking a oating point number is a number
for which b
24
= b
25
= ..0. The oating point representation of a non-zero real number
is unique because of the condition 1m < 2. In summary the above real number is
represented on the computer by
x
normal oat
= (1)
s
1.f 2
e127
, 0 < e < 255. (A.2)
These are normal numbers. The terminology oating point is now clear. The binary
point can be moved (oated) to any position in the bitstring by choosing the appropriate
exponent.
The smallest normalized number is 2
126
. The subnormal numbers are represented
by
x
subnormal oat
= (1)
s
0.f 2
126
. (A.3)
These are not normalized numbers. In fact the space between 0 and the smallest positive
normalized number is lled by the subnormal numbers.
Explicitly
s e f
Bit Position 31 30-23 22-0
Because only a nite number of bits is used the set of machine numbers (the numbers
that the computer can store exactly or approximately) is much smaller than the set of
real numbers. There is a maximum and a minimum. Exceeding the maximum we get the
error condition known as overow. Falling below the minimum we get the error condition
known as underow.
The largest number corresponds to the normal oating number with s = 0, e = 254
and 1.f = 1.111..1 (with 23 1s after the binary point). We compute 1.f = 1+0.5+0.25+
0.125 + ... = 2. Hence x
normal oat max
= 2 2
127
3.4 10
38
. The smallest number
corresponds to the subnormal oating number with s = 0 and 0.f = 0.00...1 = 2
23
.
Hence x
subnormal oat min
= 2
149
1.4 10
45
. We get for single precision oats the
range
1.4 10
45
single precision 3.4 10
38
. (A.4)
We remark that
2
23
10
6.9
. (A.5)
Thus single precision numbers have 6 7 decimal places of signicance.
There are special cases. The zero can not be normalized. It is represented by two
oats 0. Also are special numbers. Finally NaN (not a number) is also a special
case. Explicitly we have
0 = (1)
s
0.0...0 2
126
. (A.6)
= (1)
s
1.0...0 2
127
. (A.7)
NaN = (1)
s
1.f 2
127
, f = 0. (A.8)
The double precision oating point numbers (doubles) occupy 64 bits. The rst bit
is for the sign, 11 bits for the exponent and 52 bits for the signicand. They are stored
as two 32bist words. Explicitly
s e f f
Bit Position 63 62-52 51-32 31-0
In this case the bias is bias = 1023. They correspond approximately to 16 decimal places
of precision. They are in the range
4.9 10
324
double precision 1.8 10
308
. (A.9)
The above description corresponds to the IEEE 754 standard adopted in 1987 by
the Institute of Electrical and Electronics Engineers (IEEE) and American National
Standards Institute (ANSI).
A.2 Machine Precision and Roundo Errors
The gap between the number 1 and the next largest number is called the machine
precision. For single precision we get = 2
23
. For double precision we get = 2
52
.
Alternatively the machine precision
m
is the largest positive number which if added
to the number stored as 1 will not change this stored 1, viz
1
c
+
m
= 1
c
. (A.10)
Clearly
m
< . The number x
c
is the computer representation of of the number x. The
relative error
x
in x
c
is therefore such that
|
x
| = |
x
c
x
x
|
m
. (A.11)
All single precision numbers contain an error in their 6th decimal place and all double
precision numbers contain an error in their 15th decimal place.
An operation on the computer will therefore only approximate the analytic answer
since numbers are stored approximately. For example the dierence a = b c is on the
computer a
c
= b
c
c
c
. We compute
a
c
a
= 1 +
b
b
a

c
c
a
. (A.12)
In particular the subtraction of two very large nearly equal numbers b and c may lead to
a very large error in the answer a
c
. Indeed we get the error
a

b
a
(
b
c
). (A.13)
In other words the large number b/a can magnify the error considerably. This is called
subtractive cancellation.
Let us next consider the operation of multiplication of two numbers b and c to produce
a number a, viz a = b c. This operation is represented on the computer by a
c
= b
c
c
c
.
We get the error
a
=
b
+
c
. (A.14)
Let us now consider an operation involving a large number N of steps. The question
we want to ask is how does the roundo error accumulate.
The main observation is that roundo errors grow slowly and randomly with N. They
diverge as N gets very large. By assuming that the roundo errors in the individual steps
of the operation are not correlated we can view the accumulation of error as a random
walk problem with step size equal to the machine precison
m
. We know from the study
of the random walk problem in statistical mechanics that the total roundo error will be
proportional to
N, namely
ro
=
N
m
. (A.15)
This is the most conservative estimation of the roundo errors. The roundo errors are
analogous the uncertainty in the measurement of a physical quantity.
A.3 Systematic (Algorithmic) Errors
This type of errors arise from the use of approximate numerical solutions. In general the
algorithmic (systematic) error is inversely proportional to some power of the number of
steps N,i.e
sys
=

N
. (A.16)
The total error is obtained by adding the roundo error, viz
tot
=
sys
+
ro
=

N
N
m
. (A.17)
There is a competition between the two types of errors. For small N it is the systematic
error which dominates while for large N the roundo error dominates. This is very
interesting because it means that by trying to decrease the systematic error (by increasing
N) we will increase the roundo error. The best algorithm is the algorithm which gives
an acceptable approximation in a small number of steps so that there will be no time for
roundo errors to grow large.
As an example let us consider the case = 2 and = 1. The total error is
tot
=
1
N
2
+
N
m
. (A.18)
This error is minimum when
d
tot
dN
= 0. (A.19)
For single precision calculation (
m
= 10
7
) we get N = 1099. Hence
tot
= 4 10
6
.
Most of the error is roundo.
In order to decrease the roundo error and hence the total error in the above example
we need to decrease the number of steps. Furthermore in order for the systematic error
to not increase when we decrease the number of steps we must nd another algorithm
which converges faster with N. For an algorithm with = 2 and = 4 the total error is
tot
=
2
N
4
+
N
m
. (A.20)
This error is minimum now at N = 67 for which
tot
= 9 10
7
. We have only 1/16 as
many steps with an error smaller by a factor of 4.
B
The Jackknife Method and The Method of Least
Squares
The Jackknife Method: Any set of data points in a typical simulation will generally
tend to contain correlation between the dierent points. In other words the data points
will not be statistically independent and as a consequence one can not use the usual
formula to compute the standard deviation of the mean (i.e the probable error). The
aim of the Jackknife method is to estimate the error in a set of data points which contain
correlation. This method works as follows.
1) We start with a sample of N measurements (data points) {X
1
, ..., X
N
}. We com-
pute the mean
< X >=
1
N
N
i=1
X
i
. (B.1)
2) We throw out the data point X
j
. We get a sample of N 1 measurements
{X
1
, ..., X
j1
, X
j+1
, ..., X
N
}. This sample is called a bin. Since j = 1, ..., N we
have N bins.We compute the mean
< X >
j
=
1
N 1
(
N
i=1
X
i
X
j
). (B.2)
3) The standard deviation of the mean will be estimated using the formula
2
1
=
N 1
N
N
j=1
(< X >
j
< X >)
2
. (B.3)
The Jackknife error is
1
.
It is not dicult to show that
< X >
j
< X >=
< X > X
j
N 1
. (B.4)
Thus
2
1
=
1
N(N 1)
N
j=1
(X
j
< X >)
2
=
2
mean
. (B.5)
However in general this will not be true and the Jackknife estimate of the error is
more robust.
4) This can be generalized by throwing out z data points from the set {X
1
, ...., X
N
}.
We end up with n = N/z bins. We compute the means < X >
j
over the bins in
an obvious way. The corresponding standard deviation will be given by
2
z
=
n 1
n
n
j=1
(< X >
j
< X >)
2
. (B.6)
5) The z takes the values z = 1, ..., N 1. The error is the maximum of
z
as a
function of z.
The Method of Least Squares: We assume that we have N data points (x(i), y(i)).
We want to t this data to some curve say a straight line y
t
= mx +b. To this end we
dene the function
=
N
i=1
(y(i) y
t
(i))
2
=
N
i=1
(y(i) mx(i) b)
2
. (B.7)
The goal is to minimize this function with respect to b and m. We have
m
= 0 ,

b
= 0. (B.8)
We get the solution
b =
i
x(i)
j
x(j)y(j)
i
x(i)
2
j
y(j)
(
i
x(i))
2
N
i
x
2
i
. (B.9)
m =
i
x(i)
j
y(j) N
i
x(i)y(i)
(
i
x(i))
2
N
i
x
2
i
. (B.10)
C
Exercises
Exercise 1: We give the dierential equations
dx
dt
= v. (C.1)
dv
dt
= a bv. (C.2)
Write down the exact solutions.
Write down the numerical solutions of these dierential equations using Euler and
Verlet methods and determine the corresponding errors.
Exercise 2: The equation of motion of the solar system in polar coordinates is
d
2
r
dt
2
=
l
2
r
3

GM
r
2
. (C.3)
Solve this equation using Euler, Euler-Cromer and Verlet methods.
Exercise 3: The equation of motion of a free falling object is
d
2
z
dt
2
= g. (C.4)
Write down the exact solution.
Give a solution of this problem in terms of Euler method and determine the error.
We choose the initial conditions z = 0, v = 0 at t = 0. Determine the position and
the velocity between t = 0 and t = 1 for N = 4. Compare with the exact solution
and compute the error in each step. Express the result in terms of l = gt
2
.
Give a solution of this problem in terms of Euler-Cromer and Verlet methods and
determine the corresponding errors.
Exercise 4: The equation governing population growth is
dN
dt
= aN bN
2
. (C.5)
The linear term represents the rate of birth while the quadratic term represents the rate
of death. Give a solution of this problem in terms of the Euler and Verlet methods and
determine the corresponding errors.
Exercise 5: Using the Runge-Kutta method solve the following dierential equations
d
2
r
dt
2
=
l
2
r
3

GM
r
2
. (C.6)
d
2
z
dt
2
= g. (C.7)
dN
dt
= aN bN
2
. (C.8)
Exercise 6: The Lorenz model is a chaotic system given by three coupled rst order
dierential equations
dx
dt
= (y x)
dy
dt
= xz +rx y
dz
dt
= xy bz. (C.9)
This system is a simplied version of the system of Navier-Stokes equations of uid me-
chanics which are relevant for the Rayleigh-Bnard problem. Write down the numercial
solution of these equations according to Runge-Kutta method.

Computationalnotes Final

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Computationalnotes Final

Uploaded by

Copyright:

Available Formats

Lectures on Computational Physics

t(i + 1) = it , i = 1, ..., N + 1. (2.22)

t(i + 1) = it , i = 1, ..., N + 1. (2.39)

. We bisect the interval [x

(b) f(b)0(b a)f

(b) f(b). (4.15)

(x) +.. (4.17)

(x) the result

(x) we get the result

no matter what the initial conditions are. The curve

) is called a (periodic) attractor. This is because any perturbed trajectory

of Q. Below this value

at which the symmetry is spontaneously broken.

at which the transition to chaos occurs.

N then we can infer that the distribution is random.

N where N is the num-

corresponding to a probability distribution function p(y) is dis-

and variance equal to the variance of y

by one. For very large M we can replace

N where is the standard deviation of

, ferromagnetic phase. (10.44)

You might also like