The Parallel Finite Difference Time Domain (FDTD) Project

Darren Rodohan, Raymond Glover and Simon Saunders

Department of Electrical Engineering and Electronics
Brunel University
Uxbridge
Middx UB8 3PH
fdtd@brunel.ac.uk

Abstract
This paper describes the design decisions that were made during an ongoing
project at Brunel University to implement an electromagnetic field simulation
program that could be used to teach both electromagnetics and parallel
computing. The paper discusses the design decisions taken during the course
of the project and presents preliminary results obtained.

1: Introduction
The FDTD project at Brunel University is funded by the Higher Education Funding Councils (HEFC)
of England, Scotland and Wales under the new technologies initiative. The project aims to provide a
fast and accurate electromagnetic (EM) field simulation tool based on a parallel implementation of the
Finite Difference Time Domain (FDTD) algorithm [1].
The FDTD algorithm was first established by Yee [2] as a three dimensional solution of Maxwell's
curl equations. In principle, a volume of space containing any object or collection of objects is
subjected to an electromagnetic disturbance, FDTD then solves for the fields throughout the volume
as a function of time. Applications include (but are not limited to): design of both discrete and
integrated microwave circuits; radar cross section prediction; ionospheric and plasma scattering;
integrated optics; interactions between biological bodies and EM fields; electromagnetic compatibility;
antenna design etc. These applications can all be supported by the basic FDTD algorithm, requiring
only different pre and post processing interfaces to obtain the final results.
Unfortunately, even very simple problems require the processing of a large number of unit cells, in
the order of 105, in three-dimensional space and many time steps, in the order of 103. Since
sequential computing architectures require execution times in the order of tens of hours, its use is
restricted to research and analysis. However, parallel computing technology is now available to allow
fast and efficient solution of numerically intensive computation algorithms, such as FDTD, for design
purposes.
In the past several researchers have investigated parallel implementations of the FDTD algorithm
on the Connection Machine-2 [3], the Associative String Processor [4] and a Transputer network [5].
However, these investigations mapped the algorithm onto specific parallel hardware architectures and
did not take into account the portability of the code. In the following sections a portable parallel
implementation of the FDTD algorithm is described.
2: Design Requirements
The EM simulation tool must be user friendly so that students can use it with little assistance. It must
also be very portable so that many different institutions (the system is to be distributed freely to any
institution funded by the HEFC) can use the software on existing and future hardware, therefore we
must avoid hardware and software lock in problems, thereby ensuring that we can reach the widest
possible user base. In keeping with the new technologies initiative it was decided that networks of
workstations (workstation clusters) were available at the majority of educational establishments and
therefore would be the target architecture. However, it was further decided that the system should be
implemented in a way that would make it portable to other distributed memory architectures if the
need arose. For this reason the software was designed using the structure shown in Figure 1. To
allow for differences between different parallel computing systems, the parallel processing sections of
the code were divided into separate modules (labelled in Figure 1 as the parallel software harness).
This reduces the step from a network of workstations implementation to a distributed parallel
architecture which can be made with some small modifications to the code and retargetted
compilation.
The project also had another major constraint imposed by a total budget over two years of
approximately £50, 000. This was enough to employ one full time member of staff for the duration of
the project, although this does not give a completely true picture of the cost of the project as several
other members of staff are closely related to the project. The constraint on the total amount of man
power available made it imperative that the system was developed by using an enabling technology so
that the tool could be implemented quickly and so that in the future the minimum amount of effort
would be required to port the code between different hardware platforms.

GUI

CAD file
Conversion Parallel Software
Harness

Parameterised
C library

Fig. 1: Software structure.

3: Implementation
The design requirements for the system specified that it must be easy to port the code to other
computing platforms with as little alteration as possible. For this reason we required an enabling
technology for parallel computing that would allow us a large degree of portability between different
computing platforms that was available at low cost. We chose to implement the system using PVM
(Parallel Virtual Machine) v.3.5 [6] as it gave us the maximum amount of flexibility and was available
freely. PVM is a fairly mature product and has a large installed user base therefore making a system
implemented with PVM easier to distribute. PVM allowed us to develop a distributed implementation
of the FDTD algorithm for a network of workstations, while still allowing us to implement the system
on parallel computers in the future if we wish to do so.
We also had to implement a GUI that again was portable between many different platforms,
thereby reducing our overall development costs. For this we chose the Motif windows environment
as it is supported by the majority of hardware vendors.
The code was developed by first implementing a sequential 'C' implementation of the Finite
Difference code with parameterised functions. The parameterised functions were then incorporated
with the PVM run time libraries to build a distributed implementation on a workstation cluster.

4: Preliminary Results
The system has been implemented and tested on a non-dedicated cluster of SUN workstations
connected together by the University's ethernet system. Results on real world problems have been
encouraging with speed up values of up to 7.9 being achieved with 8 processors [7]. In the case of
our network the actual value of the speed up ratio for the same problem varies with time as the
network and processor load caused by other users changes. If the network were unloaded apart from
our application (or the application was run on a dedicated workstation cluster) we could optimise the
granularity of computation required per node versus the communication bandwidth available. This
requires that we optimise the number of processors to be used against the following parameters, for a
particular computing architecture:
Processor speed
Communication bandwidth
Problem size
This is the subject of current research, which will also study the effects of using a heterogeneous
cluster to compute this application.

5: Discussion
By making design decisions for maximum portability at an early stage in the development the only real
choice of computing architecture for this class of problem had to be based distributed memory. PVM
provides the flexibility to design and implement a parallel application based on a distributed memory
model without having to commit to a final target machine, thus enabling maximum flexibility. This
allowed us to implement a parallel implementation of the FDTD algorithm on a workstation cluster
that gave us a substantial improvement in performance while still allowing the possibility of porting
the application to other parallel computers. In the future we intend to port our code to parallel
computers (which support PVM) such as the Cray T3D. This will enable us to tackle larger real
world problems. We can therefore foresee a possible convergence between distributed and parallel
computing.

Acknowledgement
This work is currently funded by the Joint Information Systems Committee of the Higher Education
Funding Councils of England, Scotland and Wales under the project title The implementation of the
Finite Difference Time Domain (FDTD) algorithm on a distributed network of workstations
reference NTI/163. The FDTD method implementation on a network of workstations will be made
available free of charge to all higher education establishments funded by the above councils. The
work of the other members of the FDTD group: S.R. Cvetkovic, P.D.Beavis, S.M. Salehi-Rayhani
and M. Berwick is also acknowledged. Further information about the project can be found at the
following URL http://http1.brunel.ac.uk:8080/depts/fdtd/home.html

References
[1] The finite difference method for electromagnetics, Kunz, K.S. & Luebbers, R.J., CRC Press,
Inc., London, 1993.
[2] Numerical solution of Initial boundary value problems involving Maxwell's equations in
isotropic media, Yee, K.S., vol. AP-14, no. 3, pp. 302-307, May 1966.
[3] FDTD modelling of 3D optical pulse propagation in linear and non-linear materials, Davidson,
D.B., Ziolkowski, R.W. & Judkins, J.B., Second Int. Conf. on Computation in Electromagnetics, IEE
Conf. Pub. No. 384, pp. 166-169, 1994.
[4] A CAD tool for electromagnetic simulation on the Associative String Processor, Rodohan, D.P.
& Saunders, S.R., Proc. 1993 Application Specific Array Processors, edited by L. Dadda & B. Wah,
IEEE Comp. Soc. Press, pp. 583-592, 1993.
[5] Simulation of radiation from a microstrip antenna using three-dimensional finite-difference
time-domain (FDTD) method, Buchanan, W.J., Gupta, N.K. & Arnold, J.M., Eighth Int. Conference
on Antennas and Propagation, IEE Conf. Pub. No. 370, pp. 639- 642, 1993.
[6] PVM: Parallel Virtual Machine. A User's Guide and Tutorial for Networked Parallel
Computing., Geist, A. et al, The MIT Press, 1994.
[7] Parallel Simulation of Electromagnetic Fields for Telecommunications Applications, Rodohan,
D.P., Saunders, S.R., Cvetkovic, S.R., Beavis, P.D., & Glover, R.J., Invited paper to IEEE Singapore
Int. Conf. on Communication Systems (ICCS'94), pp. 435-440, 1994.