A Study of the Sensitivity of Automatic Image
Registration Algorithms to Initial Conditions
Jacqueline Le Moigne
Applied Information Sciences Branch
NASA/GSFC-Code 935
Greenbelt, MD 20771
Tel:301-286-8723
Fax:301-286-1776
Jacqueline.J.LeMoigne-
Stewart@nasa.gov
Jeffrey Morisette
Laboratory for Terrestrial Physics
NASA/GSFC-Code 923
Greenbelt, MD 2071
Jeffrey.T.Morisette@nasa.gov
Abstract- While automatic image registration algorithms
are usually being evaluated with regards to their
accuracy, it is often useful to relate this accuracy to the
"initial conditions", i.e., the distance between the initial
navigation geolocation and the correct result. In this
paper, we describe a modular framework that was built
to describe registration algorithms, and we utilize this
framework to attempt to classify different registration
components and algorithms in terms of their responses to
the initial conditions. Performances will be evaluated on
synthetic data, multi-temporal and multi-sensor data. All
results of the study will be presented at the conference
and will be useful for two different purposes: (1) provide
automatic quality assessment of the geo-location of
remote sensing data by performing inter-algorithm
consistency studies; and (2) be the foundations for the
design of future on-board applications including
planetary exploration.
I. INTRODUCTION
Future decision support systems, intelligent sensors and
adaptive constellations will rely on real- or near-real-time
interpretation of Earth observation data, both on-board and
from ground-based Direct Readout stations. The more expert
the system and far-reaching the application, the more
important will it be to obtain accurate data. Utilized for
global change analysis, validation of new instruments or for
new data analysis, accurate image registration is also very
useful for real-time applications such as those related to
direct readout data, or on-board navigation and processing
systems. While often, automatic image registration
algorithms have been evaluated with regards to their
accuracy, it is also useful to relate this accuracy to the "initial
conditions", i.e., the distance between the initial navigation
geolocation and the correct result. Depending on the quality
of the navigation model and of the ephemeris data, such
0-7803-8742-2/04/$20.00 (C) 2004 IEEE
Authorized licensed use limited to: M Th Koelen. Downloaded on June 12, 2009 at 10:00 from IEEE Xplore. Restrictions apply.
Arlene Cole-Rhodes Roger Eastman
Kisha Johnson Loyola College
Electrical & Comp. Eng. Dept Baltimore, MD 21210
Morgan State University reastman@loyola.edu
Baltimore, MD 21251
acrhodes&kjohnson@eng.morgan.edu Harold Stone, Ilya Zavorin
UMBC/GEST Center at GSFC
Nathan S. Netanyahu Greenbelt, MD 20771
Dept Math. & Comp.Sc. Bar-Ilan hstone@alumni.princeton.edu
Univ., Ramat-Gan, 52900, Israel, zavorin@backserv.gsfc.nasa.gov
and Center For Automation
Research, Univ. of Maryland, Peyush Jain
College Park, MD 20742 NASA/GSFC-Code 586
nathan@cfar.umd.edu Greenbelt, MD 20771
Peyush.Jain@nasa.gov
initial geolocation information may be accurate from within
one to several pixels. Larger errors may occur when model
parameters, such as position and orientation of the spacecraft,
are inaccurate. Furthermore, additional errors in the predicted
values can be introduced, e.g. during or after a maneuver of
the satellite, and the initial geolocation errors may grow
progressively during several weeks until finally discovered.
In this paper, we describe a modular registration framework
built at NASA that is being utilized to attempt to classify
different registration algorithms in terms of their responses to
different parameters, especially initial conditions. For
example, some algorithms might always be able to reach an
approximate solution even when starting far from it, although
they might be less accurate than another technique that
reaches very high sub-pixel accuracy but needs to start
relatively close from the “true” solution. These performances
will be evaluated on several test data and results will be
presented at the conference. This study will be useful for 2
different purposes: (1) provide automatic quality assessment
of the geo-location of remote sensing data by performing
inter-algorithm consistency studies; (2) be the foundations for
the design of future on-board applications and potentially
planetary exploration. This paper will address the general
problem of image registration, review some of the previous
studies performed by the authors, and will describe our
modular registration framework that enable to independently
test multiple registration components.
II. A MODULAR IMAGE REGISTRATION FRAMEWORK
The goal of our project is to develop and assess image
registration methodologies that will enable accurate multi-
source integration. Our work focuses on precision correction
or automatic image registration. While navigation often
refers to “systematic correction”, image registration refers to
“precision correction,” and while systematic correction is
model-based, while precision correction is feature-based. For
1390
many applications, it is very important to reach the sub-pixel
accuracy that can be achieved by precision correction. In our
experiments, we assume that the data has already been
corrected according to a navigation model and is at a level
equivalent to the EOS-level 1B. Assuming that the results of
the systematic correction are accurate within a few or a few
ten’s of pixels, our precision-correction algorithms utilize
selected image features or control points to refine this geo-
location accuracy within one pixel or a sub-pixel. Currently,
there is a large quantity of potential image registration
methods that have been developed for aerial or medical
images and that are applicable to remote sensing images
[1,2]. But there is no systematic study that enables to select
the most appropriate method for a remote sensing application
and predict its accuracy.
As a general definition, image registration is described as
the process that aligns one image to another image of the
same area that was acquired at the same or at different times
by different or identical sensors. Image registration can be
defined by three main steps: (1) extraction of features to be
used in the matching process; (2) feature matching strategy
and metrics; (3) resampling or indexing of the data. The
intent of our work is to survey, design, and develop different
components of the registration process and to evaluate their
performance on well-chosen multiple source data.
Many choices are available for each of the previous three
steps. In earlier work, we investigated step (1), first focusing
on correlation-based methods, then looking at optimization-
based methods.
(A) Features Experiments
Our first experiments focused on features' assessment.
Using correlation as a similarity metrics, we showed that:
• band-pass (e.g., edge or edge-like) features are more
robust, but less accurate than low-pass (e.g., intensity)
features.
• when comparing gradient-edges and wavelets, we observed
that orthogonal wavelet-based registration was usually
faster although not always as accurate than a full-resolution
edge-based registration [4,5]. This was obtained by
exploiting the multi-resolution nature of wavelets.
• among wavelet-like representations, which can be
computed in real or near-real time, the “Steerable
Pyramid,” or Simoncelli representation [6,7], is more
accurate and more robust than Daubechies’ filters.
Subsequent experiments were based on a gradient descent
optimization and used an L2 norm as similarity metrics.
Using this matching methodology, we compared features
obtained from two different multi-resolution decompositions,
the Simoncelli steerable pyramid and the Spline
decompositon [8]. While the Simoncelli steerable pyramid
produces low-pass and band-pass features, the Spline
pyramid only produces low-pass features. Results then
showed than for a gradient optimization matching,
0-7803-8742-2/04/$20.00 (C) 2004 IEEE
Authorized licensed use limited to: M Th Koelen. Downloaded on June 12, 2009 at 10:00 from IEEE Xplore. Restrictions apply.
Simoncelli/low-pass features have a better radius of
convergence, while Simoncelli/band-pass features are the
best in terms of accuracy and consistency, but than when they
converge, the Spline features present the best accuracy.
(B) Similarity Measures Experiments
Using Simoncelli band-pass features as registration
features, exhaustive search as well as a stochastic gradient
optimization matching strategy (Spall’s algorithm [10]) were
utilized to compare 2 similarity measures: correlation and
mutual information [9]. The results show that similarity
peaks obtained with mutual information are sharper than
those obtained with correlation; this can be important when
we need to reach sub-pixel registration accuracy.
Furthermore, when using mutual information with a
stochastic gradient, results show an accuracy of 0.01 pixel on
synthetic test data, 0.64 pixel on multi-temporal (cloudy)
data, and 0.34 pixel on multi-sensor data.
Following these preliminary experiments focusing on one
or another of the components of the registration process, the
next step is then to perform systematic registration
components comparisons, where all these components can be
combined in different ways and assessed on well-chosen test
data. In order to perform these systematic studies and to
enable new components to be tested in a rigorous fashion, we
started building a modular image registration framework. The
concept guiding this framework is that various components
of the registration process can be combined in several ways
in order to reach optimum registration on a given type of data
and under given circumstances. Thereby, the purpose of this
framework is double-fold:
1. it represents a testing framework to:
a. Assess various combinations of components as
a function of the applications,
b. Assess a new registration component compared
to other known ones.
2. it is be the basis of a registration tool where a user can
“schedule” a combination of components as a function of
the application at hand, the available computational
resources and the required registration accuracy.
Figure 3 illustrates this concept, where a registration
algorithm is defined as the combination of a set of features, a
similarity measure, and a matching strategy. In our current
framework:
- features can be either gray levels, low-Pass features from
Simoncelli steerable filters decomposition or from a Spline
decomposition, or Simoncelli band-Pass features,
- similarity metrics can be either cross-correlation, the L2
Norm, Mutual Information or an Hausdorff distance,
- matching strategies are either based on a Fast Fourier
Correlation, three different types of optimization - pure
gradient descent, a Marquard-Levenberg approach
(developed by Thevenaz et al [10], denoted TRU in the
remaining of this paper) or a stocchastic gradient approach
(developed by Spall et al) [9,12], and a Robust Feature
Matching approach [11].
1391
 Spline or
Simoncelli
Features Gray Levels Simoncelli
Band-Pass
Low-Pass

Similarity Correlation L2 Norm Mutual Information Hausdorff Distance

Measure

Thevenaz,
Gradient Rutimann, Spall’s Robust
Strategy FFT
Descent Unser Optimization Feature
Optimization Matching

Figure 3
Modular Approach to Image Registration Combining Various Choices
for Feature Extraction, Similarity Metrics and Matching Strategy

III. EXPERIMENTS AND TEST DATASETS reference chip of that area are computed and extracted, and
III.1. TEST
DATASETS local chip/window registrations are performed. Then, for each
Three different datasets are utilized for this study. pair of scenes, a global registration is being computed with a
Previous experiments were reported in [13,14,15] that generalized Least Mean Squares method that combines all
utilized subsets of these datasets while testing limited previous local registrations. Manual registration is available
numbers of component combinations shown in Figure 3. for this dataset to compute algorithms' accuracies.
These early experiments showed accuracies in the range of
[0.019,0.5] for synthetic images, [0.21,0.64] for multi- Perform Comparison of Different Combinations of Registration Components
temporal or multi-sensor data, and inter-algorithm on the Same Pair of Images
(Definition: Algorithm = "Features + Similarity_Measure + Strategy")
consistencies of about 1/3 pixel. The experiments described
below are now underway, and complete results will be READ Reference, Input, Masks, and Sizes
shown at the conference.
CHOOSE "RST" type and Interpolation Method
(A) Synthethic Dataset Experiments
The first dataset is synthetically created using a 512x512 LOOP on Feature methods
section of a Landsat-TM (band 4) of a Pacific Northwest
DECOMPOSE Reference +
scene. This reference image is geometrically transformed DECOMPOSE Input
using the combination of a scaling, a rotation and a translation, Record Timing
and radiometrically transformed using different amounts of
LOOP on Similarity Measures
Gaussian noise and by convolution with a simulated Point-
Spread Function (PSF). In these experiments, rotations vary LOOP on Strategy Methods
between -20 and +20 degrees, translations between -20 and
+20 pixels, scales between 0.8 and 1.2, and noise variance CALL "image_register" subroutine
between -15 and +20. Both rotation and scaling are done with
respect to the center of the image. After the transformation is OUTPUT and STORE: Deformation
applied, the 256x256 centers of the transformed images are Parameters,timing, Simil-Measure_Value,
extracted and registered to the 256x256 center of the original and Overlap for that Algorithm
reference image. Figure 4 shows a flow chart of the testing of
the multiple registration components of Figure 3 applied to Figure 4
Experiments Setup
register synthetic images transformed with the various
geometric and radiometric transformations defined above. (C) Multi-Sensor Dataset Experiments
The third dataset used in this study represents multi-sensor
(B) Multi-Temporal Dataset Experiments data acquired by four different sensors over four of the
The multi-temporal dataset has been acquired over two MODIS Validation Core Sites. The four sites represent four
areas, Central Virginia and the Washington DC/Baltimore different types of terrain in the United States:
area. For each area, one reference scene is chosen and 6 to 8
• Coast Reserve area with the Virgina site, data acquired
reference chips of size (256x256) are extracted. Also for each
in October 2001.
area, the dataset includes 4 input scenes known from their
UTM (Universal Transverse Mercator projection) coordinates. • Agricultural area with the Konza Prairie in the state of
From these coordinates, windows corresponding to each Kansas, data acquired July to August 2001.

0-7803-8742-2/04/$20.00 (C) 2004 IEEE 1392

Authorized licensed use limited to: M Th Koelen. Downloaded on June 12, 2009 at 10:00 from IEEE Xplore. Restrictions apply.
• Mountainous area with the CASCADES site, data
acquired in September 2000.
• Urban area with the USDA, Greenbelt, Maryland, site,
data acquired in May 2001.
The four sensors and their respective bands and spatial
resolutions involved in the study are:
• IKONOS Bands 3 (Red) and 4 (Near-Infrared), spatial
resolution of 4 meters per pixel,
• Landsat-7/ETM+ Bands 3 (Red) and 4 (Near-Infrared),
spatial resolution of 30 meters per pixel.
• MODIS Red and Near-Infrared bands, spatial resolution
of 500 meters per pixel.
• SeaWIFS Red and Near-Infrared bands, spatial
resolution of 1000 meters per pixel.
Figures 5 and 6 show examples of these data.
ETM+
Figure 5
ETM and IKONOS data of the Virginia Coastal Area
For this test, multi-sensor registrations are performed in
"cascade": IKONOS to ETM+, ETM+ to MODIS, and
MODIS to SeaWIFS. Wavelet decomposition is utilized not
only to compute registration features, but also to bring
various spatial resolution data to similar resolutions, by
performing recursive decimation by 2. For example, after 3
levels of wavelet decomposition, the IKONOS spatial
resolution is brought to 32 meters that, compared to the
Landsat spatial resolution, corresponds to a scaling of 1.07.
This will be the scaling expected when registering IKONOS
to Landsat data in our study. For this dataset, previous
experiments have shown algorithms to be self-consistent
within 1/8 pixel.
V. CONCLUSION
The experiments presented in this paper deal with multi-
sensor, multi-resolution precision correction or image
registration. Using gray levels or wavelet features, several
similarity metrics and search strategies are being tested using
synthetic data as well as Landsat multi-temporal data and
IKONOS, Landsat, MODIS and SeaWIFS data over four
well-chosen EOS Land Validation Core Sites. From these
tests, we will be able to categorize the different algorithms in
terms of their accuracy but also in terms of their sensitivity
to initial (i.e., navigation) conditions, as well as their
computational and memory requirements. Full results of
these experiments will be shown at the conference.
0-7803-8742-2/04/$20.00 (C) 2004 IEEE
Authorized licensed use limited to: M Th Koelen. Downloaded on June 12, 2009 at 10:00 from IEEE Xplore. Restrictions apply.
ETM+ IKONOS
Figure 6
ETM and IKONOS data of the CASCADES Mountainous Area
ACKNOWLEDGEMENTS
The authors wish to acknowledge the support of the NASA
Intelligent Systems/ Intelligent Data Understanding Program.
IKONOS
REFERENCES
[1] L. Brown, “A Survey of Image Registration Techniques,” ACM
Computing Surveys, vol. 24, no.4, 1992.
[2] J. Le Moigne, W.J. Campbell, and R.F. Cromp, 2002, “An Automated
Parallel Image Registration Technique of Multiple Source Remote
Sensing Data,” IEEE Transactions on Geoscience and Remote Sensing,
Vol. 40, No. 8, pp. 1849-1864, August 2002.
[3] J. Le Moigne et al, 2000, “Geo-Registration of Landsat Data by Robust
Matching of Wavelet Features,” IGARSS'2000, Intern. Geoscience and
Remote Sensing Symposium, Honolulu Hawaii, July 24-28, 2000.
[4] J. Le Moigne, and I. Zavorin, “Use of wavelets for image registration,”
SPIE/Aerosense'2000, Orlando, FL, Apr.00.
[5] J. Le Moigne et al, “ Multi-Sensor Registration of Earth Remotely Sensed
Imagery,” 8-th SPIE Int. Symp. Remote Sensing, Im&Sign. Proc. for
Remote Sensing VII, Vol.4541, France, Sept. 2001.
[6] J. Le Moigne et al, “ Multi-Sensor Image Registration for On-the-Ground
or On-Board Science Data Procesisng,” Science Data Processing
Workshop, SDP'2002, Greenbelt, January 2002.
[7] H.S. Stone, “Progressive Wavelet Correlation Using Fourier Methods,”
IEEE Transactions on Signal Processing, Vol. 47, No. 1, 97-107, 1999.
[8] R. Eastman, J. Le Moigne, "Gradient-Descent Techniques for Multi-
Temporal and Multi-Sensor Image Registration of Remotely Sensed
Imagery," FUSION'2001, 4-th Int. Conf. Info. Fusion, CA, August 2001.
[9] P. Thévenaz et al, “A pyramid approach to subpixel registration based on
intensity”, IEEE Trans. on Image Processing, Vol. 7, No. 1, .1998.
[10] I. Zavorin et al, “Evaluating Performance of Automatic Techniques for
Sub-Pixel Registration of Remotely Sensed Imagery,” SPIE Electronic
Imaging 2003, Image Processing: Algorithms and Systems II.
[11] K. Johnson et al, 2001, “Multi-Resolution Image Registration of
Remotely Sensed Imagery Using Mutual Information,” SPIE Aerosense
2001, Wav. Appl. VIII, Florida.
[12] A. Cole-Rhodes et al, 2002, “Multi-Resolution Registration of Remote
Sensing Images Using Stochastic Gradient,” SPIE Aerosense 2002,
Wavelet Applications IX, Orlando, FL, April 2002.
[13]N. Netanyahu et al, “Geo-Registration of Landsat Data by Robust
Matching of Wavelet Features,” to appear in IEEE Transactions on
Geoscience and Remote Sensing, 2004.
[14] J.Le Moigne et al, "Earth Science Imagery Registration," IGARSS'2003,
International Geoscience and Remote Sensing Symposium, Toulouse,
France, July 20-24, 2003.
[15] J. Le Moigne et al, "Registration of Multiple Sensor Earth Science Data,"
Earth Science Technology Conference, Palo Alto, CA, June 22-24, 2004.
1393