Teodoro 2016

Comparison of performance of
object-based image analysis

techniques available in open source
software (Spring and Orfeo Toolbox/
Monteverdi) considering very high
spatial resolution data
Ana C. Teodoro
Ricardo Araujo
Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Comparison of performance of object-based image
analysis techniques available in open source software
(Spring and Orfeo Toolbox/Monteverdi) considering
very high spatial resolution data
Ana C. Teodoroa,b,* and Ricardo Araujob

a
Earth Sciences Institute (ICT), FCUP-University of Porto, Rua Campo Alegre, 4169-007 Porto,
Portugal
b
University of Porto, Department of Geosciences, Environment and Land Planning,
Faculty of Sciences, Porto, Portugal
Abstract. The use of unmanned aerial vehicles (UAVs) for remote sensing applications is
becoming more frequent. However, this type of information can result in several software prob-
lems related to the huge amount of data available. Object-based image analysis (OBIA) has
proven to be superior to pixel-based analysis for very high-resolution images. The main objective
of this work was to explore the potentialities of the OBIA methods available in two different
open source software applications, Spring and OTB/Monteverdi, in order to generate an urban
land cover map. An orthomosaic derived from UAVs was considered, 10 different regions of
interest were selected, and two different approaches were followed. The first one (Spring)
uses the region growing segmentation algorithm followed by the Bhattacharya classifier. The
second approach (OTB/Monteverdi) uses the mean shift segmentation algorithm followed by
the support vector machine (SVM) classifier. Two strategies were followed: four classes
were considered using Spring and thereafter seven classes were considered for OTB/
Monteverdi. The SVM classifier produces slightly better results and presents a shorter process-
ing time. However, the poor spectral resolution of the data (only RGB bands) is an important
factor that limits the performance of the classifiers applied. © 2016 Society of Photo-Optical
Instrumentation Engineers (SPIE) [DOI: 10.1117/1.JRS.10.016011]
Keywords: segmentation; classification; support vector machine; unmanned aerial vehicle; land
use.
Paper 15404P received Jun. 1, 2015; accepted for publication Jan. 19, 2016; published online
Feb. 15, 2016.
1 Introduction
Limitations associated with traditional aerial imagery platforms and remote sensing images
acquired by satellite sensors can be overcome by using unmanned aerial vehicles (UAVs),
which have been developed in recent years into a new aerial platform for image acquisition
with a tremendous potential for environmental,1 urban/land use,2 and agricultural studies.3
The use of UAVs for remote sensing applications is becoming more frequent as the tech-
nologies on on-board cameras and the platform itself are becoming a serious contender to sat-
ellite and airplane imagery, mainly because of the relative low cost for medium/small areas. The
use of this kind of platform brings a new level in terms of spatial resolution. Subdecimeter res-
olution is now easily achieved. However, this type of detailed information can result in several
software problems related to data processing.4
UAVs can fly at low altitudes, allowing them to take very high spatial resolution (VHR)
images and to detect small objects/features, which has not previously been possible. For in-
stance, in rangelands, UAV imagery provides the ability to quantify spatial patterns and patches
*Address all correspondence to: Ana C. Teodoro, E-mail: amteodor@fc.up.pt

1931-3195/2016/$25.00 © 2016 SPIE
Journal of Applied Remote Sensing 016011-1 Jan–Mar 2016 • Vol. 10(1)

Teodoro and Araujo: Comparison of performance of object-based image analysis techniques. . .
of vegetation and soil not detectable with piloted aircraft or satellite imagery.5,6 Due to low pay-
load capabilities of small- and medium-size UAVs, imagery is often acquired with inexpensive
off-the-shelf digital cameras. Moreover, UAVs can supply images even on cloudy days, and the
time needed to prepare and initiate the flight is reduced, which allows greater flexibility in sched-
uling the imagery acquisition. Other advantages of UAVs are their low cost and their great flex-
ibility of configuration when compared with piloted aircraft, which allows the utilization and
testing of low-cost sensors such as conventional digital cameras.7
Detecting changes over the urban area is essential for city monitoring and disaster response,
as well as for map/three-dimensional (3-D) model updating.8 Gathering this type of information
is valuable for applications at various scales: at the landscape scale, it provides information for
studying land cover evolution; at the building level, it has the potential to reduce labor costs for
city management. There are numerous studies on change detection on remote sensing images
with low-to-medium resolution,9–12 which mainly focus on the landscape level. The advent of the
UAV provides a possible platform for such missions. As a small flying vehicle, it is much
cheaper and more flexible than traditional aerial platforms, and the autonomous capability ena-
bles the operator to locate the target area more easily and accurately.
Several studies have investigated change detection/monitoring techniques for applications
with UAVs. Coulter et al.13 used UAV images for monitoring country borders, and a thresh-
old-based method in the image space was used to reach real-time performance, yet this method
was purely based on image comparison, which might be affected by perspective distortion and
luminance difference. Shi et al.14 combined feature points with segmentation results to perform
image-based change detection. Qin15 presented an objected-based change detection method on
UAV images for frequent urban monitoring. Two experiments were performed using regularly
overlapped UAV images with 5-cm footprints. For each experiment, UAV images were taken at
15-month intervals. The accuracies and errors were established with true positives, false pos-
itives (FP), false negatives, and true negatives in pixelwise and objectwise. An overall accuracy
(OA) of 99.6% and 99.2% was archived for both of the experiments. However, an FP rate of
58.2% and 42% was obtained due to the strong seasonal variance of the datasets.
More recently, Feng et al.16 presented an accurate hybrid method via combining random
forest (RF) and texture analysis for vegetation mapping in heterogeneous urban landscapes
based on UAV remote sensing. The VHR images (7 cm) acquired by UAV provide sufficient
details for urban vegetation extraction. An RF consisting of 200 trees was used to classify two
selected UAV orthophotos (RGB) in typical urban landscapes. Experimental results showed that
OA for Image-A and Image-B increased from 73.5% to 90.6% and 76.6% to 86.2% after the
inclusion of texture features, respectively, which indicated that texture plays a significant role in
improving classification accuracy.
While the spatial resolution of UAV imagery is high, the spectral resolution is not, and
imagery usually lacks a near-infrared band. For that reason, texture can be a potentially useful
parameter for mapping rangeland vegetation and soils with this imagery.16 Texture measures
have been used extensively in remote sensing, particularly with high and VHR images and
with panchromatic imagery.17 In general, classification accuracies are improved by the use
of texture.18–21 When texture is calculated from segmented imagery, as is done in OBIA, the
boundary problem is minimized,22 because the segments are relatively homogenous and texture
is calculated per segment.
Most of the imagery classification methods are based on the statistical analysis of each sep-
arate pixel. These methods have shown good performance when used for images with a relatively
large pixel size. The recent emergence of VHR images has introduced a new set of possibilities
for land-cover types at fine levels of detail. OBIA has proven to be successful and is often supe-
rior to pixel-based analysis with high and VHR images that exhibit a large amount of shadow,
low spectral information, or a low signal-to-noise ratio.23 Assuming that landforms can be asso-
ciated with segments as collections of adjacent cells with similar values, OBIA becomes a valu-
able approach to their mapping.24 Generally, OBIA involves two steps: segmentation and
classification. One main issue in OBIA is to transfer the implicit knowledge of an expert
into machine-understandable classification rules.25
With increased use of VHR imagery (aerial and satellite), OBIA has become more common-
place in recent years due to its ability to extract meaningful image objects by segmentation and to

bridge remote sensing and geographical information systems (GISs).26,27 The ability to incor-
porate elements traditionally used in aerial photo interpretation (color, size, shape, texture, pat-
tern, and contextual information), is one of the strengths of OBIA. However, the availability of
hundreds of spectral, spatial, and contextual features for each image object can make the deter-
mination of optimal features a time consuming or subjective process. OBIA is highly suitable for
VHR imagery, where pixel-based classification is less successful due to the high spatial vari-
ability within objects of interest.
Optimal features for classification may be scale dependent, and features used in the analysis
of moderate resolution imagery (i.e., satellite or aerial photography) may not be applicable to
finer resolution data. Digital airborne imagery and UAV data can now be acquired at subdecim-
eter resolution while maintaining sufficient image overlap for photogrammetric processing,
allowing for creation of orthophotos and digital surface models (DSMs) at very high
resolution.28 This imagery exhibits great potential for urban mapping at very high resolution,
despite multiple challenges such as high spatial frequency, the effect of shadows, viewing geom-
etry, and illumination.29
Several studies conclude that OBIA of small automated extracted features or ones repre-
sented in the images by a few grouped pixels presents better results when compared with
the traditional pixel-based classification, considering UAV data acquired. For instance, Peña
et al.30 presented a robust and entirely automatic OBIA procedure that was developed on a series
of UAV images using a six-band multispectral camera (visible and near-infrared range) with the
ultimate objective of generating a weed map in an experimental maize field in Spain. The algo-
rithm employed efficiently identified all the crop rows based on their linear pattern and on the
contextual features of the vegetation objects that belong to the rows.
Laliberte and Rango31 investigated texture measures at multiple scales in OBIA for the pur-
pose of differentiating broad functional groups of vegetation in arid rangelands with subdecim-
eter UAV imagery. The results demonstrate that UAVs are viable platforms for rangeland
monitoring and that the drawbacks of low-cost off-the-shelf digital cameras can be overcome
by including texture measures and using OBIA, which is highly suitable for VHR imagery.
Most of the works cited used OBIA algorithms available in different proprietary software,
where eCognition (now called Definiens Professional) is the most used.30,31 However, different
open source software such as Spring,32 OTB/Monteverdi,33 and Grass34 present different OBIA
strategies.
Spring is a state-of-the-art GIS and remote sensing image processing system with an object-
oriented data model, which provides for the integration of raster and vector data representations
in a single environment. Spring is a product of Brazil’s National Institute for Space Research
(INPE/DPI-Image Processing Division).35 Facilities for digital image processing include contrast
enhancement, spatial filtering, radiometric correction, arithmetic operations, image statistics,
maximum likelihood (statistical) and segmentation classifiers, and a specific module for
radar images.
The Orfeo Toolbox (OTB) is an open source remote sensing image processing software tool-
kit developed by Centre National d’Etudes Spatiales (CNES) in France, which aims at gathering
a large set of state-of-the-art building blocks for processing chains.36 The OTB applications
package makes available a set of simple software tools, which were designed to demonstrate
what can be done with OTB. It supports raster and vector data and integrates most of the already
existing OTB applications. The architecture takes advantage of the streaming and multithreading
capabilities of the OTB pipeline. It also uses features such as processing on demand and auto-
matic file format I/O. The application is called Monteverdi33 since this was the name of the Orfeo
composer. In 2013, the Monteverdi software was revamped to take into account users’ feedback
not only regarding how useful the tool was but also regarding what should be improved to move
toward greater usability and operationality. The Monteverdi integrated application is based on
OTB and provides a user-friendly graphical interface to many of the algorithms available in
the OTB.
The main objective of this work was to explore the potentialities of the OBIA methods avail-
able in two different open source software programs: Spring (version 5.2.6) and OTB/
Monteverdi (version 1.23.0), in order to generate an urban land-cover map using VHR data
(UAV data).

2 Previous Work
The huge amount of data provided by UAVs represents a new challenge regarding development
of image processing techniques. In a previous work, Teodoro and Araújo4 employed an OBIA
approach through Spring. Different “similarity” and “area” parameter combinations were com-
puted in the segmentation stage (region-based algorithm). During the segmentation process,
there were several problems related to software “crashes” and response losses, which led to
a processing time that exceeded 60 h. After that, a supervised classification (Bhattacharya clas-
sifier) was employed, considering seven classes based on Corine land cover nomenclature.
Several parameter combinations were tested and the best result was obtained considering a sim-
ilarity and area value of 5 and 100, respectively (with an acceptance threshold of 99.9%) with a
kappa >0.9 and an OA >90%. Combinations of very low similarity values with low area values
and very high similarity values with high area values are the combinations that produce the worst
results. On the other hand, the high similarity and high area values produce few objects. These
objects may contain pixels that are very different and should be grouped in different objects, but
as they are still within the threshold defined, they are put together. However, several objects were
not classified. The advantage of using unsupervised classification is that this type of algorithm
creates the number of classes needed to classify all objects generated in the segmentation proc-
ess. Due to the high resolution of the data (4 cm), the unsupervised classification performed
identified 27 classes. After that, a new supervised classification (Isoseg classifier) was per-
formed, considering 22 of the 27 classes identified with an OA of 82.58% and a Kappa of
0.817. We conclude that the algorithms employed in Teodoro and Araújo4 are not the most suit-
able for this kind of spatial resolution or when the entire image is considered. The use of more
robust algorithms and/or small regions could improve the results.
3 Methodology
This paper describes the results obtained considered two OBIA approaches in order to generate
an urban land-cover map using UAV data. In order to make an exhaustive evaluation and com-
parison of the OBIA methods available in the two applications considered, 10 regions of interest
(ROIs), representing the heterogeneity of the scene, were considered. The selection of 10 ROIs
was also considered in order to solve the problems that come from the image size.4 The meth-
odology proposed in the present work comprises several steps, as shown in Fig. 1.
OBIA takes into account the form, textures, and spectral information. Segmentation is the
first stage of the object-oriented classification procedure, and its aim is to create meaningful
objects. This means that the shape of each object in question should ideally be represented
by an appropriate image object. This shape combined with further derivative color and texture
properties can be used to initially classify the image by classifying the generated image objects.
After that, different supervised classifications could be employed in order to accurately classify
the entire image.
3.1 Data and Study Area

The UAV system used in this work to collect the images was a Swinglet from Sensefly®. The
swinglet CAM has a flight time of up to 30 min, enabling it to cover up to 6 km2 in a single flight.
Its 12MP RGB camera can shoot aerial imagery at a resolution of down to 4 cm∕pixel. These
images can then be transformed into orthomosaics and 3-D elevation models with a relative
accuracy of down to 3 to 5 cm. A sequence of 41 overlapped images was collected in order
to cover the study area, on January 20, 2012. An important task prior to image analysis was
the combination of all the individual scenes and overlapped images by applying a process
of mosaicking. The imagery had a 60% side-lap and an 80% forward-lap to allow correct
image mosaicking to generate a complete map in the whole study area. The image orthorecti-
fication (RMS 0.0076 m in planimetric and 0.742 m in altimetry) and orthomosaic computation
was performed using Agisoft PhotoScan Professional Edition (Agisoft LLC, St. Petersburg,
Russia). The orthomosaic, with 0.04 m of pixel size, from January 20, 2012, of Coimbra
city (Portugal) with an approximate area of 500 × 400 m was produced using the original

Fig. 1 Workflow of the methodology employed. Gray box: preprocessing tasks; blue box: pro-
cedure carried out in Spring; green box: procedure carried out in OTB/Monteverdi; red box: post-
processing carried out in GIS open source software (QGIS); purple box: class definitions and
training object selection considered after segmentation stage.
41 images (Fig. 2). The first step in orthorectification requires the development of a DSM made
of a 3-D point cloud from pixels matched in overlapping images. The DSM will be not used in
further analysis (in the classification process), because the data granted for this work only
include the orthomosaic. The DSM inclusion in the OBIA algorithms tested in this work
would allow higher classification accuracies to be obtained. The inclusion of both spectral
and spatial information is a natural OBIA application.
Fig. 2 Orthomosaic produced (with the 10 ROIs marked).

The city of Coimbra is one of the most important urban centers in Portugal (after the much
larger cities of Lisbon and Porto), playing a central role in the northern-central littoral and
interior of the country. It is the principal center in the Portugal Centro region.
3.2 Segmentation
OBIA mainly involves segmentation and classification (or recognition) steps. Segmentation is
the process by which an image is partitioned into a set of spatially contiguous image objects
composed of a group of pixels with homogeneity or semantic significance. For the substantial
details of segmentation techniques, readers can refer to the literature.37–42 Theoretically, any
image segmentation algorithm could be used to partition a remote sensing scene. However,
the great challenge is to define appropriate segmentation parameters (e.g., spectral homogeneity,
object size, or both) for the varied sizes, shapes, and spatially distributed image-objects compos-
ing a scene so that the resulting segments are visually meaningful or correspond to entities of
ecological or managerial interest.43
The abundance of literature on image segmentation makes categorization both necessary and
challenging. Most of the earlier literature42–46 has categorized image segmentation methods as:
(i) edge based, (ii) point/pixel based, (iii) region based, and (iv) hybrid approaches. In the past
decades, some popular segmentation algorithms have been proposed, such as watershed segmen-
tation,47–49 mean-shift segmentation (MSS),50 the fractal net evolution approach,51 and hierar-
chical segmentation/recursive hierarchical segmentation.52 In this work, a region-based approach
[region growing (RG) algorithm] and an MSS algorithm were applied. The detailed principles
and the algorithms are addressed in Secs. 3.2.1 and 3.2.2.
3.2.1 Region growing algorithm

The RG algorithm is a region-based method. A region-based method assumes that neighboring pixels
within the same region should have similar values, e.g., intensity, color, and texture.53,54 The appli-
cation of the RG algorithm requires the selection of a similarity criterion, based on a statistical
hypothesis test, that checks the average DN among adjacent regions, and a minimum area threshold
for each segment to obtain desired segmented output. The appropriate selection of the similarity
criterion and minimum area threshold depends on image characteristics, such as spatial resolution,
radiometric/spectral resolution, and scene characteristics. The RG algorithm is an iterative procedure
by which regions are merged starting from individual pixels or another initial segmentation and
growing iteratively until every pixel is processed. It can be described by the following steps.55
1. Segment the entire image into pattern cells (one or more pixels).
2. Each pattern cell is compared with its neighboring cells to determine if they are similar,
using a similarity measure. If they are similar, merge the cells to form a fragment and
update the property used in the comparison.
3. Continue growing the fragment by examine all of its neighbors until no joinable regions
remain. Label the fragment as a completed region.
4. Move to the next uncompleted cell, and repeat these steps until all cells are labeled.
The algorithm available in Spring is based on the traditional RG techniques56,57 with some
modifications, which partially solve the problem of dependence on the order of the merges. The
detailed stages of the procedure are illustrated in the flowchart shown in Fig. 3.
The similarity threshold value must be manually provided by the user and, therefore, a trade-
off is inevitable: if it is set too low, the growing process will generate oversegmented regions;
otherwise, segments representing different land cover will be incorrectly merged together. The
choice of this threshold value as well as the area threshold value will greatly depend on the
specific application and data characteristics.55
3.2.2 Mean shift segmentation algorithm

The MSS algorithm has received a lot of attention from the remote sensing community.58–60 It has
been found to perform well with various remote sensing data ranging from medium to VHR and

Fig. 3 Flowchart of the RG algorithm presented in Spring. R is used to denote the set of regions of
the image, and R ∈ R is an element of this set; T ðt Þ is the threshold value below which two regions
are considered similar at instant t ; M i is the mean value vector of the region R i ; DðR i ; R k Þ ¼ kM i −
M k k is the Euclidean distance between the spectral mean values of the regions R i and R k ; NðRÞ is
the set of neighboring regions of R. The region R k is the most similar neighboring region of R i if
DðR i ; R k Þ ≤ DðR i ; R l Þ for every R l ∈ N ðR i Þ.55
in various applications.61 The MSS algorithm is not really a segmentation algorithm by itself.61 It
is a nonparametric method first introduced by Fukunaga and Hostetler62 in 1975 for the esti-
mation of modes in a multivariate density of probability function. In 2002, Commaniciu63 pro-
posed a spatial extension of the algorithm by applying the mode estimation to the joint spatial
and spectral domain. For each pixel, several steps are computed. The detailed stages of the pro-
cedure are illustrated in the flowchart presented in Fig. 4. More details about this algorithm can
be found in Michel et al.61 The MSS algorithm applied in this work is available in OTB/
Monteverdi,64,65 where for a given pixel, the MSS algorithm will build a set of neighboring pixels
within a given spatial radius and color range. The spatial and color center of this set is then
computed and the algorithm iterates with this new spatial and color center. There are three
main parameters: the spatial radius used for defining the neighborhood, the range radius
used for defining the interval in the color space, and the minimum size for the regions to be
kept after clustering. Then, each segmented region is converted to a map of labeled objects.
3.3 Classification
Classification involves labeling image objects as geographical objects. Instead of analyzing the
spectral behavior of individual pixels, OBIA groups neighboring pixels into objects/segments.
This process produces a meaningful representation of object shapes and causes spatial informa-
tion, particularly contextual information, to become explicit.
A supervised evaluation method compares segmented results with the reference by region
overlapping, boundary distance, or shape difference.66 In this work, two supervised classification

Fig. 4 Flowchart of the MSS algorithm,61 where x i and z i , i ¼ 1; : : : ; n, denote pixels from the input
and output joint-domain image; y i;j is the current mode estimation for pixel x i at step j, K ðx Þ is a
kernel function, Nðx Þ is the set of pixels within the spatial range hs and spectral range hr of x ,
superscripts s and r denote the spatial and spectral components of the joint-domain image pixels,
j max denotes the maximum number of iterations, and t denotes the convergence threshold.
algorithms, available in Spring and in OTB/Monteverdi, were applied. Descriptions of these

algorithms are addressed in Secs. 3.3.1 and 3.3.2.
3.3.1 Bhattacharya classifier

The Bhattacharya classifier is a supervised algorithm used to classify segmented images. The
steps of this classification are the same as those of the pixel classification. This distance is cal-
culated using Eq. (1):67

1 1 jðCa þ CbÞj∕2
B ¼ MH þ ln ; (1)
8 2 ðjCajjCbjÞ1∕2
EQ-TARGET;temp:intralink-;e001;116;339
where Ca and Cb are the covariance matrices of classes A and B; MH is the Mahalanobis dis-
tance defined for two distinct classes using Eq. (2):
−1 1∕2
Ca þ Cb
MH ¼ ðμa − μbÞT ðμa − μbÞ ; (2)
2
where μ is the mean of each class.

More details about this classifier can be found in Choi and Lee.67
3.3.2 Support vector machine

Support vector machine (SVM) classifiers68,69 have demonstrated their effectiveness in several
remote sensing applications.70 In particular, several researchers addressed the problem of VHR
ground cover classification using SVM.71–73 SVM is a nonparametric supervised classifier rely-
ing on Vapnik’s statistical learning theory.68 This classifier aims at building a linear separation
rule between examples in a higher dimensional space induced by a mapping function φð·Þ on
training samples. A linear separation in that space corresponds to a nonlinear separation in the
original input space. The core of such an algorithm is given by the kernel trick: since in the SVM
formulation mapped samples appear only in the form of dot products, these operations can be

replaced by valid “kernel functions” kð·; Þ returning directly the inner product value in that space
[dual formulation, Eq. (3)]. The solution is given by the hyperplane with maximal margin width
that guarantees the best generalization ability on previously unseen data. In the dual optimization
formulation, one has to optimize74
X
N
1XX X
N
max αi − αi αj ωi ωj kðxi ; xj Þ s:t: 0 ≤ αi ≤ C; αi ωi ¼ 0; (3)
α 2
i¼1 i¼1
where C is a user-defined parameter controlling the tradeoff between complexity and training
error of the model, αi are the coefficients determining the solution of the optimization, and ωi ∈
fþ1; −1g (binary case) are the class labels associated with samples xi .
When the solution to Eq. (3) is found, the label of an unknown sample x 0 is given by the sign
of the decision function, i.e., its position with respect to the separating hyperplane [Eq. (4)]:
" #
Xj
0 0
ω ¼ sign αi ωi kðxi ; x Þ þ b : (4)
i¼1
More details about this classifier can be found in Volpi et al.69
3.3.3 Evaluating the performance of a classifier

Accuracy assessment is an important step in the classification process. The accuracy and inter-
pretability of the classification is fully dependent on the accuracy of both training data and val-
idation data. The accuracy of the training data will influence the success of the classification, and
the validation data are assumed to be 100% correct in an accuracy assessment, so that any dis-
crepancies between the land-cover map and the validation data are assumed to be errors on the
map.75 The results are summarized in a confusion matrix, which cross-tabulates the allocated
class against the reference data for the sample locations.76 A confusion matrix is a square array of
numbers set out in rows and columns, which expresses the number of sample units assigned to a
particular category relative to the actual category as verified by ground truth information or a
reference data set. The confusion matrix can be used to calculate OA and class-specific mea-
sures, such as user accuracy (UA) and producer accuracy (PA). A description of the matrix and
the measures that can be derived from it can be found in Congalton.75 Kappa analysis is a popular
multivariate technique for accuracy assessment77 and gives a measure that indicates whether the
confusion matrix is significantly different from a random result. More details about the Kappa
coefficient can be found in Congalton and Green.78
4 Results
The results obtained through Spring and OTB/Monteverdi in order to generate an urban land-
cover map for each ROI selected are described in the following sections. In this work, all the
algorithms were implemented in the same computer with a CPU Intel 2500k at 4.8 GHz and
12 GB RAM memory.
4.1 Regions of Interest Definition

As referred to in Teodoro and Araújo,4 the total time taken in processing the entire scene in
Spring was about 60 h, using the same computer considered in this work. Despite the processing
time being very substantial, during this process, several problems happened with the software (e.
g., “crashes” and response losses), which led to the repetition of several combinations; therefore,
the time spent was much higher than 60 h. In order to reduce the processing time and optimize
the process, 10 ROIs were defined (Fig. 5). These regions were chosen based on areas that could
be more problematic and representative of the scene complexity. One of the major problems
found in the ROIs selected is the high degradation in the rooftops and the high variability
in the color and state of the shingles. These problems are common to ROIs I to IV and in

Fig. 5 ROI identification: (a) ROI I, (b) ROI II, (c) ROI III, (d) ROI IV, (e)ROI V, (f) ROI Vi, (g) ROI
VII, (h) ROI VIII, (i) ROI IX, and (j) ROI X.

ROI VI. The presence of skylights in some of the rooftops can also become problematic. The
heterogeneous vegetation areas, more visible in ROI I, ROI II, and ROI IV, can also provide an
extra layer of complexity. Several extra difficulty elements/structures are present in the 10 ROIs
defined. For instance, in ROI I, the presence of a car; in ROI II, the presence of garbage/recycling
bins; in ROI III, the presence of a solar panel; ROI IV corresponds to a junction area of two
images and presents a high distortion in the habitation roof; in ROI V, the presence of light-
colored roof very similar to pavement; ROI VI presents high geometric distortion, and the pres-
ence of a blue sunblind and road markings makes the classification process difficult; in ROI VII,
the presence of two swimming pools with distinct shapes; ROI VIII corresponds to an artificial
sports field with several boundary lines marked; in ROI IX, the presence of a fountain; and in
ROI X, the presence of a set of white pipes on the top of a building. The 10 ROIs defined are
presented in Fig. 5, where the scenes’ complexity can be checked.
4.2 Spring
Considering the 10 ROIs identified in Sec. 4.1, new segmentations (RG algorithm) and classi-
fications (Bhattacharya classifier) were executed in Spring. In the segmentation stage, three com-
binations of area and similarity parameters were considered, (100, 5), (100, 10), and (25, 15).
These combinations correspond to the best three conjugations of segmentation parameters
obtained in a previous work using the same data.4 After that, considering the in situ knowl-
edge/identification (ground visits) of this area, the segmented regions were aggregated in training
objects and the Bhattacharya classifier was applied. The classes defined for each ROI are pre-
sented in Table 1. The training objects selected for ROI III are presented in Fig. 6.
A random set of objects were generated and classification results are compared with the true
information classes in the reference image. In Table 2, the accuracy assessment is presented (OA,
UA, PA, and Kappa), considering the best result, which was obtained considering the segmen-
tation combination of (100, 5) for area and similarity parameters, respectively, for all the ROIs
selected.
Although this approach produced very good results, proved by the values presented in
Table 2, with a simple visual inspection of the classification results, it is possible to detect several
classification errors. One of the major gaps is the lack of total classification. For instance, despite
the 100% OA for ROI I, there are several areas not classified, as can be seen in Fig. 7(a), where
the objects/areas not classified are represented in white. The worst result was obtained for ROI
IV, with an OA of 69.85% and a UA of 53.1% for the “pavement” class [Fig. 7(b)]. The UA, from
the perspective of the end user of the classified map, represents how accurate the map is. This low
Table 1 Classes identified for each ROI considering a maximum of four

classes (Spring and OTB/Monteverdi).
ROI Classes Spring (maximum of 4)
I 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement
II 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement
III 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement
IV 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement
V 1-Vegetation 2-Habitation 3-Outdoor benches 4-Bare soil
VI 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement
VII 1-Vegetation 2-Habitation 3-Water 4-Wall
VIII 1- Sports flooring 2-White lines 3-Yellow lines
IX 1-Vegetation 2-Water 3-Bare soil
X 1-Rooftop 2-White pipe

Fig. 6 ROI III training objects resulting from the application of RG algorithm (in Spring). Training
objects assigned as “1” correspond to Vegetation, “2” correspond to Habitation, “3” correspond to
Asphalt, and “4” correspond to Pavement.
value of UA is an indicator of an error of commission. In this ROI, beyond several segments not
classified, there are several classification errors. For instance, the identification of a pavement
class (yellow) in the habitation roof (red) and the confusion between the classes pavement (yel-
low) and asphalt (gray) in the sidewalk classification.
Considering these results, a new approach considering the OBIA algorithms available in
OTB/Monteverdi was implemented using the same classes.
Table 2 Accuracy assessment [overall accuracy (OA), producer’s accuracy (PA), user’s accu-
racy (UA), and Kappa] for each ROI defined in Spring. In the producer’s accuracy (PA) and user’s
accuracy (UA), the values given are according to the class order presented in Table 1.
ROI OA (%) Kappa PA (%) UA(%)
I 100.00 1 100:100;100:100 100:100;100:100
II 99.17 0.976 100:100;100:100 100:100;100:100
III 84.30 0.802 100:100;100:100 100:100;100:100
IV 69.85 0.598 100;90.1;85.2;100 100;100;100;53.1
V 100.00 1 100:100;100:100 100:100;100:100
VI 97.09 0.961 100;100;100;74.6 100;65.4;100;100
VII 100.00 1 100:100;100;100 100:100;100;100
VIII 100.00 1 100:100;100 100:100;100
IX 98.64 0.978 100:100;100 100:100;100
X 100 1 100:100 100:100

Fig. 7 Classification results considering: (a) ROI I and (b) ROI IV. Classes shown as “green” cor-
respond to Vegetation, “red” correspond to Habitation, “gray” correspond to Asphalt, and “yellow”
correspond to Pavement. The white objects are the segments not classified.
4.3 OTB/Monteverdi
In this work, the strategy adopted was to process the 10 ROIs separately. However, the MSS
algorithm was first tested considering the entire image. This process took a few hours (less than
in Spring), and several problems related to software “crashes” and response losses were also
verified.
As already mentioned, the MSS algorithm allows one to generate a segmented image.
Basically, this module could be used before processing supervised classification through
SVM. For the MSS algorithm, it was necessary to define different options: spatial radius, spectral
radius, and minimum region size. Several combinations were tested and the combination of 15,
15, and 500 for the spatial radius, spectral radius, and minimum region size, respectively, was
chosen because it was the one that had a better identification (segmentation) of the different
objects available.
After that, and also considering the in situ knowledge/identification (ground visits) of this
area, the regions were aggregated in training objects, and the SVM classifier was applied. The
training objects resulting from the application of MSS algorithms for ROI IV are presented
in Fig. 8.
In the classification stage, the kernel type chosen was linear (by default) and the parameter
optimization was performed using libsvm. libsvm is integrated software for support vector
classification.79 The number of active samples (segments) considered for each class was 10.
In order to properly compare the results, the same classes defined in Spring were considered
(Table 1). The classification results for each ROI are presented in Figs. 9 and 10.
Fig. 8 ROI IV training objects resulting from the application of the MSS algorithm (in OTB/
Monteverdi). Training objects assigned as “1” correspond to Vegetation, “2” correspond to
Habitation, “3” correspond to Asphalt, and “4” correspond to Pavement.

Fig. 9 Classification (SVM) resulting from the MSS algorithm for ROI I to ROI VI, considering a
maximum of four classes: (a) ROI I, (c) ROI II, (e) ROI III, (g) ROI IV, (i) ROI V, (k) ROI VI and a
maximum of 6∕7 classes: (b) ROI I, (d) ROI II, (f) ROI III, (h) ROI IV, (j) ROI V, (l) ROI VI.

Fig. 10 Classification (SVM) resulting from the MSS algorithm for ROI VII to ROI X, considering a
maximum of four classes: (a) ROI VII, (b) ROI VIII, (c) ROI IX, and (d) ROI X.
Unlike the classification algorithm available in Spring, the SVM classifies all the image seg-
ments. In ROI I [Fig. 9(a)], it is possible to observe some distortion caused by the mosaic process
and some overlapping causing a disfiguration in the roof. The roof is one of the more problematic
areas, and an accurate classification of this area is a great challenge. On the other hand, the
vegetation classification is very consistent. The car is classified according to its color.
Visually comparing the classification result with the original image (Fig. 5), it can be seen
that a reasonable reliable classification was produced. The OA obtained was 90.91%, which
is a very satisfactory result.
In ROI II [Fig. 9(c)], the distortion caused by the mosaic is even more visible than in the
classification performed in Spring. The differences on the two sides of the roofs, the high number
of skylights, the transparency in some tree areas, and the different types of vegetation are the
most problematic areas. A simple visual inspection shows that, despite the majority of the clas-
sification being good, there are still some areas that were misclassified. However, the OA
obtained was 90.00%, which is also a very good result.
In ROI III [Fig. 9(e)], the distortion is less noticeable. However, the different types of roofs
and the presence of a solar panel and two cars provide a serious challenge for the classification
process. Visually inspecting the classification, it is possible to realize that it fails to correctly
classify some objects/elements. The classification of the solar panel and the cars as “asphalt” is
an error than could possibly have been corrected with the addition of new classes. Despite these
problems, the OA achieved was 93.75%, a better result than what was achieved with Spring.
In ROI IV [Fig. 9(g)], the car is badly classified (according to its color), and a high distortion
on the roof is also very visible. However, the OA obtained was 92.68%, a much higher value than
that obtained in the classification performed in Spring. This increase of the OA is related to a
better PA achieved in the “pavement” class.
In ROI V [Fig. 9(i)], despite the 100% OA obtained, some classification errors can be identified.
Due to having a very similar color to outdoor benches, the lighter roof that should be classified in the
“habitation” class is wrongly classified as outdoor benches (gray color). The red pavement is also
wrongly classified as “habitation.” Generally, the vegetation class presents a good classification.
In ROI VI [Fig. 9(k)], the transparency in some tree areas and the classification as vegetation
of a blue sunblind are the major errors. The OA achieved (87.80%) was lower than that obtained
in Spring.

For ROI VII [Fig 10(a)], the OA obtained (96.55%) was similar to what was obtained in
Spring. The SVM algorithm perfectly classifies the two pools and defines their boundaries
well. The major classification errors are related to the joining of the two white objects located
in the upper right pool (although these are separate) and the classification as “habitation” of a
part of a tree.
For ROI VIII [Fig. 10(b)], an OA of 100% was obtained. Despite the incorrect identification
of some lines, this classification is very accurate and much better than the one given by Spring.
In ROI IX [Fig. 10(c)], the classification accuracy is very satisfactory (95.24%), and the
classifier can differentiate the areas with and without water inside of the fountain.
Finally, in ROI X [Fig. 10(d)], the OA was maximum (100%), and in general, the classifi-
cation produces very good results.
4.3.1 More classes

Despite the high values of OA obtained for most of the ROIs, considering a maximum of four
classes (Table 2), a simple visual analysis is allowed to identify several errors, i.e., various
objects/elements have been incorrectly classified. In order to improve the classification,
more classes (depending on the ROI selected) were defined (Table 3).
The results for each ROI are presented in Figs. 9(b), 9(d), 9(f), 9(h), 9(j), and 9(l). The num-
ber of classes defined for ROI VII to ROI X is the same, because the number of classes available
in the ROI was already equal to or less than 4. As already stated, the SVM classifies all the image
segments.
In ROI I [Fig. 9(b)], it is possible to check that the car was classified into class “paint” and
“other.” In this classification, the “other” class was used to try to correctly classify the car.
However, as can be observed in Fig. 9(b), this was not successfully achieved. The OA obtained
was 80.43%.
In the second ROI [Fig. 9(d)], the introduction of three new classes, “other,” “skylights,”
and “paint,” increases the visual accuracy of this classification, but not the OA (80.00%). In
this ROI, the “other” class was used to classify the garbage/recycling bins, which were well
classified. There are some areas that still remain wrongly classified, mainly the sidewalk,
which was classified into four different classes, but in general, this classification is more rep-
resentative of the scene heterogeneity when compared with results obtained considering four
classes.
Some of the major problems found in the classification for ROI III [Fig. 9(e)] were when four
classes were considered, as the incorrect classification of the solar panel or the skylights can be
Table 3 Classes identified for each ROI in OTB/Monteverdi considering more classes (maximum
of 7).
ROI Classes Monteverdi (maximum of 7)
I 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement 5-Paint 6-Other
II 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement 5-Paint 6-Skylights 7-Other
III 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement 5-Paint 6-Skylights 7-Other
IV 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement 5-Paint 6-Other
V 1-Vegetation 2-Habitation 3-Outdoor benches 4-Bare soil 5-Light rooftop 6-Red pavement
VI 1-Vegetation 2-Habitation 3-Asphault 4-Pavement 5-Paint 6-Other
VII 1-Vegetation 2-Habitation 3-Water 4-Wall
VIII 1- Sports flooring 2-White lines 3-Yellow lines
IX 1-Vegetation 2-Water 3-Bare soil
X 1-Rooftop 2-White pipe

solved with the addition of the three extra classes: “other,” “skylights,” and “paint,” as can be
seen in Fig. 9(f). The SVM correctly classifies most of the car surfaces in the class “other” (blue);
however, several dark-colored parts/segments of the image were incorrectly classified in the
same class (“other” class). The OA obtained was the lowest for all the ROIs processed consid-
ering the SVM classifier, with a value of 79.77%.
For ROI IV [Fig. 9(h)], the value of OA (84.9%) was again lower than the value obtained
when only four classes were considered, but several classification improvements could be easily
checked. For instance, the car is properly classified in the class “other” (blue) and the white
antenna on top of the building was identified.
In ROI V [Fig. 9(j)], due to the introduction of new classes, both the clear roof (white) as well
as the red floor (blue) were correctly classified in the class assigned. The OA obtained was very
satisfactory (91.49%).
For ROI VI [Fig. 9(l)], the OA achieved (81.13%) was slightly lower than that obtained when
only four classes were considered. The blue sunblind was well classified and some road mark-
ings were properly identified. However, the classification of the habitation roof (disrepair) still
presents several errors.
5 Discussion
For each ROI, 10 training objects from each class were chosen (when possible). The training
objects are not exactly the same for the different approaches employed, since the two segmen-
tation methods applied produce distinct objects. The segments used to define these training
objects were selected from the same zones, in order to properly compare the classification
results.
The OA obtained, considering the classifications performed in OTB/Monteverdi, is presented
in Table 4. It was only possible to check the OA. When the error matrix was assessed, an error
message appeared: “big images are not supported,” and it was not possible to continue. The
classification obtained considering four classes presented OA values very similar for the two
approaches employed. The Spring software approach presents slightly better results.
However, when we visually check the results, the OTB/Monteverdi classifications are much
more representative of the land/urban cover types that could be found in each ROI. Unlike
the Bhattacharya classifier, the SVM algorithm classifies all the image segments. The adoption
of more classes will increase the land/urban cover detail; however, the OA is lower. This aspect
could be justified due to the low accuracy of some individual classes, which have a very small
Table 4 OA obtained for each ROI defined considering the OBIA strategy
implemented in OTB/Monteverdi (for a maximum of 4 and 7 classes).
ROI Monteverdi (4 classes) Monteverdi (6/7 classes)
I 90.91% 80.43%
II 90.00% 80.00%
III 93.75% 79.77%
IV 92.68% 84.91%
V 100% 91.49%
VI 87.80% 81.13%
VII 96.55% —
VIII 100% —
IX 95.24% —
X 100% —

area extent. A valid alternative could be a postclassification correction that involved integrating
ancillary information. The recent approach is to let the geographical data, such as digital eleva-
tion models (slope, aspect), geology/soils, hydrology, and vegetation, have more weight in the
decision rather than let statistically derived parameters dictate the analysis.80 The inclusion of
DSM in the OBIA algorithms tested could also be allowed to obtain higher classification accu-
racies. The inclusion of both spectral and spatial information is a natural OBIA application.
Another factor that can justify the lower accuracy for some ROIs is related to the high geometric
distortions verified.
The processing time was identical in all the ROI selected, considering the two programs
(around 2 min in Spring and less than 1 min in OTB/Monteverdi). Nevertheless, the procedure
is always faster in OTB/Monteverdi. Considering the segmentation and classification procedures
separately, the segmentation stage is more time consuming than the classification step in both
approaches. For instance, for ROI II, the segmentation and classification times were 2’ 10”+ 1’
25” and 32”+23,” for the Spring and OTB/Monteverdi approaches, respectively. The procedure
time in OTB/Monteverdi is identical considering a maximum of seven classes or a maximum of
four classes. This aspect could be crucial when the image size is high, as was discussed in
Teodoro and Araújo.4 The processing time is always lower for the smaller regions (e.g.,
ROI VII) and higher for bigger regions (e.g., ROI III), as expected.
6 Conclusions
The first objective of this work was to explore the OBIA methods present in different image
processing open source software to process UAV data, in order to generate an urban land-
cover map. Unfortunately, it was not possible to obtain results from open source software
other than Spring and OTB/Monteverdi, mainly because most of the open source software
does not have OBIA algorithms available or cannot deal with the size of the UAV images.
The size of UAV scenes is a real problem. In order to consider the amount of data available
and produce urban land cover maps, it was necessary to select different ROIs with small
sizes and analyze these regions separately. From this work, it is possible to conclude that
the different strategies implemented produced good results. SVM classifies all of the objects
obtained from the segmentation, unlike the Bhattacharya classifier, which leaves several seg-
ments to classify. SVM is less sensitive to overtraining, as they are designed specifically to
avoid overspecifying class decision boundaries. The size of UAV images was also a problem
in accuracy assessment in OTB/Monteverdi.
Obviously, when there is much diversity/heterogeneity in the ROI analyzed, if considering
few classes, there are elements that have to be aggregated in the same class, but that does not
mean a less accurate classification. As shown, when more classes are considered, there is a
greater possibility of confusion between them, and generally, the accuracy results decrease.
However, the results are much more representative of the land/urban cover types that could
be found.
The poor spectral resolution of the data (only RGB bands are available) is another factor
that limits the performance of the classifiers applied. However, despite all these limitations, it
is effectively impossible to produce a land cover map that is completely accurate and satisfies
the needs of all the end users. The accuracy assessment of the results, with high detail and
without any additional data, shows the effectiveness of the proposed approach. As a general
rule, the level of accuracy obtainable in a remote sensing classification depends on diverse
factors, such as the suitability of training sites; the size, shape, distribution, and frequency
of occurrence of individual areas assigned to each class; the sensor performance and resolu-
tion; the methods involved in classifying (visual photo interpreting versus computer-aided
statistical classifying); and others. Two factors are important in achieving suitable high accu-
racy: spatial resolution (which influences the mixed pixel effect) and the number of spectral
bands involved in the classification. There is no one ideal classification of land use and land
cover, and it is unlikely that one could ever be developed. There are different perspectives in
the classification process, and the process itself tends to be subjective, even when an objective
numerical approach is used.

UAV data can bring a lot of advantages related to budget and data availability, but its use is
still limited due to the extremely high spatial data resolution. The wider use of this technology
can push forward the innovation and research of algorithms better suited for this kind of
classification.
Acknowledgments
The authors would like to thank to Sinfic SA (João Marnoto) for ceding the UAV images and to
Prof. José Alberto Gonçalves for the image orthorectification and mosaic creation.
References
1. M. Bryson et al., “Airborne vision-based mapping and classification of large farmland envi-
ronments,” J. Field Robot. 27, 632–655 (2010).
2. L. L. Ma et al., “Cultivated land information extraction from high-resolution unmanned
aerial vehicle imagery data,” J. Appl. Remote Sens. 8(1), 083673 (2014).
3. F. Garcia-Ruiz et al., “Comparison of two aerial imaging platforms for identification of
Huanglongbing-infected citrus trees,” Comput. Electron. Agric. 91, 106–115 (2013).
4. A. C. Teodoro and R. Araújo, “Exploration of the OBIA methods available in SPRING
noncommercial software to UAV data processing,” Proc. SPIE 9245, 92451F (2014).
5. A. S. Laliberte, A. Rango, and J.E. Herrick, “Unmanned aerial vehicles for rangeland map-
ping and monitoring: a comparison of two systems,” in Proc. ASPRS Annual Conf., Tampa,
Florida (2007).
6. A. Rango et al., “Using unmanned aerial vehicles for rangelands: current applications and
future potentials,” Environ. Pract. 8, 159–168 (2006).
7. J. Torres-Sánchez et al., “Multi-temporal mapping of the vegetation fraction in early-season
wheat fields using images from UAV,” Comput. Electron. Agric. 103, 104–113 (2014).
8. S. Freire et al., “Introducing mapping standards in the quality assessment of buildings
extracted from very high resolution satellite imagery,” ISPRS J. Photogramm. Remote
Sens. 90, 1–9 (2014).
9. R. D. Macleod and R. G. Congalton, “A quantitative comparison of change-detection algo-
rithms for monitoring eelgrass from remotely sensed data,” Photogramm. Eng. Remote
Sens. 64(3), 207–216 (1998).
10. D. J. Hayes and S. A. Sader, “Comparison of change-detection techniques for monitoring
tropical forest clearing and vegetation regrowth in a time series,” Photogramm. Eng. Remote
Sens. 67(9), 1067–1075 (2001).
11. C. Song et al., “Classification and change detection using Landsat TM data: When and how
to correct atmospheric effects?,” Remote Sens. Environ. 75, 230–244 (2001).
12. J. Rogan, J. Franklin, and D. A. Roberts, “A comparison of methods for monitoring multi-
temporal vegetation change using thematic mapper imagery,” Remote Sens. Environ. 80,
143–156 (2002).
13. L. Coulter et al., “Near real-time change detection for border monitoring,” in Proc. of 2011
the ASPRS Annual Conf., pp. 1–5, Milwaukee, Wisconsin (2011).
14. J. Shi, J. Wang, and Y. Xu, “Object-based change detection using georeferenced UAV
images,” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 38, 177–182 (2011).
15. R. Qin, “An object-based hierarchical method for change detection using unmanned aerial
vehicle images,” Remote Sens. 6, 7911–7932 (2014).
16. Q. Feng, J. Liu, and J. Gong, “UAV remote sensing for urban vegetation mapping using
random forest and texture analysis,” Remote Sens. 7, 1074–1094 (2015).
17. I. I. Epifanio and P. Soille, “Morphological texture features for unsupervised and supervised
segmentations of natural landscapes,” IEEE Trans. Geosci. Remote Sens. 45(4), 1074–1083
(2007).
18. S. Ryherd and C. Woodcock, “Combining spectral and texture data inthe segmentation of
remotely sensed images,” Photogramm. Eng. Remote Sens. 62(2), 181–194 (1996).
19. M. A. Wulder et al., “Aerial image texture information in the estimation of northern decidu-
ous and mixed wood forest leaf area index (LAI),” Remote Sens. Environ. 64(1) 64–76
(1998).

20. S. E. Franklin et al., “Incorporating texture into classification of forest species composition
from airborne multispectral images,” Int. J. Remote Sens. 21(1), 61–79 (2000).
21. C. A. Coburn and A. C. B. Roberts, “A multiscale texture analysis procedure for improved
forest stand classification,” Int. J. Remote Sens. 25(20), 4287–4308 (2004).
22. A. Lobo, K. Moloney, and N. Chiariello, “Fine scale mapping of agrass land from digitized
aerial photography: an approach using image segmentation and discriminant analysis,” Int.
J. Remote Sens. 19(1), 65–84 (1998).
23. M. E. Hodgson et al., “Synergistic use of Lidar and color aerial photography for mapping
urban parcel imperviousness,” Photogramm. Eng. Remote Sens. 69(9), 973–980 (2003).
24. S. d’Oleire-Oltmanns et al., “An object-based workflow to extract landforms at multiple
scales from two distinct data types,” IEEE Geosci. Remote Sens. Lett. 10(4), 947–951
(2013).
25. S. Lang, “Object-based image analysis for remote sensing applications: Modeling reality-
dealing with complexity,” in Object-Based Image Analysis, T. Blaschke, S. Lang, and G.
Hay, Eds., pp. 3–27, Springer-Verlag, Berlin, Germany (2008).
26. T. Blaschke, S. Lang, and G. J. Hay, Eds., Object-Based Image Analysis: Spatial Concepts
for Knowledge-Driven Remote Sensing Applications, p. 817, Springer-Verlag, Berlin
(2008).
27. T. Blaschke, “Object based image analysis for remote sensing,” ISPRS J. Photogramm. 65,
2–16 (2010).
28. A. Wiechert and M. Gruber, “DSM and true ortho generation with the UltraCam-L– a case
study,” in Proc. of the American Society for Photogrammetry and Remote Sensing Annual
Conf., pp. 26–30, San Diego, California (2010).
29. A.S. Laliberte et al., “Hierarchical object-based classification of ultra high resolution digital
mapping camera (DMC) imagery for rangeland mapping and assessment,” J. Spat. Sci.
55(1), 101–115 (2010).
30. J.M. Peña et al., “Weed mapping in early-season maize fields using object-based analysis of
unmanned aerial vehicle (UAV) images,” PLOS One 8(10), e77151 (2013).
31. A. S. Laliberte and A. Rango, “Texture and scale in object-based analysis of subdecimeter
resolution unmanned aerial vehicle (UAV) imagery,” IEEE Trans. Geosci. Remote Sens.
47(3), 761–770 (2009).
32. Spring-DPI/INPE, http://www.dpi.inpe.br/spring/english/ (March 2014).
33. ORFEO Toolbox, http://www.orfeo-toolbox.org (September 2014).
34. GRASS GIS, http://grass.osgeo.org/ (September 2014).
35. G. Camara et al., “Spring: integrating remote sensing and GIS by object-oriented data mod-
elling,” Comput. Graphics 20(3), 395–403 (1996).
36. E. Christophe and J. Inglada “Open source remote sensing: increasing the usability of cut-
ting-edge algorithms,” IEEE Geosci. Remote Sens. Newslett. 150, 9–15 (2009).
37. R. Pal and K. Pal, “A review on image segmentation techniques,” Pattern Recognit. 26(9),
1277–1294 (1993).
38. H. D. Cheng et al., “Color image segmentation: advances and prospects,” Pattern Recognit.
34(12), 2259–2281 (2001).
39. U. C. Benz et al., “Multi-resolution, object-oriented fuzzy analysis of remote sensing
data for GIS-ready information,” ISPRS J. Photogramm. Remote Sens. 58(3–4), 239–258
(2004).
40. G. J. Hay et al., “An automated object-based approach for the multiscale image segmenta-
tion of forest scenes,” Int. J. Appl. Earth Obs. Geoinf. 7(4), 339–359 (2009).
41. Q. Q. Yang and W. X. Kang, “General research on image segmentation algorithms,” Int. J.
Image Graphics Signal Process. 1(1), 1–8 (2005).
42. V. Dey, Y. Zhang, and M. Zhong, “A review on image segmentation techniques with remote
sensing perspective,” in ISPRS TC VII Symposium–100 Years ISPRS, Vol. XXXVIII, W.
Wagner and B. Szekely, Eds., pp. 31–42, IAPRS, Vienna, Austria (2010).
43. T. Blaschke and G. Hay, “Object-oriented image analysis and scale-space: theory and meth-
ods for modeling and evaluating multi-scale landscape structure,” Int. Arch. Photogramm.
Remote Sens. 34(2), 22–29 (2001).

44. T. R. Reed and J. M. H. D. Buf, “A review of recent texture segmentation and feature extrac-
tion techniques,” Comput. Vision Graphics Image Process. Image Underst. 57(3), 359–372
(1993).
45. J. Schiewe, “Segmentation of high-resolution remotely sensed data-concepts, applications
and problems,” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 34(4), 358–363 (2002).
46. B. U. Shankar, “Novel classification and segmentation techniques with application to
remotely sensed images,” Lect. Notes Comput. Sci. 4400, 295–380 (2007).
47. L. Vincent and P. Soille, “Watersheds in digital spaces: an efficient algorithm based on
immersion simulations,” IEEE Trans. Pattern Anal. Mach. Intell. 13(6), 583–589 (1991).
48. P. Li and X. Xiao, “Multispectral image segmentation by a multichannel watershed-based
approach,” Int. J. Remote Sens. 28(19), 4429–4452 (2007).
49. D.R. Li et al., “An edge embedded marker-based watershed algorithm for high spatial res-
olution remote sensing image segmentation,” IEEE Trans. Image Process. 19(10), 2781–
2787 (2010).
50. D. Comaniciu and P. Meer, “Mean shift: a robust approach toward feature space analysis,”
IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002).
51. M. Baatz and A. Schape, “Multiresolution segmentation-an optimization approach for high
quality multi-scale image segmentation,” in Proc. 12th Angew, pp. 12–13, Geographis
Informations Verarbeitung (2000).
52. J. C. Tilton, “Parallel implementation of the recursive approximation of an unsupervised
hierarchical segmentation algorithm,” in High Performance Computing in Remote
Sensing, A. J. Plaza and C.-I. Chang, Eds., pp. 1766–1768, Chapman & Hall, New
York (2007).
53. A. Tremeau and N. Bolel, “A region growing and merging algorithm to color segmentation,”
Pattern Recognit. 30(7), 1191–1203 (1997).
54. S. A. Hojjatoleslami and J. Kittler, “Region growing: a new approach,” IEEE Trans. Image
Process. 7(7), 1079–1084 (1998).
55. L. S. Bins, L. M. G. Fonseca, and G. J. Erthal, “Satellite imagery segmentation: a region
growing approach,” in Proc. of III Simpósio Brasileiro de Sensoriamento Remoto, pp. 677–
680, INPE, Salvador, Brasil (1996).
56. S. W. Zucker, “Region growing: childhood and adolescence,” Comput. Graphics Image
Process. 5, 382–399 (1976).
57. R. P. H. M. Schoenmakers, G. G. Wilkinson, and Th. E. Schouten, “Segmentation of
remotely-sensed images: a re-definition for operational applications,” in IGARSS’91
Espoo, Vol. 2, pp. 1087–1090, Finland (1991).
58. N. Jiang et al., “Semi-automatic building extraction from high resolution imagery based on
segmentation,” in Proc. Int. Workshop EORSA, pp. 1–5 (2008).
59. X. Huang and L. Zhang, “An adaptive mean-shift analysis approach for object extraction
and classification from urban hyperspectral imagery,” IEEE Trans. Geosci. Remote Sens.
46(12), 4173–4185 (2008).
60. D. Ming et al., “Semivariogram-based spatial bandwidth selection for remote sensing image
segmentation with mean-shift algorithm,” IEEE Geosci. Remote Sens. Lett. 9(5), 813–817
(2012).
61. J. D. Michel, D. Youssefi, and M. Grizonnet, “Stable mean-shift algorithm and its appli-
cation to the segmentation of arbitrarily large remote sensing images,” IEEE Trans. Geosci.
Remote Sens. 53(2), 952–964 (2015).
62. K. Fukunaga and L. Hostetler, “The estimation of the gradient of a density function, with
applications in pattern recognition,” IEEE Trans. Inf. Theory 21(1), 32–40 (1975).
63. D. Commaniciu, “Mean shift: a robust approach toward feature space analysis,” IEEE
Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002).
64. M. Grizonnet and J. Inglada, “Monteverdi—remote sensing software from educational to
operational context, remote sensing for science, education, and natural and cultural herit-
age,” in EARSeL, R. Reuter Ed., pp. 749–756 (2010).
65. The ORFEO Tool Box Software Guide Updated for OTB-4.4, https://www.orfeo-toolbox.
org//packages/OTBSoftwareGuide.pdf (September 2014).

66. C. Lee and E. Choi, “Bayes error evaluation of the Gaussian ML classifier,” IEEE Trans.
Geosci. Remote Sens. 38(3), 1471–1475 (2000).
67. E. Choi and C. Lee, “Feature extraction based on the Bhattacharyya distance,” Pattern
Recognit. 36(8), 1703–1709 (2003).
68. V. Vapnik, Statistical Learning Theory, Wiley, New York (1998).
69. M. Volpi et al., “Supervised change detection in VHR images using contextual information
and support vector machines,” Int. J. Appl. Earth Obs. Geoinf. 20, 77–85 (2013).
70. G. Camps-Valls and L. Bruzzone, Kernel Methods for Remote Sensing Data Analysis, John
Wiley & Sons, Chichester, United Kingdom (2009).
71. L. Bruzzone and L. Carlin, “A multilevel context-based system for classification of very
high spatial resolution images,” IEEE Trans. Geosci. Remote Sens. 44, 2587–2892 (2006).
72. J. Inglada, “Automatic recognition of man-made objects in high resolution optical remote
sensing images by SVM classification of geometric image features,” ISPRS J. Photogramm.
Remote Sens. 62, 236–248 (2007).
73. D. Tuia et al., “Classification of very high spatial resolution imagery using mathematical
morphology and support vector machines,” IEEE Trans. Geosci. Remote Sens. 47, 3866–
3879 (2009).
74. B. Boser, I. Guyon, and V. Vapnik, “A training algorithm for optimal margin classifiers,” in
Proc. 5th ACM Workshop COLT, pp. 144–152 (1992).
75. R. G. Congalton, “A review of assessing the accuracy of classifications of remotely sensed
data,” Remote Sens. Environ. 37(1), 35–46 (1991).
76. G. M. Foody, “Status of land cover classification accuracy assessment,” Remote Sens.
Environ. 80(1), 185–201 (2002).
77. R.G. Congalton and R.A. Mead, “A quantitative method to test for consistency and correct-
ness in photointerpretation,” Photogramm. Eng. Remote Sens. 49(1), 69–74 (1983).
78. R.G. Congalton and K. Green, Assessing the Accuracy of Remotely Sensed Data: Principles
and Practices, 2nd ed., CRC Press, Taylor & Francis, Boca Raton, Florida (2008).
79. R.-E. Fan, P.-H. Chen, and C.-J. Lin, “Working set selection using second order information
for training SVM,” J. Mach. Learn. Res. 6, 1889–1918 (2005).
80. A.C. Teodoro and L. Duarte, “Forest fire risk maps: a GIS open source application– a case
study in Norwest of Portugal,” Int. J. Geogr. Inf. Sci. 27(4), 699–720 (2013).
Ana Teodoro received her BS degree in surveying engineering, MS degree in geomatica, and
PhD degree in geomatica from the University of Porto, Porto, Portugal, in 1998, 2001, and 2007,
respectively. Since 1999, she has been with the University of Porto. Between 1999 and 2010, she
was with the applied mathematics department as a teaching assistant (until 2007) and assistant
professor (2007–2010). She is currently an assistant professor in the Department of Geosciences,
Environment, and Land Planning at Faculty of Sciences, University of Porto. She is a scientific
researcher in the Institute of Earth Sciences (ICT), University of Porto. Her research interests
include remote sensing: coastal zones, environmental and urban applications, image processing,
and GIS.
Ricardo Araujo received his BS degree in surveying engineering, from the University of Porto,
Porto, Portugal, in 2015. Since 2013, he has been work in several scientific projects related to
image processing and remote sensing at the University of Porto. His research interests include
remote sensing, UAV data acquisition and processing, and GIS.

Teodoro 2016

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Teodoro 2016

Uploaded by

Copyright:

Available Formats

Comparison of performance of

object-based image analysis

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Ana C. Teodoroa,b,* and Ricardo Araujob

*Address all correspondence to: Ana C. Teodoro, E-mail: amteodor@fc.up.pt

Journal of Applied Remote Sensing 016011-1 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Journal of Applied Remote Sensing 016011-2 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Journal of Applied Remote Sensing 016011-3 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

3.1 Data and Study Area

Journal of Applied Remote Sensing 016011-4 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Fig. 2 Orthomosaic produced (with the 10 ROIs marked).

Journal of Applied Remote Sensing 016011-5 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

3.2.1 Region growing algorithm

3.2.2 Mean shift segmentation algorithm

Journal of Applied Remote Sensing 016011-6 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Journal of Applied Remote Sensing 016011-7 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

algorithms, available in Spring and in OTB/Monteverdi, were applied. Descriptions of these

3.3.1 Bhattacharya classifier

where μ is the mean of each class.

3.3.2 Support vector machine

Journal of Applied Remote Sensing 016011-8 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

More details about this classifier can be found in Volpi et al.69

3.3.3 Evaluating the performance of a classifier

4.1 Regions of Interest Definition

Journal of Applied Remote Sensing 016011-9 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Journal of Applied Remote Sensing 016011-10 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Table 1 Classes identified for each ROI considering a maximum of four

ROI Classes Spring (maximum of 4)

I 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement

II 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement

III 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement

IV 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement

V 1-Vegetation 2-Habitation 3-Outdoor benches 4-Bare soil

VI 1-Vegetation 2-Habitation 3-Asphalt 4-Pavement

VII 1-Vegetation 2-Habitation 3-Water 4-Wall

VIII 1- Sports flooring 2-White lines 3-Yellow lines

IX 1-Vegetation 2-Water 3-Bare soil

X 1-Rooftop 2-White pipe

Journal of Applied Remote Sensing 016011-11 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

ROI OA (%) Kappa PA (%) UA(%)

I 100.00 1 100:100;100:100 100:100;100:100

II 99.17 0.976 100:100;100:100 100:100;100:100

III 84.30 0.802 100:100;100:100 100:100;100:100

IV 69.85 0.598 100;90.1;85.2;100 100;100;100;53.1

V 100.00 1 100:100;100:100 100:100;100:100

VI 97.09 0.961 100;100;100;74.6 100;65.4;100;100

VII 100.00 1 100:100;100;100 100:100;100;100

VIII 100.00 1 100:100;100 100:100;100

IX 98.64 0.978 100:100;100 100:100;100

X 100 1 100:100 100:100

Journal of Applied Remote Sensing 016011-12 Jan–Mar 2016 • Vol. 10(1)

Downloaded From: http://remotesensing.spiedigitallibrary.org/ on 02/18/2016 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Journal of Applied Remote Sensing 016011-13 Jan–Mar 2016 • Vol. 10(1)