Multispectral Image Classification of Shaded and Non-

Shaded Land-Cover Patterns

Wanessa da Silva , Jos Renato Garcia Braga , Mateus Habermann, Elcio

Hideiti Shiguemori, Haroldo Fraga de Campos Velho

1 Geointelligence Division, Institute for Advanced Studies IEAv, So Jos dos Campos,
Brazil
2 National Institute for Space Research INPE, So Jos dos Campos, Brazil
{wanessa.ieav, jgarciabraga}@gmail.com, {habermann, elcio}@ieav.cta.br, harol-
do@lac.inpe.br

Abstract. This work addresses the multispectral image classification consider-

ing shaded land-cover patterns. In order to achieve this purpose, a conjunction
of Artificial Neural Network and methods of dimensionality reduction (Princi-
pal Components Analysis and Non-Linear Principal Components Analysis) was
used. The robustness of Artificial Neural Network approach and the resem-
blance of shaded and non-shaded multispectral patterns permitted a good classi-
fication of land-cover areas.

1 Introduction

In situations in which occur the presence of shades over some regions it is difficult to
perform a correct classification of multispectral images [1]. Thus, there is a concern
for detecting and classifying shade spots on aerial images [2][3][4].
The automatic pattern classification can be compromised when a region of an im-
age is covered by shade [1][2]. In a previous research, the classification of shaded
patterns yielded poorly because those elements were not used during the training
phase of the algorithm [1]. In this paper, the same problem addressed in [1] has been
studied, but considering the presence of shades, and, then, a comparison between the
two studies is done. It is desired to investigate if the neural network may improve the
classification even with the presence of shade.
The multispectral images employed in this study have seven main classes: body
water, asphalt road, red tile roof, concrete, bare soil, trees and grass. The algorithms
are Artificial Neural Network (ANN), Principal Components Analysis (PCA) [7] and
Non Linear Principal Components Analysis (NLPCA) [10]. The Artificial Neural
Network used in this work is Multilayer Perceptron (MLP), for it features an efficient
method and it has been successfully employed in a wide range of problems, including
pattern classification [7]. To determine the optimal or near optimal architecture of
MLP, the Multiple Particle Collision Algorithm (MPCA) was used [6]. PCA has been
employed to find a new set of orthogonal axes in order to decrease the overall data
variance [7], and it has been widely used in data dimension reduction of multispectral
images [8]. The Non-linear Principal Components Analysis (NLPCA) [9] also has
been employed to perform the data dimension reduction. The result obtained from
classification using data reduction by PCA are compared with the results produced by
NLPCA. For this very purpose, PCA was also used in this work.
The main objective of this work is evaluated of pattern classification of shaded
and non-shaded land cover in multispectral image that were reduced by PCA and
NLPCA.

2 Multispectral Images

A multispectral image may be considered as a set of scenes taken simultaneously

from a single area, which are acquired by a sensor that is capable of sensing several
electromagnetic radiation intervals.
The multispectral images acquisition is performed by sensors orbital or suborbital
levels, such as HSS (Hyperspectral Scanner System), which goes onboard an aircraft,
and delivers 3 meters-spatial-resolution images, depending on the flight level [11]
Multispectral images classification is used to extract information from remote sens-
ing data, and then it performs the clustering of homogeneous regions on an image.
Since most remote sensing images feature coarse spatial resolution, their wide spec-
tral range allows the distinction amongst different targets [7].
Before the classification itself, the data often undergo a preprocessing step, which
means the transformation of raw data in terms of radiometry and geometry.

3 Classification Using ANN+PCA and ANN+NLPCA

The Neural Network here employed is the Multilayer Perceptron (MLP), because it
has an efficient method and it has been successfully used in a wide range of applica-
tions, including Pattern Classification [7].
Seeking an optimal architecture of a MLP may be considered an optimization prob-
lem (referenciar beceneri). Equation 1 is the objective function of an MLP.

(1)

where parameter equals 1, and , are Etrain and Egen relevance ad-
justment terms, and their values are 0.1 and 1, respectively.

2
(2)

=1


2 (3)

=1

The training (Equation 2) and generalization (Equation 3) errors are MLPs archi-
tecture-dependent.
In order to determine the optimal architecture of all MLP, the MPCA algorithm
was employed. It is an extension of PCA [5], and it seeks a set of variables that mini-
mizes the objective function of Equation 1.
In this work, HSS sensor images with 37 bands (VNIR, SWIR) were employed,
ranging from 0.44 until 2.4 m [11].

The collected data undergo the normalization described in (1).

(1)

Where X, Min and Max refer to, respectively, each class element of the data, mini-
mum band value of a particular element, and maximum band value of that same ele-
ment.
The dimensionality of data set has been reduced by PCA and NLPCA.

A) Principal Components Analysis.

PCA is a technique for forming new variables which are linear composites of the
original ones. The maximum number of variables is equal to the number of original
variables, and these new variables are uncorrelated among themselves [7].
PCA calculates the singular value decomposition of the covariance matrix of the
data, and using only the singular values, or small part of this set, it performs a dimen-
sional reduction of data.

B) Non-Linear Principal Components Analysis.

The fact that the PCA achieve the reduction in dimensionality of the data by a linear
transformation can lead to an undesirable loss of information. This limitation motivat-
ed the study of new techniques of non-linear data reduction.
One of these techniques is the NLPCA, which uses an self-associative MLP to
promote the data reduction [9][10].
The architecture of MLP self-associative is shown in Figure 1.
 Fig. 1 Self-associative MLP, operator of NLPCA method to data reduction.

The purpose of self-associative MLP is to produce at the output of neural network an

approximation of input data. From the layer called bottleneck, which has a small
amount of artificial neurons, there is a reconstruction of data, thus this layer has a
non-linear representation of the data with a reduced dimensionality.
The optimal architecture of self-associative MLP, which will be the operator of
NLPCA, is shown in Table 1.

Table 1. Optimal architecture determined by MPCA for self-associative MLP, which will
perform the NLPCA.
Optimal Configuration of Self-associative MLP

Input vectors dimension 37 37

Mapping Layer 25 28
Bottleneck Layer 7 11
Remapping Layer 25 28
Output vectors dimension 37 37
Number of patterns for training 55 680
Number of patterns for validation 15 20
Number of patterns for test 35 140

The data reduction obtained by NLPCA was 37 to 7 in the data non-shaded and 37 to
11 in the data shaded.

The MLP Neural Network was designed to use the first six principal components,
which involve 100% of the data variance in shaded data or non-shaded data. Table 2
exhibits the configuration of MLP which will be performed the data classification.
Table 2. Optimal configuration of MLP determined by MPCA, which will perform the data
classification.
Network Configuration Table

ANN ANN ANN ANN

Settings + + + +
PCA NLPCA PCA NLPCA
Input vectors dimension 6 7 6 11
Output vectors dimension 3 3 3 3
Number of neuron of
10 12 11 15
hidden layer
Number of patterns for
55 55 680 680
training
Number of patterns for
15 15 20 20
validation
Number of patterns for
35 35 140 140
testing

4 Tests

The objective of this work is to present an approach for multispectral image classi-
fication by means of ANN. Besides, the comparison amongst results here achieved is
done. Thus, it was necessary to select and characterize regions of the images, in order
to obtain their reflectance values, and, then, to proceed to the data normalization be-
fore using them in an ANN. Before the ANN training phase, the data dimension is
reduced by PCA or NLPCA.
In total, 4 networks were designed. Two of them were trained and generalized with
images with non-shaded data, and those will be henceforth called ; and the other
two remaining networks used images comprising shadowed areas, and those networks
will be called
networks had their data reduced by either PCA or NLPCA, and the same oc-
curred with one networks. There are, thus, 4 experiments: ANN+PCA and
ANN+NLPCA for and groups.
For all the above-cited cases, it was sought an optimal MLP configuration by
means of MPCA.
The images considered in this work were obtained in may/30 th/2006, over So Jos
dos Campos, by HSS airborne sensor, with 2.9 meters of spatial resolution in Nadir
[11].
A total of 945 regions 33 matrices were extracted. The targets are body water,
asphalt road, red tile roof, concrete, bare soil, trees and grass. In Table II are shown
examples of those classes and their spectral signatures.
Input data: for it was used 10 elements of each class for the training phase, and
5 elements of each kind for the generalization phase; whereas for the respec-
tive amounts were 100 and 20. Table 3 displays the class of interest in the image.
Table 3. Patterns example, class of interest in the image.
Examples
Classes Spectral Signature

Body water

Asphalt road

Red tile roof

Concrete

Bare soil

Trees

Grass

In this work, regions of the image with shadow have special attention. These re-
gions have been used in the training and in the generalization test phases. As it is
desired to investigate if the neural network may improve the classification standards
even with the presence of shade, in Figure 2, spectral signatures of shaded and non-
shaded grass regions are exhibited.
 Fig. 2. Spectral signatures of shaded and non-shaded grass regions

It can be seen from Fig. 1 that the both signatures contain similar spectral infor-
mation, therefore one may infer that they belong to the same class. Thus, those data
may be used in an ANN in order to improve the classification of shaded areas.
Thus, the strength of agreement of the classifiers has been measured by means of
Kappa index (K) [12], according to Table 4.

Table 4. Interpretation of the kappa index values.

Kappa statistic (K) Strength of Agreement
0 Poor
0 < K 0.2 Slight
0.2 < K 0.4 Fair
0.4 < K 0.6 Moderate
0.6 < K 0.8 Substantial
0.8 < K 1 Almost Perfect

4.1 Test results of classification of patterns non-shaded.

After of data reduction by PCA and NLPCA and training of MLP to perform the data
classification, 4 regions formed by pixels of the classes of interest (shown in Figure 3
A. Where A.1.1, A.2.1, A.3.1, A.4.1 are respectively Region 1, Region 2, Region 3
and Region 4), were reduced by PCA, NLPCA and, subsequently, classified by MLP.
Table 5 displays the results of kappa index of classification of non-shaded data. The
Figure 3 shows the result of classification of 4 regions of interest, in Figure 3 B are
the ground-truth of 4 regions of interest, where B.1.1, B.2.1, B.3.1 and B.4.1 are the
ground-truth of data non-shaded, so there are pixels unclassified, in B.21, B.2.2, B.3.2
and B.4.2 are the ground-truth of data shaded. The Figure 3 C.1.1, C.2.1, C.3.1, C.4.1
show the result of data classification, which used the PCA method to data reduction
and the Figure 3 C.2.1, C.2.2, C.2.3 and C.2.4 show the result of data classification,
which used the NLPCA method to data reduction.

Table 5. Results of classification of data non-shaded.

PCA + NLPCA +
Kappa Index Kappa Index
Region 1 0.3783 0.4075
Region 2 0.7800 0.4996
Region 3 0.8266 0.6763
Region 4 0.3765 0.6177

4.2 Test Results of patterns with shaded.

After of data reduction by PCA and NLPCA and training of MLP to perform the data
classification, 4 regions formed by pixels of the classes of interest, were reduced by
PCA, NLPCA and, subsequently, classified by MLP. Table 6 displays the results of
kappa index of classification shaded data. The Figure 3 D.1.1, D.2.1, D.3.1, D.4.1
show the result of data classification, which used the PCA method to data reduction
and the Figure 3 D.2.1, D.2.2, D.2.3 and D.2.4 show the result of data classification,
which used the NLPCA method to data reduction.

Table 6. Results of classification of data shaded.

PCA + NLPCA +
Kappa Index Kappa Index
Region 1 0.6861 0.5701
Region 2 0.8629 0.8351
Region 3 0.7643 0.8255
Region 4 0.7671 0.7434

5 Conclusion

This work addressed the problem of multispectral image classification of shaded

data and non-shaded data using artificial neural networks. Different situations were
compared, considering the use of MLP, the MLP combined with PCA, the MLP com-
bined with NLPCA and usage patterns of shaded data and non-shaded data in the
training set and generalization test. As expected, the results obtained indicate that the
artificial neural network shows satisfactory results in classification as well as the
classification with the use of shaded patterns, compared with classification of non-
shaded patterns considering this. Comparing the results produced by methods of re-
ducing the dimensionality of the data, the PCA presents with an effective method, as
well as the NLPCA. In future work will be considered different categories of shad-
ows, which may vary with weather conditions and times of day.
The Figure 3 show the results of all testing of classification.

Fig. 3 In A is shown the 4 regions of interest, where A.1.1, A.2.1, A.3.1, A.4.1 are the Region 1, Region 2,
Region 3, Region 4 respectively. In B.1.1, B.2.1, B.3.1 and B.4.1 are shown the ground-truth of non-shaded
data, and in B.1.2, B.2.2, B.3.2 and B.4.2 are shown the ground-truth of shaded data. In C is the result of
non-shaded data classification, where C.1.1, C.2.1, C.3.1 and C.4.1 are the result of data which were
reduced by PCA. In C.1.2, C.2.2, C.3.2 and C.4.2 are the result of data which were reduced by NLPCA. In
D is the result of shaded data classification, where D.1.1, D.2.1, D.3.1 and D.4.1 are the result of data
which were reduced by PCA. In D.1.2, D.2.2, D.3.2 and D.4.2 are the result of data which were reduced by
NLPCA

The Figure 4 show the legend used to distinguish the interest classes in Figure 3.

Fig. 4 Legend used to distinguish the interest classes.

References

1. Silva, W.; Habermann, M.; Shiguemori, E.H.; Andrade, L.L.; Castro, R.M., Multispectral
image classification using Multilayer Perceptron and Principal Components Analysis. In:
BRICS-CCI Congress on Computacional Intelligence, 11, 2013, Porto de Galinhas.
2. Makarau, A.; Richter, R.; Muller, R.; Reinartz, P., "Adaptive Shadow Detection Using a
Blackbody Radiator Model," Geoscience and Remote Sensing, IEEE Transactions on ,
vol.49, no.6, pp.2049,2059, June 2011.
3. Yi-Ta Hsieh; Shou-Tsung Wu; Chen-Sung Liao; Yau-Guang Yui; Jan-Chang Chen; Yuh-
Lurng Chung, "Automatic extraction of shadow and non-shadow landslide area from ADS-
40 image by stratified classification," Geoscience and Remote Sensing Symposium
(IGARSS), 2011 IEEE International , vol., no., pp.3050,3053, 24-29 July 2011.
4. Ruizhi Ren; Lingjia Gu; Haofeng Wang, "Clouds and Clouds Shadows Detection and Match-
ing in MODIS Multispectral Satellite Images," Industrial Control and Electronics Engineer-
ing (ICICEE), 2012 International Conference on , vol., no., pp.71,74, 23-25 Aug. 2012.
5. Sacco, W. F., Oliveira, C. R. E. A. A new stochastic optimization algorithm based on a
particle collision methaeuristic. 6th World Congress of Structural and Multidisciplinary
Optimization, Rio de Janeiro (1986).1
6. Luz, E.F.P., Becceneri, J.C., Velho, H.F.C. A new multi-particle collision algorithm for
optimization in a high-performance environment. Journal of Computacional Interdiscipli-
nary Sciences, pp 1--7, v. 1, (2008).
7. R. O. Duda, P. E. Hart, D. G. Stork, Pattern Classification, John Wiley and Sons, 2006.
8. Agarwal, A.; El-Ghazawi, T.; El-Askary, H.; Le-Moigne, J., "Efficient Hierarchical-PCA
Dimension Reduction for Hyperspectral Imagery," Signal Processing and Information Tech-
nology, 2007 IEEE International Symposium on , vol., no., pp.353,356, 15-18 Dec. 2007.
9. Licciardi, G., Reddy, P. M., Chanussot, J. , Benediktsson, J. A. Linear Versus Non-linear
PCA for the Classification of Hyperspectral Data Based on the Extended Morphological
Profiles. IEEE Geoscience and Remote Sensing Letters, pp 447--451, v. 9, (2012).
10. Del Frate, F., Licciardi, G., Duca, R. Autoassociative Neural Networks For Features
Reduction of Hyperspectral Data. IEEE Geoscience and Remote Sensing Letters, pp 447--
451, v. 9, (2009).
11. Castro, R.M.; Moreira, R.C.; Esposito, E.S.C.; Lucca, E.V.D., Avaliao do rudo em
sensores eletro-pticos: abordagem da imagem escura no HSS. In: Simpsio Brasileiro de
Sensoriamento Remoto, 12, 2005, Goinia.
12. Cohen, J., A coefficient of agreement for nominal scales, Educational and Psychological
Measurement, 1960, 20, 37-46.