Advanced Methods of Thermal Image Processing for

Medical and Biological Applications

B. Wicek
1
, C. Peszynski-Drews
2
, M. Wysocki
2
, T. Jakubowska
2
,
R.Danych
1
, S. Zwolenik
1

1
Institute of Electronics, Technical University of Lodz
2
Laser Diagnostic and Therapy Center, Technical University of Lodz
Abstract
In this paper, advanced image processing tools are presented for conversion thermal and visual images,
mainly for application in medicine and biology. A novel method for area and distance evaluation based on
statistical differencing is discussed. In order to increase the measurements accuracy, the interpolation and
subpixel bitmap processing are chosen. Additionally, the usefulness of the first and second order statistical
parameter to calculate signatures of thermal images to distinguish healthy breast one with malignant tumor. We
use image histogram and co-occurrence matrix to get thermal signatures and analyze symmetry between left and
right side. The proposed approach was used in the pilot investigations in the medical centre which is permanently
using thermography for breast cancer screening and therapy, as an adjacent method for other classical therapy,
such as mammography.

1. Introduction
Defining regions of interest precisely is one of the important problem in thermal and
visual image processing in medicine and biology. Once the boundaries of the investigated
objects are calculated, thermal parameters, such as average temperature, standard
deviation, histograms, etc. can be easily obtained. Morphological algorithms for finding
boundaries with sufficient accuracy of area estimation are one of the possible approach [8-
11]. Many professional packages are using methods such as skeletoning, opening, dilatation
and erosion, although in many practical cases, they are not successfully applied when the
image is colorful with shadows. Our approach assumes to use thermal and visual images
captured almost simultaneously. Visual image is provides enough information to generate
boundary mask that is applied to get the area of interest on thermal scene.
Breast cancer and other breast diseases are a major issue in women's health today,
particularly in advanced industrialized nations. It takes years for a tumor to grow, and the
earliest possible indication of abnormality is needed to allow for the earliest possible
treatment and intervention. Thermal imaging can be useful as early diagnostic technique that
can detect breast cancer. One of the principal feature of breast thermographs is symmetry of
temperature pattern. Thermal images of breast are usually asymmetrical in pathological
cases. Any significant asymmetries, or any specific blood vessel patterns in one breast that
do not appear in the other indicate a physiologic abnormality. This may be pathological
(including cancer, fibrocystic disease, an infection or a vascular disease) or it might indicate
an anatomical variant. [1,8,9]
The aim of this research is to find image features that describe symmetry of breast
thermographs and to use these parameters to separate thermal image of breast with and
without malignant tumors. There are many different methods that describe image features.
Large group of methods is based on statistical parameters calculations. Parameters like
mean value, standard deviation, skewness, kurtosis, etc. can be used to compare thermal
images. We consider both the first and second order statistical parameters [2,7]. The first
order statistical parameters methods use images histogram to compute signatures, while the
second order statistical parameters are defined for so-called co-occurrence matrix of the
image.
The next group if methods are based on image transformations, such as linear filtering,
Fourier or wavelet analysis. All these methods allow to regenerate an image which is
processed or converted, and signatures are defined in different domain, eg.: space frequency
or scale domain. Well-known Karhunen-Loeve transform is implemented in form of PCA
2
(Principle Component Analysis). PCA is a technique that is usually used for reducing the
dimensionality of multivariate data preserving most of the variance [4-6].
2. Processing of thermal images
2.1. Statistical differencing
One of the method for contrast enhancement and proper edge filtering is based on
statistical differencing, where each pixel value is scaled by the standard deviation calculated
in its neighborhood, as below [8]:

) , (
) , (
) , (
j i S
j i F
j i G =

(1)
where: F(i,j,), S(i,j,) denote the original pixel value and standard deviation, respectively.
The higher standard deviation, the lower pixel value. It means, that the part of the image
with low color or luminance variation will be enhanced, and more image details are visible.
Standard deviation for WxW window of the image is calculated as:

( ) ( ) [ ]
2 / 1
2
, ,
1
) , (
(

=

+

w i
w i
w j
w j
n m M n m F
W
j i S

(2)
where M(m,n) denotes the mean value of the part of the image in the window.
There is a question of the size of the window for calculating standard deviation. In
practice, it can vary from 4x4 up to 32x32, but we are aware, that larger windows are
statistically more significant. This method is very sensitive to the noise level, and
preprocessing in the form of low pass filtering or averaging is recommended. Lately, Wallis
introduced the modified operator for statistical differencing based on first and second order
moments [6,8].
2.2. Interpolation and subpixel resampling
Due to the limited sampling frequency during the image acquisition by digital system, the
thermal picture resolution is typically low, and not sufficient for more accurate calculation. It is
especially required for medical applications, where often the temperature change is very low.
Interpolation and resampling is one of the method to improve the accuracy of geometrical
parameter estimation, such as areas, lengths, mean temperature in regions, etc. There are
many interpolation functions that can be used for image processing. Sinc, square, triangle,
bell, cubic, Gaussian functions are the most used. Obviously, the nonlinear interpolation
gives better results as it involves more pixels in the processing.
Tradeoff between processing time and the order of interpolation function has to be
considered. For thermal images, which are typically of lower resolution, the cubic
interpolation can be easily implemented. Normalized cubic B-spline interpolation function
takes a form [8]:

( )
( )

s s
s s +
=
2 1 2
6
1
1 0
2
1
3
2
) (
3
2 3
x for x
x for x x
x R

(3)

2.3. Adaptive histogram modification
Histogram modification is typically used for contrast enhancement. During image
capturing, the signal from the camera is quantized, but often not in the full range.
Additionally, using linear quantization, the scene is often dark, with plenty of non-visible
details due to the nonlinear characteristic of human eye sensitivity. Among the known
methods, the linear and nonlinear scaling of pixel luminance or color components are mostly
used. Histogram hyperbolization is applied to compensate eye characteristics, however it is
computationally intensive and it is difficult to get contrast enhancement in real time. In order
3
to choose the right transfer function for histogram modification, we need to know the
probability density model of the image. In non-adaptive approach, uniform, exponential,
Rayleigh, hyperbolic model are used.
Lately, the adaptive histogram modification has been introduced [8]. In such a method
different transfer functions are involved for different pixels in an image. The new approach is
based on moving window and the histogram is computed for any pixel neighborhood.
3. Application for inflammation
Thermal, visual and radiological image processing in parallel brings many advantages.
As shown in Figure 1, there is a visible correlation between thermal and radiological images
during pneumonia inflammation in children. The hot spots indicate the initial state of
pneumonia [11]. The investigations seem to be very practical, while the thermography is
much less invasive and stressed for human body in comparison to radiological tests.
Histograms modification is one the useful method of radiological images processing for
contrast enhancement, as shown in Figure 2. One of the most helpful functions is on-line
interactive histogram processing, together with image segmentation (Figure 3). Enhanced
image is then binarised by user-defined threshold level. One of the important problems is to
define optimal threshold level. This level can be found by fixing the margins for the histogram
and eliminating pixel below and above certain levels as shown in Figure 3.


*>35,0C
*<32,0C
32,0
32,5
33,0
33,5
34,0
34,5
35,0
*>35,0C
*<32,0C
32,0
32,5
33,0
33,5
34,0
34,5
35,0

Figure 1: Thermal and radiological images of a child with pneumonia inflammation, initial state
of the disease, and after 14 days of treatment





Figure 2: Radiological image processing by histogram equalizing


The algorithm is fully interactive, and the user can adjust the margins for the histogram,
as well as the threshold level for segmentation. The histogram equalizing together with
binarisation makes possible to find a border of the inflamed area. The similar procedure can
4
be performed for thermal images. It could hopefully replace radiological inspections by
thermal ones in the future. Such investigations are the very preliminary, and should be
confirmed by more cases, not only for pneumonia inflammation. The first trials are very
promising. We believe that thermoregulation of the human body will not reduce significantly
thermal effects on the skin caused by internal temperature increase.




Figure 3: Image segmentation by user defined threshold level

4. Area and thermal signature measurement
Low temperature variation, and low thermal and visual contrast in many applications
cause the demand of high accuracy in image processing, especially in biological and medical
applications. As an application, we have chosen the analysis of leaves during plants growth
in different conditions temperature, humidity, pressure, sun illumination, etc.



Figure 4: Thermal image of the plant used for analysis, non-recognized area of interest




Figure 5: Original visual image of the plant used for analysis with shadows which make the
processing difficult

The aim of the research was to calculate mean temperature and standard deviation in the
region of interest very precisely defined on visual image captured together with thermal one.
A few algorithms have been tested, and because of colorful visual images used, we
5
proposed novel approaches based on separate processing of color components (R-red, G-
green, B-blue) of the image. Although the leaves look green, the more precise analysis
shows, that it is impossible to make the proper segmentation by differentiating using single
color component. Additionally, the analysis is very sensitive to the shading and external
illumination. In the general case, it may not be assumed that the light position is always the
same, so the problem of e.g. sun light illumination has to be carefully considered.
The following procedure is proposed:
Statistical differencing filtering with window WxW
Segmentation with threshold TH
Median filtering
Differencing filtering can work with different windows size, which can vary from 10x10 to
100x100 pixels. Having smaller window we can visualize more details, depending on their
local statistical characteristics. On the other hand, larger window makes the edge of the area
of interest smother, and some negligible details are automatically removed. It is clearly
presented in Figure 8.
Segmentation is performed globally using three R,G,B components of each pixel.
Formula below describes how the segmentation is defined in the presented method.

( ) ( )
( ) ( )

+ >
+ >
=
otherwise
j i R TH j i G
and j i B TH j i G
if
j i S
1
, ,
, ,
0
) , (

(4)

where S(i,j) is the value of the pixel after segmentation, and R,G,B denote color
components of the original image.
The threshold level TH for segmentation has to be chosen very carefully. Using too low
TH value, we can artificially place many non expected details, especially on the border of the
considered area, while the high TH value causes the unexpected thinning of the object. All
these features of the method are presented in Figures 7-8.
Nonlinear median filtering successfully removes the border non-continuity, pixels or their
groups randomly placed next to the area of interest. Median filtering was included into the
algorithm mainly because of shading existing in the image. Shadows can significantly change
the results of area measurements, and as a consequence, the thermal index for the region
can vary. In order to avoid shadows in the images, the median filter ends the algorithm.
Quantitative results confirm the influence of windows size and threshold level on
accuracy of region measurement. In Table 1, relative errors are presented. It shows the area
of the region obtained by the proposed algorithm referring to the area measured manually
(Figure 6).


31x31(WxW) 21x21(WxW) 11x11(WxW)
TH=30 0.41 6.5 21.6
TH=40 12.8 7.3 7.0
TH=50 23.5 18.4 5.9
TH=60 34.0 28.9 18.4
Table 1: Relative error of area measurement by the proposed (in %)







6


Figure 6: Visual image after statistical differencing filter processing histogram equalizing,
contrast enhancement, more details visible (left), region of interest obtained manually (right)




Figure 7: Influence of threshold level in the proposed method on the area if interest: TH=30,
WxW=31x31 (left), TH=60, WxW=31x31(right)




Figure 8: Influence of window size for statistical differencing on the area if interest: TH=40,
WxW=31x31(left), TH=40, WxW=11x11 (right)


5. Histogram-based first-order thermal signatures
An image is assumed to be a rectangular matrix of discretised data (pixels) pix[m,n],
where m=0, 1,,M, n=0,1,N. Each pixel takes value from the range ie<0,L1>. The
histogram describes the frequency of existence of pixels of the same intensity in whole
image or in the region of interest (ROI). Formally, the histogram represents the distribution of
the probability function of the existence of the given intensity in the image and it is defined
using Kronecker delta function as:

=
=
=
= =

i n m p for
i n m p for
i n m p where
L i for i x m p i H
N
n
M
m
] , [ 0
] , [ 1
) ], , [ ( :
, 1 ,... 1 , 0 ) ], , [ ( ) (
1 1
o
o

(5)

Histogram is used for defining first order statistical features of the image, such as mean
value
H
, variance o
H
, skewness, kurtosis, energy and entropy. The definitions of these
parameters are given below.
7
) ( ) ) ( ( log
) (
3
) ( ) (
) ( ) (
) ( ) (
) (
1
0
2
1
0
2
4
1
0
4
3
1
0
3
1
0
2
1
0
i p i p entropy
i p energy
i p i
kurtosis
i p i
skewness
i p i
i ip
L
i
L
i
H
L
i
H
H
L
i
H
L
i
H H
L
i
H

=
=
=

=
=
=

(6)

Histogram gives global information on the image. By converting histogram we can obtain
some very useful image improvement, such as contrast enhancement [1-2]. In this paper, the
first order statistical parameters were used to separate physiological and pathological breast
thermal images. The results are in the next chapter.
6. Second order statistical parameters
More advanced statistical information on thermal images can be derived from second
order parameters. They are defined on so-called co-occurrence matrix. Such a matrix
represents the joint probability of two pixels having i-th and j-th intensity at the different
distances d, in the different directions. Co-occurrence matrix gives more information on
intensity distribution over the whole image, and in this sense it can effectively be used to
separate and classify thermal images.
Lets assume that each pixel has 8 neighbors lying in 4 directions: horizontal, vertical,
diagonal and antidiagonal. In this work we consider only the nearest neighbors, so the
distance d=1.


6 7 8
5 * 1
4 3 2


Figure 9: 8 neighbouring pixels in 4 direct horizontal, vertical, diagonal and antidiagonal


As an example lets take an image 4x4 with 4 intensity levels given as:


1 1
1
1 1 1
2 2
2
2 2 2
3
3
0 0


Figure 10: Example of 4x4 image with 4 discrete intensity levels


8
For horizontal direction the co-occurrence matrix takes a form:

(
(
(
(

=
0 2 2 0
2 4 2 0
2 2 4 1
0 0 1 2
horisontal
m

(7)

[ ]

=
=
+
=

=
= = =
=
=
=
1
0
2
1
0
1
0
2
1
0
1
0
1
0
1
0
1
0
1
0
2
1
0
1
0
2
1
0
) , ( log ) , (
) ( 1
) , (
) , ( ) (
2 ,..., 1 , 0 , , ) , ( ) (
) ( var
) , ( ) (
) , (
L
j
L
i
L
j
L
i
y x
L
j
y x
L
i
L
j
L
i
y x
y x
L
j
L
i
L
j
L
i
j i p j i p Entropy
j i
j i p
difference Inverse
j i p ij
n Correlatio
G l l j i for j i p l p where
p Var iance Difference
j i p j i Variance
j i p Energy
o o


(8)

The co-occurrence matrix is always square and diagonal with the dimension equal to the
number of intensity levels in the image. After normalization we get the matrix of the
probabilities p(i,j). Normalization is done by dividing the all elements by number of possible
couple pixels for a given direction of analysis. For horizontal and vertical directions this
number is equal to 2N(M-1) and 2M(N-1), while for diagonal directions it is 2(M-1)(N-1),
respectively.
In the above equations
x
,
y
, o
x
, o
y
are mean values and standard deviations for the
elements of co-occurrence matrixes calculated for horizontal and vertical directions,
respectively.
7. Results in investigations of breast tumors
32 healthy patients and 10 patients with malignant tumors were analyzed using
thermography. There were four images registered for every patient that represented each
breast in direct and lateral direction to the camera. Histograms were created for these
images and on the basis statistical parameters were calculated: mean temperature, standard
deviation, variance, skewness and kurtosis. Afterwards, absolute differences for parameters
values for left and right breast were calculated. There was estimated degree of symmetry on
basis of these differences.
Mean temperature in healthy group was estimated at the level 30.24 1.78 C in direct
position and 29.75 1.86C in lateral one. Mean temperature was higher in 8 cases of 10 in
malignant tumor group. Moreover, there were 6 cases of 32 in healthy group which mean
temperature exceeded normal temperature range. Therefore, we have found that it is
necessary to analyze symmetry between left and right breast. Comparison of mean
temperature was not sufficient to separate physiological and pathological images. Among
analyzed parameters skewness was the most promising for successful classification of
thermal images. Absolute differences of skewness for left and right side was equal to 0.41
0.34 C in frontal position and 0.63 0.46 C in lateral one in healthy group. These
differences were higher for images in lateral position in all cases in pathological group in
comparison to healthy patients images.
9



Figure 11: Thermal image of the healthy breast




Figure 12: Thermal image of the breast with malignant tumor (left side)


The figures below confirm the evidence of asymmetry between left and right side for
healthy and malignant tumor cases.
Analyzing the first order statistical parameter let us conclude that it is quite hard to use
them for the image classification and detecting tumors. Only mean temperature and the
skewness let us to separate and classify thermal images for breast with and without
malignant tumors. Frontal and lateral positions were used during this investigation, but no
significant difference of the obtained result was noticed.
It is should be confirmed that the first order parameters do not give the satisfactory
results, and due to some physiological changes of the breast we could observe that these
parameters do not allow to separate patients with and without tumors. That was the main
reason, that we have chosen second order statistical parameters for the further
investigations.

10
As67-healthy breast
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
28 30 32 34 36 38
temperature [oC]
left side
right side


Figure 13: Histograms of healthy breast thermographs


jk33-malignant tumor in left breast
0
0.005
0.01
0.015
0.02
0.025
0.03
0.035
26 28 30 32 34 36 38
temperature [oC]
left side
right side


Figure 14: Histograms of breast thermographs with malignant tumor


Next, second order statistical parameters were taken into account. Most of them
successfully discriminate healthy and malignant tumor cases. The protocol of the
investigation assumes the symmetry analysis in the following way. At first, the square ROI
were chosen for analysis. Then, co-occurrence matrixes were calculated for left and right
breast to evaluate second order statistical parameters for different directions. Only the
neighboring pixels are considered in these investigations (d=1). Finally, the differences of the
values of these parameters for left and right sides were used for the image classification.
Figures below illustrate that the differences of second order parameters for left and right side
are typically greater for pathological cases. Taking two parameters such as difference
variance and variance allows successfully separating almost all healthy and malignant tumor
cases.


11
Diagonal co-ocurence matrix, d=1
0
20
40
60
80
100
120
0 20 40 60 80 100 120
Variance
D
i
f
f
e
r
e
n
c
e

V
a
r
i
a
n
c
e
Pathology
Physiology


Figure 15: Difference variance versus variance obtained from co-occurrence matrix for
horizontal direction


Diagonal co-ocurence matrix, d=1
0
50
100
150
200
250
0 50 100 150 200 250
Variance
D
i
f
f
e
r
e
n
c
e

V
a
r
i
a
n
c
e
Physiology
Pathology


Figure 16: Difference variance versus variance obtained from co-occurrence matrix for
diagonal direction


We need to underline that these pilot investigations are still in the development, and more
then two parameters will be taken for image classification. In fact, this paper does not
consider any classifying algorithm, eg.: neural network.
8. Conclusion
In this paper we present chosen image processing methods suitable for thermal and
visual image processing. The main aim was to use visual image to compute thermal
signatures for precisely defined areas of interest. The histogram modification gives many
possibilities for contrast enhancement and details visualization. As in medicine and biology,
the images are very non-homogenous, the processing using local statistical characteristics of
the image is recommended.
The preliminary results of the feature analysis for thermal images used in breast oncology
have been presented. Thermography as the additional and adjacent method can be very
helpful for early and image classification that helps to recognize tumors. At first, we consider
first and second order statistical parameters that can be successfully used in breast cancer
screening. Although we do not have many pathological cases for investigations yet, the first
results are very promising. While the second order parameters are more sensitive to the
overall structure of the image, they provide better features that can be used in the image
12
classification. Actually, the study is being extended by choosing right second order
parameters for multivariate data classification. Our approach will then include the PCA
analysis to reduce the dimensionality of the problem and by selecting the eigen vectors we
will be able to get the image which represent the tumors more evidently.
References

[1] E.Y.K. Ng, L.N. Ung, F.C. Ng, L.S.J. Sim Statistical Analysis Of Healthy And Malignant
Breast Thermography, Journal of Medical Engineering & Technology, Vol. 25 No 6
(Nov/Dec 2001) p.253-263./
[2] P. Cichy, Texture analysis of digital images doctoral thesis, Technical University of
Lodz, Institute of Electronics, Lodz, 2000, in Polish
[3] I. T. Jolliffe, Principal Component Analysis. New York, Springer-Verlag, 1986.
[4] B. F. J Manly, Multivariate Statistical Method: A Primer. London, Chapman & Hall,
1994.
[5] D. R. Causton, A Biologist's Advanced Mathematics, London, Allen and Unwin, 1987.
[6] A. Materka, M. Strzelecki, R. Lerski, L. Schad, Evaluation of Texture Features of Test
Objects for Magnetic Resonance Imaging, Infotech Oulu Workshop on Texture Analysis
in Machine Vision, June , 1999, Oulu, Finland.
[7] Michael Bennett Breast Cancer Screening Using High-Resolution Digital
Thermography Total Health, Vol. 22 No 6 p.44, 1985.
[8] W. K. Pratt, Digital Image Processing, Willey & Sons, Inc, Ney York, 1991.
[9] B. Wicek, S. Zwolenik, Thermal wave method - limits and potentialities of active
thermography in biology and medicine, 2nd Joint EMBS-BMES Conference, 24th
Annual International Conference of the IEEE Engineering in Medicine and Biology
Society, BMES-EMS 2002, Huston, 23-26 Oct. 2002.
[10] B. Wicek, S. Zwolenik, P. Sawicki, Advanced Multichannel Thermal and Visual
System, III International Workshop Advances in Signal Processing for NDE of
Materials, The American Society for Nondestructive Testing, Inc., Columbus Ohio, 1998,
289-294.
[11] B. Wicek, S. Zwolenik, A. Jung, J. Zuber, Advanced Thermal, Visual and Radiological
Image Processing for Clinical Diagnosis, European Journal of Thermology (formerly
Thermologie Osterreich) 1998, 8, 4, 139-144.