You are on page 1of 6

IEEE International Workshop on Imaging

Systems and Techniques – IST 2008


Chania, Greece, September 10–12, 2008

Automatic Segmentation of the Lung Fields in


Portable Chest Radiographs based on Bézier
Interpolation of Salient Control Points
Dimitris K. Iakovidis George Papamichalis
st
Dept. of Informatics and Computer Technology 1 Dept. of Thoracic Surgery
Technological Educational Institute of Lamia Chest Hospital of Athens “Sotiria”
Lamia, Greece Athens, Greece
dimitris.iakovidis@ieee.org

Abstract—Portable chest radiographs are performed on the most radiographs. Many of them have focused on the segmentation
critically-ill patients, who need highly skilled patient care and the of the lung fields, whereas fewer have focused on the
best diagnostic tools. However, such radiographs are usually of segmentation of the rib cage or other anatomic structures of the
low quality, mainly due to misaligned body positioning during chest [3]. The importance of the segmentation of the lung fields
their acquisition, and subject to a higher misinterpretation rate becomes obvious considering that it is a prerequisite for almost
than the ones obtained with a non-portable x-ray device. This any computational radiographic image analysis task. Lung field
paper presents a pioneering computational approach that copes segmentation approaches that have been proposed include rule-
with the segmentation of the lung fields in portable chest based methodologies [5]-[10], machine learning algorithms
radiographs. The proposed methodology involves detection of
[11]-[13], active shape models [4][14]-[17], and graph cuts[18].
salient points on the anatomic structures around the lung fields
by subsequent application of simple intensity and edge feature The studies [5]-[10] indicate that the segmentation
extraction techniques. The salient points detected are performance of the existing rule-based methodologies usually
interpolated using Bézier curves intuitively approximating the degrades if the quality of the radiograph is poor or the
boundaries of the lung fields. Unlike current methodologies, the positioning of the patient deviates from the standard. Machine
proposed one does not exclude the overlapping region of the learning algorithms and active shape models can be less
heart from the lung fields, where abnormalities can also be
affected by a patient’s positioning; however, they both require
present. The results illustrate the robustness of the proposed
methodology on a set of real portable radiographs of patients
training, which makes them dependent on the subjective choice
with bacterial pulmonary infections. A qualitative comparison of representative training samples. The methodologies reported
with a state of the art approach based on graph cuts validates its in [14],[16],[17] have been mainly evaluated on radiographs of
effectiveness. normal or minimally distorted lungs. Robustness against the
presence of abnormalities that interfere with the interior
Keywords-image segmentation; portable chest radiography; intensities of the lungs has been demonstrated in [4],[15] using
salient points, Bézier curves; lung; infections active shape models. The graph cuts algorithm proposed in [18]
is the most recent of the limited approaches coping with
I. INTRODUCTION unsupervised segmentation of abnormal lung fields. It is based
on morphological operations for obtaining an initial estimate of
Radiography is a valuable tool to chest screening providing the region of the lung boundaries, and dilates the contour
useful information for the prevention and diagnosis of various obtained by graph cuts within a potential contour band. The
diseases [1]. Plain chest radiographs are usually obtained in a evaluation of this algorithm on a limited set of standard
controlled setup where the patients are positioned in a standard radiographs of patients infected with severe acute respiratory
way at the x-ray device. However, in the case of critically ill syndrome (SARS) showed that although it exhibits some
patients, this is not always feasible as they may be in pain or resistance to boundary discontinuities and noise, the detected
disabled. To cope with this problem, portable x-ray devices are boundaries of the lung fields are affected by the edges of the
commonly used to obtain the radiographs from various relative ribs and result in a rough output contour.
distances and angles to the patient who is immobilized in bed -
not necessarily in the preferred position. Portable radiographs To the best of our knowledge, the problem of lung field
are usually of low quality mainly because of misaligned body segmentation in portable chest radiographs has not been
positioning during their acquisition [2]. Consequently, they are previously addressed. In this paper we accept the challenge of
more difficult to read and subject to a higher misinterpretation coping with automatic segmentation of the lung fields in
rate. A methodology that would automatically analyze such portable chest radiographs, and we propose a novel
radiographs would be an asset to the medical community. methodology that is resistant to patient positioning, to
A variety of image processing and analysis methodologies boundary discontinuities, and to the presence of abnormalities
has been proposed for the segmentation of plain chest interfering with the interior intensity of the lungs.

This work was supported in part by the European Commission’s Seventh


Framework Information Society Technologies (IST) Programme, Unit ICT for
Health, project DEBUGIT (no. 217139). 82
978-1-4244-2497-9/08/$20.00 ©2008 IEEE
The proposed methodology involves detection of salient P2
points on the anatomic structures around the lung fields by Internal control points
subsequent application of simple intensity and edge feature
extraction techniques. The salient points detected, are
P1
interpolated using Bézier curves [19] smoothly approximating
the boundaries of the lung fields.
The rest of this paper consists of three sections. Section II Piecewise linear interpolation
presents the proposed lung field segmentation approach. Its
performance is evaluated in Section III, and a summary of P3
conclusions derived from this study is provided in the last Bézier interpolation
section.
P0
II. PROPOSED APPROACH
Figure 1. Bézier vs piecewise linear interpolation of four points.
A. Overview
ive thresholding algorithm identifies the intensities of the
The proposed methodology assumes that: (1) the patient’s
anatomic structures lying in the region of the spine by accumu-
body may not necessarily be aligned with the portable x-ray
accumulating the most frequent intensities of samples acquired
device, since the patient may be immobilized in bed; (2) the
from the detected spine points. Using only the most frequent
spine lies roughly somewhere in the middle of the radiograph,
intensities from each sample, the possibility of discarding
considering that the radiograph displays both the patient’s
intensities belonging to the lung fields is minimized. Moreover,
lungs; (3) the spine and the bones of the ribcage are the densest
all the intensities that are higher than those belonging to the
structures visible in a chest radiograph [20], and consequently
region of the spine are also set to zero, since they are unlikely
they are generally characterized by higher intensities than other
to belong to the lung fields. It is worth noting that the resulting
anatomic structures.
thresholded image will have zero intensities not only between
Based on these assumptions, points belonging to the densest the lung fields, but also in other regions of similar densities,
structures bounding the lung fields are detected. As indicated in including the diaphragms which define the lower boundaries of
[10] the local maxima of the horizontal and vertical image the lung fields.
profiles - i.e. signals produced by averaging image intensities
The inner lung field boundaries are then determined by the
across columns and rows, respectively - can be useful for the
maximum edge points between the spine and the lung fields
detection of such points. However, in the case of portable
(Subsection II.C: Step 5). The edges are obtained via
radiographs the local maxima corresponding to the structures
convolution with directional Sobel edge detectors [21]. This
bounding the lung fields are not located in particular positions
way, unlike current approaches [3]-[13],[15]-[18], the proposed
as in the case of the stationary radiographs used in [10]. To
one does not exclude the overlapping region of the heart from
cope with this problem, the proposed approach involves
the lung fields, where abnormalities can also be present [1]. A
additional steps (Subsection II.C: Steps 2 and 3) for selecting
similar approach is followed for the detection of the lower lung
local maxima of the horizontal profiles corresponding to the
field boundaries (Subsection II.C: Step 6). In both cases the
spine and to the outer left and right boundaries of the lung
resulting points are plenty but include many outliers, most of
fields. Local maxima with intensities deviating from the
which can be discarded by using a k nearest neighbor approach.
average intensities of each boundary are discarded as outliers.
The remaining salient points from the outer, the inner and
Considering that the region of the spine is typically
the lower boundaries of each lung field are smoothly
characterized by the highest intensities in a radiograph, the
interpolated using Bézier curves. The subsequent curves are
inner lung field boundaries will be more or less identifiable by
finally manipulated so that they form continuous closed curves
their edges. These edges will be less prevalent in regions where
segmenting the lung fields.
the lung fields exhibit higher intensities such as the region of
the heart, and regions with abnormalities appearing as bright
shadows next to the inner lung field boundaries. The B. Bézier interpolation
determination of these edges becomes even more difficult if the Given a set of control points P0, P1, …,Pn, a Bézier curve is
airing of the lungs is poor during the acquisition of the estimated according to the equation
radiograph, which is usually the case in portable radiography as
practiced in intensive care units. However, even if the inner n
⎛n⎞
lung field edges are prevalent they could easily get confused B(t ) = ∑ ⎜⎜ ⎟⎟ Pi (1 − t ) n − i t i (1)
with the edges of the ribs within the lung fields. i =0 ⎝ i ⎠

In order to minimize the likelihood of such confusions, the where t ∈ [0,1] .


intensities of the anatomic structures lying in the region of the
spine are set to zero via a novel selective thresholding Bézier curves are intuitively controlled by the detected
algorithm. This algorithm results in an auxiliary image referred salient points, in the sense that the curve is attracted by the
to as thresholded (Subsection II.C: Step 4) which is utilized in internal control points, without necessarily passing through
the subsequent steps of the proposed methodology. The select- them (Fig. 1). This provides several advantages over other

83
interpolation methods that include insensitivity to outliers, and
production of smoother curves.

C. Algorithmic implementation
The salient control points of a radiograph I of N×M pixels,
are detected and interpolated as follows (Fig. 2):
Step 1. Detect local maxima of horizontal profiles.
a) Acquire sh consecutive non-overlapping rectangular (a) (b)
samples of h×M, h<N pixels from the whole image.
b) For each sample calculate its average horizontal
profile.
c) Detect the local maxima of each profile (white points
in Fig.2b).
Step 2. Detect the central, the left and the right columns
(indicated in black in Fig. 2b)
(c) (d)
a) Acquire sv non-overlapping rectangular samples of
N×w, w<M pixels from the whole image.
b) For each sample calculate the average intensity.
c) The central column C is the one with the maximum
average intensity within columns M/2-M/4 and
M/2+M/4.
d) The left column L is the one with the maximum
average intensity within columns 0 and M/2-M/4.
e) The right column R is the one with the maximum (e) (f)
average intensity within columns M/2+M/4 and M-1.
Step 3. Approximate the spine, and the outer left and right
boundaries of the left and the right lung field
respectively.
a) For each of the sh samples
- The points closest to C are considered as spine
points.
- The points closest to the left side of column L+(C- (g) (h)
L)/2 (left white column in Fig. 2b) are considered
to belong to the outer left boundary of the left lung
field.
- The points closest to the right side of column
R+(R-C)/2 (right white column in Fig. 2b) are
considered to belong to the outer right boundary of
the right lung field.
b) For each set of detected points discard outliers. (i) (j)
c) Interpolate the remaining points of each set (Fig. 2c). Figure 2. Steps of the proposed methodology for the segmentation of
a portable chest radiograph.
Step 4. Threshold all the intensities of I corresponding to
anatomic structures lying in the region of the spine. Step 5. Detect edge points of the inner right and left
a) For each spine point acquire a square sample of x2 boundaries of the left and the right lung field
pixels. respectively.
b) Calculate the intensity histogram of each sample. a) Convolve I with the vertical 3×3 Sobel operator
c) Sum up the largest histogram components of all having the negative signs on the left to obtain a map
samples into a single histogram. Evl(I) of vertical left image edges (Fig. 2e).
d) Find the last non-zero component m of the resulting b) Convolve I with the vertical 3×3 Sobel operator
histogram. having the negative signs on the right to obtain a map
e) Set the intensities that correspond to non-zero Evr(I) of vertical right image edges (Fig. 2f).
histogram components, and the intensities that are c) For each row start from the spine point and move left
larger than m, to zero, to obtain a thresholded image to find the point of Evl(I) with the maximum intensity
T(I) (Fig. 2d). for which T(I) has zero intensity.

84
100%
d) For each row start from the spine point and move
right to find the point of Evr(I) with the maximum
95%
intensity for which T(I) has zero intensity.
e) Discard points deviating from the average horizontal 90%
position of its k nearest neighbors as outliers.

Accuracy
f) Interpolate the detected points (Fig 2g). 85%

Step 6. Detect edge points of the lower boundaries of the 80%


lung fields.
a) Convolve I with the horizontal 3×3 Sobel operator to 75%
obtain a map Eh(I) of horizontal image edges (Fig.
2h). 70%
b) For each column start from row M-1 and move 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
upwards to find the point of Eh(I) with the maximum h (pixels)

intensity for which T(I) has zero intensity. (a)


c) Discard points deviating from the average vertical
position of its k nearest neighbors as outliers. 100%

d) Interpolate the detected points (Fig 2i).


95%
Finally, the interpolating Bézier curves estimated in steps
3c, 5f and 6d, are cropped or extended so that they form 90%

Sensitivity
continuous closed curves segmenting the lung fields (Fig.2j).
85%
This algorithm was developed in Java using a custom
image processing and analysis framework providing cross- 80%
platform compatibility.
75%
III. RESULTS
70%
The effectiveness of the proposed methodology was 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
evaluated on a set of twenty four anonymous radiographs h (pixels)
obtained with a portable x-ray device from patients with
pulmonary bacterial infections hospitalized in an intensive care (b)
unit of the Chest Hospital of Athens “Sotiria”. Such infections
100%
produce abnormalities most usually manifested as
consolidations that are visible as bright shadows interfering
95%
with the interior intensities of the lungs [1],[2].
The segmentation performance was assessed in terms of 90%
Specificity

accuracy, sensitivity and specificity, which are defined by the


following equations: 85%

TP + TN (2) 80%
accuracy =
TP + TN + FP + FN
75%
TP (3)
sensitivity =
TP + FN 70%
8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
TN (4) h (pixels)
specificity =
TN + FP
(c)
where TP is the number of true positive pixels, TN is the Figure 3. Results obtained with the proposed methodology for a
number of true negative pixels, FP is the number of false range of values h, in terms of average (a) accuracy, (b) sensitivity,
positive pixels, and FN is the number of false negative pixels. and (c) specificity. The error bars depict the corresponding standard
The ground truth region of the lung fields has been designated deviations.
by an expert.
error bars, the average segmentation performance is not
All radiographs used in the experiments have been digitized affected significantly by the selection of h. Best segmentations
at 8 bits and have been downscaled to fit a 256×256-pixel were obtained for h=9 and h=17. The sensitivity and the
bounding box. The parameters used in the experimental study specificity in the former case reached 95.3% and 94.3%, and in
presented in this paper include w=h, x=32, and k=5. Figure 3 the latter 95.0% and 93.6%. Indicative segmentations are
illustrates the results obtained with the proposed methodology illustrated in Fig. 2j and Fig. 4. In both cases the lung fields
using different values of h. As indicated by the overlapping appear misaligned; whereas the boundaries of the right lung

85
• it achieves smoother delineation of the lung field
boundaries than the state of the art methodology based
on graph cuts [18];
• unlike current methodologies, it does not exclude the
overlapping region of the heart from the lung fields,
where abnormalities can also be present.
Future objectives include enhancement of the proposed
(a) (b) methodology with shape prior information, further
Figure 4 Portable radiograph of a patient with bacterial pulmonary experimentation with larger datasets and comparisons with
infection. (a) Original. (b) Segmented using the proposed other state of the art methodologies. The perspectives of this
methodology. research extend to the development of a multimodal data
mining system for adverse events detection, which will be
capable of co-evaluating radiographic findings of patients with
bacterial infections [22].

REFERENCES
[1] N.L. Müller, T. Franquet, K.S. Lee, C. Isabela, and S. Silva, Imaging of
Pulmonary Infections, Lippincott Williams & Wilkins, 2006
[2] F.S. Bongard, and D.Y. Sue, Current Critical Care Diagnosis &
Treatment, 2nd ed., McGraw-Hill/Appleton & Lange, 2002
[3] B.V. Ginneken, B.T.H. Romeny, and M.A. Viergever, “Computer-Aided
Diagnosis in Chest Radiography: A Survey,” IEEE Trans. Medical
(a) (b)
Imaging, vol. 20, no. 12, pp. 1228-1241, Dec. 2001.
Figure 5. Segmentation of radiographs obtained from patients [4] B.V. Ginneken, S. Katsuragawa, B.T.H. Romeny, K. Doi, and M.A.
infected with SARS, using the proposed methodology. The original Viergever, “Automatic Detection of Abnormalities in Chest Radiographs
images have been obtained from [18], where segmentation results Using Local Texture Analysis,” IEEE Trans. Medical Imaging, vol. 21,
with various methodologies are also provided. no. 2, pp. 139-149, Feb. 2002.
[5] P.H. Meyers, H.C. Nice, C.M. Becker, N.J. Nettleton, J.W. Sweeney,
and G.R. Meckstroht, “Automated Computer Analysis of Radiolographic
field in Fig. 4 are hardly distinguishable even by an expert. As Images,” Radiology, vol. 13, pp. 1029-1034, 1964.
noted in the introduction, current lung field segmentation [6] J. Duryea, and J. Boone, “A Fully Automatic Algorithm for the
Segmentation of Lung Fields in Digital Chest Radiographic Images,”
methodologies do not cope explicitly with the segmentation of Medical Physics, vol. 22, no. 2, pp. 183-191, 1995.
portable chest radiographs. However, the state of the art [7] H. Luo, R. Gaborski, R. Acharya, “Automatic Segmentation of Lung
methodology presented in [18], which has been proposed for Regions in Chest Radiographs: A Model Guided Approach,” in Proc.
lung field segmentation in standard radiographs, could be Int. Conf. Image Processing (ICIP 2000), vol. 2, 2000, pp. 483-486.
considered as the most relevant methodology to compare with, [8] B.V. Ginneken, and B.T.H. Romeny, “Automatic Segmentation of Lung
since it is also unsupervised and it copes with infected lung Fields in Chest Radiographs,” Medical Physics, vol. 27, no. 10, pp.
2445-2455, 2000.
fields. The results from the application of the proposed [9] I. Iglesias, P.G. Tahoces, M. Souto, A. Martinez de Alegria, M.J. Lado,
methodology on the images presented in [18] are illustrated in and J.J. Vidal, “Lung Segmentation on Postero-anterior Digital Chest
Fig. 5. The obtained segmentations are less noisy than the ones Radiographs Using Active Contours,” Fred A. et al. (Eds.): SSPR&SPR
presented in [18], illustrate higher sensitivity, whereas the 2004, Lecture Notes in Computer Science, vol. 3138, pp. 538-546, 2004.
region of the heart overlapping the lung fields has not been [10] X.-W. Xu, and K. Doi, “Image Feature Analysis for Computer Aided
Diagnosis: Accurate Determination of the Ribcage Boundary in Chest
excluded, enabling the detection of possible abnormalities even Radiographs,” Medical Physics, vol. 22, no. 5, pp. 617-626, 1995.
behind the heart. [11] G. Coppini, S. Diciotti, M. Falchini, N. Villari, and G. Valli, “Neural
Networks for Computer-Aided Diagnosis: Detection of Lung Nodules in
Chest Radiograms,” IEEE Trans. on Information Technology in
IV. CONCLUSIONS Biomedicine, vol. 7, no. 4, pp. 344-357, Apr. 2003.
We presented a novel methodology for automatic [12] N. Vittitoe, R. Vargas-Voracek, and C. Floyd Jr, “Markov Random Field
segmentation of portable chest radiographs. It is capable of Modeling in Posteroanterior Chest Radiograph Segmentation,” Medical
Physics, vol 26, no. 8, pp. 1670-1677, 1999.
detecting salient points which are intuitively interpolated via [13] B.V. Ginneken, M.B. Stegmann, M. Loog, “Segmentation of Anatomical
Bézier curves. Structures in Chest Radiographs using Supervised Methods: A
The following major conclusions can be derived from the Comparative Study on a Public Database,” Medical Image Analysis, vol.
10, pp. 19-40, 2006.
experimental evaluation of the proposed methodology: [14] T. Cootes, C. Taylor, D. Cooper, and J. Graham, “Active Shape Models
• it produces accurate segmentation of portable - Their Training and Application,” Computer Vision and Image
radiographs regardless of the patients’ positioning; Understanding, vol 61, no. 1, pp. 38-59, 1995.
[15] X. Xie, X. Li, S. Wan, and Y. Gong, “Mining X-Ray Images of SARS
• it generally remains unaffected by the presence of Patients,” in G.J. Williams and S.J. Simoff (Eds.) Data Mining, Lecture
Notes in Artificial Intelligence, vol. 3755, pp. 282-294, 2006.
abnormalities originating from bacterial pulmonary [16] J. Liang, T. McInerney, and D. Terzopoulos, “United Snakes. Medical
infections; Image Analysis, vol. 10, pp. 215-233, 2006.

86
[17] T. Yu, J. Luo, N. Ahuja, “Shape Regularized Active Contour using Geometric Modelling.” San Francisco, CA: Morgan Kaufmann, pp. 211-
Iterative Global Search and Local Optimization,” in Proc. Computer 245, 1998.
Vision and Pattern Recognition Conference (CVPR 2005), vol.2, pp. [20] Novelline, R.A. (1997) Squires’s Fundamentals of Radiology.
655- 662, 2005. Cambridge: Harvard University Press.
[18] S. Chen, L. Cao, J. Liu, X. Tang, “Automatic Segmentation of Lung [21] R.C. Gonzalez, and R.E. Woods, Digital Image Processing, 3rd ed.,
Fields from Radiographic Images of SARS Patients Using a New Graph Prentice Hall, Upper Saddle River, NJ, 2008.
Cuts Algorithm,” in Proc. Int. Conf. Pattern Recognition (ICPR 2006), [22] C. Lovis, D. Colaert, V.N. Stroetmann, “DebugIT for Patient Safety -
vol. 1, pp. 271-274, 2006. Improving the Treatment with Antibiotics through Multimedia Data
[19] R.H. Bartels, J.C. Beatty, and B.A. Barsky, Bézier Curves. Ch. 10 in Mining of Heterogeneous Clinical Data,” Stud Health Technol. Inform.,
“An Introduction to Splines for Use in Computer Graphics and vol. 136, 641-646, 2008.

87