Computer Aided Ultrasound Laboratory

V.Cerny
Institute of Physics, Comenius University, M l y n s k i dolina, 842 15 Bratislava
R.Zajac
D e p a r t m e n t of Urology, Medical Faculty Hospital, Hlbok6 7, 811 05 Bratislava

Abstract

A computer support system works in our sonography laboratory providing essential services like the data-base for medical findings combined with the image data-base. The system
is based on the digitization of the video-signal output of the sonograph. W e have studied
the possibility of using various picture processing methods to further support the diagnosismaking process. W e report here on our experience with three particular methods. So far
the most successful method we have developed was the quasi-tomographical processing of
images. W e take up to 16 scans of the same section plane from different positions of the
ultrasonic probe. The final image is obtained as a suitable average of the matched images.
The method proved to be very helpful in the diagnostic process and is used on an everyday
basis. O n a somewhat more academic level we are studying the use of simple neural net
classifiers to evaluate textural content in the images. The nets are trained on sets of texture patterns and then used to classify the testing samples. W e present here two particular
examples: classifying malignant tissue i n testes and segmentation (sinus vs parenchyma) in
kidney. Our experience with these methods is still limited. However, our conjecture is that
the texture information can be used as a supportive tool in clinical praxis.

1: Introduction
Confronted with the problem of handling the large amount of d a t a and images in our
sonography laboratory we have build a computer support system t o provide essential services like the data-base for medical findings combined with the image data-base. The
system is based on the digitization of the video-signal output of the sonograph. Having
digitized images we, of course, have a possibility t o process the images using standard textbook methods like filtration, histogram changing operations, greylevel transformations etc.
O u r experience with standard methods is basically this: the sonograph images are different
from, say, blurred camera images with respect to performance of standard picture processing techniques. It seems that there is not enough redundant context-free information in the
image for simple methods to work well.
In the method which we call quasi-tomographical processing we increase the amount of
redundant information by taking more scans of the same objects. We take several images
from one section plane, then we match the images and do a weighted sum t o improve the
quality of the resulting image. The quasitomographical processing was found very useful
and is now routinely used in daily praxis. Some more details are presented in Section 2.
'e-mail cerny@fmph.uniba.sk

22

1063-7125197 $10.000 1997

Tenth IEEE Symposium on Computer-Based Medical Systems

23

In addition t o simple local filtering methods we have experimented with higher-level

methods based on textural analysis. Our conjecture is that these methods can be used as
a supportive tool in the diagnostic decision-making process. In this contribution we report
on two experiments in progress, both based on neural network texture analysis. In Section
3 we present our results with texture-based classification of testes tissue and in Section 4
we report on our very preliminary results with a segmentation algorithm.

2: Quasitomographical processing of sono-images

To increase the amount of redundant information with respect t o a simple sonographic
image we take more scans of the sa.me field of interest and process the scans t o get one
resulting image.
In the first step we take a series of up t o 16 images of the same section-plane. The
sonograph works in the real-time mode: image freezing is performed by software. It is
essential that during this process the cutting plane seen by the ultrasonic probe remains
constant. However, the probe is not kept fixed, it changes its position and angle provided
the cutting plane remains invariant. One gets different views: the same objects seen from
different viewpoints corresponding to different positions and orientations of the probe.
An experienced examiner does not find it too difficult to perform such movements with
the probe which keep the cutting plane invariant. He observes the real time image on the
control monitor so he is able to judge the successfulness of the process. Anyhow, in the
off-line processing those cases where the cutting plane is found t o be non-invariant are
automatically rejected.
The second step is done off-line. There are essentially two actions t o be performed.
The first one is transforming the images to achieve maximal matching, the second one is
producing the output image out of series of matched images.
We take one image from the series as the reference one. All the other images from the
series need t o be transformed (rotated and/or translated) in order to achieve the maximal
matching with the reference image. So we arrive at an optimization problem. The task
is to find optimal values of transformation parameters (for each image from the series) for
which the matching between the images is optimal. The quality of matching is measured
by a suitable cost function.
Suppose we have the reference image and what we call TBT-image (To-Be-Transformed
image). Suppose we have the pixel (I, y) of the reference image and we want to know what
pixel (I,y) of the TBT-image corresponds t o it. The relation between (z, y) and (I,y)
is given by the following equations

+ (I - 20) * c o s 4 + (y
y = yh + (y yo) * cos 4 (z

I = I&

IL,

yo) sin 4

- 10)

sin 4

where
yh, 4 are the three parameters of the transformation to be determined. (IO,yo)
are the coordinates of a suitably chosen fixed point in the reference image (the center of the
region of interest). To simplify the presentation we have omitted from the equations the
correction for the aspect ratio. Of course, in actual computation we take this correction
into account.
For any fixed values of the parameters I;, yh, 4 we can assign t o each pixel of the reference image the corresponding pixel of the T B T image. We can compare the greylevel

24

Tenth IEEE Symposium on Computer-Based Medical Systems

values of the corresponding pixels and evaluate the quality of matching. We use a cost
function which sums the absolute values of the greylevel differences between 100 pairs of
corresponding pixels in the central region of the image. To reduce the effect of rounding
errors (discretization effects) we use for matching reasonably smoothed (filtered) images.
However, the reconstruction (after finding the transformation parameters) we do with original (unsmoothed) images.
The optimization itself is fairly straightforward though technically subtle procedure
which we d o not describe here in details. T h e optimization algorithm provides t,he values of the three parameters of the transformation for which the matching between the
reference image and the TBT-image is optimal.
In this way we find the transformation parameters of all the TBT-images from the series.
After the TBT-images are all transformed t o optimal matching, we construct the final image
in the following way. For each pixel of the final image we take the greylevel values from
the corresponding pixels of the transformed images and calculate the (weighted) median.
The weights are proportional t o the quality of matching. In what follows we shall call the
resulting image as QT-image (quasitomographical-image).
There is a general tendency of the QT-image to be more smooth then the original one,
however, the boundaries between the 1a.rge stmctnres are usually more sharp then on the
original image. So the small scale textures are disappearing in favor of large and more
uniform areas: the final image is more "tomograph-like". This fact seems t o support the
idea, that we are getting rid of some artifacts present in the original images while the
structures of objective value are enhanced. An example of quasitomographically processed
image is presented in Figure 1.

Figure 1. The standard and the quasitomographically processed images

We claim t h a t it is highly improbable t o create some new structure on the QT-image as
an artifact of the method. If evidence of structure is found on the QT-image enhanced with
respect t o the original image, we consider it t o be real and therefore of diagnostic value.
On the other hand it is perhaps possible t h a t some real structures which are present on
some of the images from the series might get smeared out on the QT-image. Therefore
one cannot base the diagnostic conclusions on the QT-image alone. The QT-image should
be considered as an additional source of valuable information. The original (unprocessed)

Tenth IEEE Symposium on Computer-Based Medical Systems

25

images should be evaluated as well.

3: Neural-network based tissue classification

The analysis of texture in images[5] differs a lot from many other laboratory methods
used in medicine: the findings cannot be represented by a single number. Basically the
texture has t o be characterized by a feature vector, a relatively large set of numbers, which
is difficult to comprehend. The evaluation of, say, ten parameter vector is certainly too a
complex problem for a human being.
We decided to push further the computer aid and t o perform the evaluation process by
the computer, to get a sort of yes/no result by the machine. It is clear to u s that we are
overshooting with the computer aid in this way. However, we do not suggest to take the
machine decision literally. The machine findings are to be considered as a piece of d a t a
presented in a very compressed form.
We have studied texture samples from a collection of ultrasonic images of testes. We
worked with two subcollections: the first one were images for patients with normal testes,
the second one for patients with tumor. The standard B-mode Sonoline images were digitized to 256 x 256 pixels 256 grey-level images. From the images texture samples were
taken interactively from windows 16 x 16 pixels large. The samples were taken from regions
classified (by the physician) as tumor tissue, and from the regions classified as normal.
Actually, we took more then one texture sample from each image for different depths below
the ultrasound probe, since we learned in our previous studies that the texture features
depend on the distance from the probe. This fact was reported by other authors as well [4].
Therefore we also included the vertical distance h of the texture window from the probe
point to be one element of the texture feature vector.
After experimenting with different texture features we decided t o use the Haralick[2]
contrast parameters Ck,l defined as

where gk,i;;,j are the probabilities to find in the given texture window two pixels separated
by IC columns and I rows and having greylevels i a j .
We decided to use the following 12-component feature vectors t o characterize the texture

These vectors were calculated for each of the sample windows. Since the samples were
classified by the supervisor (physician), we obtained a set of vectors classified as tumor
and as normal.
Our next task was t o build a neural net with 12 real-valued input lines and a single binary
output. The two output states correspond to the classification results tumor/normal.
Our sample statistics is much lower compared to what is usually used in typical neural
network experiments. We expected therefore, that one should consider only very simple
nets with just a few neurons. Therefore we decided t o use the S T E P N E T kind of network[3].
The S T E P N E T network is not a classical neural net since in addition to standard neurons
it contains also explicit logical gates (like and, or). Its main advantage is that it is
adaptively build in the process of training. The S T E P N E T training is an iterative process.

26

Tenth IEEE Symposium on Computer-Based Medical Systems

In the first step it attempts t o separate the classes using just single neuron trained for the
best linear separation on the set of training examples. If the separation after the first step
is not perfect further neurons are added to the network, trying to "subclassify" the d a t a
independently on each side of the first neuron separation hyperplane.
For the training purposes we divided the complete d a t a collection into two subsets: the
training set and the validation set. Both sets contained both classes of samples. The division
of samples into two sets is important for the prevention of overtraining the network. One
trains the net on the training set and then checks the generalization ability on the validation
set.
It appeared that the training is saturated on the level of three neurons[l]. For larger
networks one already observes bad generalization performance. For the optimal net we
observed the following performance on the validation set. Out of set of validation samples
87 percent were correctly classified. About 10 percent of validation "tumor" samples were
miss-classified as "normal".
We stress again that our approach was intentionally extremalistic. In real life nobody
would base the diagnosis on an isolated piece of information like the texture feature vector. Getting no-nonsense results for very crude approach is perhaps suggesting that the
information on texture might be significant for the diagnosis.

4: Neural-network based segmentation algorithm

At present we are in progress of developing another texture-analysis based method. We
evaluate a scheme based on a neural network classifier for segmentation purposes in sonographic images. In particular we study the segmentation in the sonographic images of
kidneys (like parenchyma versus sinus). Segmentation, after all, is a kind of classification
process. The individual pixels are t o be labeled according t o the type of tissue they correspond to. Neural network classifier is in principle far more flexible labeler with respect
t o classical segmentation algorithms and the corresponding algorithms are expected t o be
highly adaptive, able t o learn from examples.
The general procedure is similar t o the method described in the previous section. For
segmentation purposes, however, we take much smaller texture windows, 6 x 6 or even 4 x 4
pixels. Consequently we use a smaller number of features. In a particular experiment we
used just three features extracted from a 4 x 4 window: mean greylevel value, variance of
greylevels and the difference between the highest and the smallest greylevel values within
the texture window.
T h e training and the segmentation is performed in a same image. First a training set
of texture windows is taken as typical examples of the two kinds of textures one wants
to distinguish. A STEPNET type of network is build and trained based on this training
sample. Typically three neurons saturate the training. Then the image is scanned entirely
and each pixel is labeled with one from two labels based on the texture sample taken from
the window centered around the pixel.
A typical result is presented in Figure 2. We are still in the phase of tuning the algorithm.
Our first experience shows, however, that the texture-based segmentation is better then a
segmentation based on the degenerated algorithm based on one dimensional feature vectors
working with mean greylevels only.
We did not use at present any proximity reference between pixels being labeled. We
are currently investigating the possibility t o use this kind of information t o improve the

Tenth IEEE Symposium on Computer-Based Medical Systems

27

performance and t o simplify the learning stage.

Figure 2. The original image of kidney and the its segmentation

5 : Conclusions
The computer support is now essential for the everyday working of our sonography
laboratory. Although the most important thing is perhaps still the usage of the obvious
supportive tools like patient and image databases we have demonstrated that the computer
aided diagnosis is not just a virtual slogan. The quasitomographical method we have
developed is used on everyday bases and we have a record of several cases when this method
was essential for stating the diagnosis correctly. The neural network applications we have
presented in this report are still in a stage of testing. But our experience is encouraging
and we certainly want t o deepen the study of the presented methods.

References
[I] V. Cerny, L. Fekete, and R. Zajac. In V. Skala, editor, Proceedings WSCG95, volume 11.
University of West Bohemia, Plzen, 1995.
[2] R. M. Haralick, K. Hanmugam, and I. Dinstein. IEEE Trans. Systems Man Cybernet.,
SMC-3~610,1973.
[3) S. Knerr, L. Personnaz, and G. Dreyfus. In F. Fogelman and J. Herault, editors, Neurocomputing, Algorithms, Architectures and Applications, NATO AS1 Series. Springer,
1990.
[4] D. T. Morris. Ultrasound in Medicine and Biology, 14387, 1988.
[5] A. Rosenfeld and A. C. Kak. Digital Picture Processing. Academic Press, New York,
1982.