Professional Documents
Culture Documents
I. Introduction
Recently, speech recognition with the aid of visual information, sometimes called lip-reading, has proved to be great
interest to many researchers[16] . In the category of biometrics, lip-reading is not only applied to help the deaf or people
with hearing disability communicate with other people, or enable an intelligence agent to obtain information under noisy
environment conditions[1] , but also widely used in the eld of
personal identity recognition.
Robust and accurate lip segmentation, as the rst step
of most lip-reading systems, is of key importance for the accuracy and eciency of the whole system. Many dierent
approaches of lip segmentation have been proposed in recent
years. Generally, there are two distinctive trends in lip segmentation: the rst one, based on classic image and region
segmentation techniques[713] ; and the second one, based on
lip contour estimation and shape template tting[1420] . For
classic region-based approaches, the goal is to detect every single pixel in lip image that belongs to lip region. This kind of
approaches usually assumes that lip and skin pixels have different color features. Sometimes, a fast detection of the lip
region is obtained by using this approach; however, the result
is not accurate for lip edge detection.
This paper focuses on the contour-based approach due to
the inadequacy of classic region-based approach. In this kind
of approaches, the goal is to nd a set F = {f1 , f2 , , fn } of
parameterized functions fn , also known as features or observations of estimation, which is a preferred way to represent lip
information normally invariant to translation, rotation, scaling and illumination. Then estimation of lip contour can be
made by using F . To obtain an accurate lip contour estimation is unquestionably a dicult job. As is known to all, the
optimization of estimation that performs well for a particular
application in this case, lip contour estimation depends upon
many factors. The primary concern is the selection of a good
model, a suitable estimator and proper features. It should be
complex enough to describe principal features of the lip, but
at the same time simple enough to allow an estimator that is
optimal to be easily implemented. Consequently, this paper
tries to gure out the best model, best estimator and best features for lip estimation respectively in order to optimize the
estimation of lip contour.
The paper hereunder is presented as follows: The choice
of lip model is presented in Section II and details of the optimization of lip contour estimator described in Section III.
Section IV outlines how optimal features for estimation are
chosen, including the length of snake of feature points and
distance between these feature points. The experimental result obtained by our estimator is shown in Section V. Finally,
Manuscript Received Aug. 2013; Accepted Sept. 2013. This work is supported by the National Natural Science Foundation of China
(No.61271319, No.60702043, No.61071152).
342
Section VI draws the conclusion.
2014
For comparative experiment, several lip images are arbitrarily chosen from AR face database[23] . Half of them are
female lips, and others male lips. According to the result of
comparative experiment shown above, the model with quartic works obviously worse than that with parabola or cubic.
The reason might be that feature points far from the dip point
are not close to the lip contour and the quartic attempts to t
these bad points. Thus we can see that the curve of higher order is more deformable, which makes models with higher-order
curves fail to correct errors of individual feature points. Hence,
high-order curves are not suitable for lip estimation. For models with parabola and cubic, the result is quite similar. Therefore, the model consisting of three independent parabolas is
chosen so as to reduce computation complexity and improve
the eciency of estimation.
In brief, the proposed lip model can be described as follows:
The function for lower lip contour starting from the
left lip corner C1(xcornerL , ycornerL ) to the right lip corner
C2(xcornerR , ycornerR ):
y a1 x2 + b1 x + c1 = 0
(1)
where
A=
x2i
x2i+1
..
.
x2i+n
xi
xi+1
..
.
xi+n
1
1
..
.
1
343
x2i
yi
xi
1
x2
y
a
m
1
x
i+1
i+1
i+1
A=
..
..
..
, u = bm , v = ..
.
.
.
.
cm
x2i+n xi+n 1
yi+n
By simple derivation, we have
u = (AT W A)1 AT W v
(7)
(4)
yi
am
i+1
.
,u =
,
v
=
b
m
.
cm
yi+n
(5)
where
W = diag(W1 , W2 , WN )
(6)
344
2014
Fig. 6. Comparison of the results of models with dierent distances (d =2, 3, 4, 5, 7, 10) between feature points
345
V. Experimental Results
According to the study outlined above, the complete optimized lip contour estimation method is formed. The chosen lip
model consists of three parabolas. Weighted least square estimator is chosen as our estimator. Two important parameters
that determine the number of features or observations of estimation are dened as follows. The length of snake of feature
points should be 3/4 width of lip and the distance between
feature points is 4 pixels.
The optimized estimation method was rstly tested on AR
face database [23] . Each lip image was acquired from an image
in the rst ve parts of AR face database (consisting of 500
lip images).
Features for estimation are obtained using the algorithm
proposed in Ref.[4]. It is noted that lip contours can be estimated accurately using our optimized estimation method in
most cases, with dierent shapes of lips and even with presence of facial hair and with shadow. The experimental results
for the optimized estimation method are shown as Fig.9.
In some cases, the optimized lip contour estimation
method does not improve some poor results before optimization (Fig.10). The main reason is that the optimized algorithm
VI. Conclusion
Through optimization of lip model, estimator and parameter of features, the optimized lip contour estimation method
proposed in this paper has noticeably improved the result of estimation. The optimized lip model consisting of three parabolas is employed to describe dierent lip shape, which strengthens the robustness of algorithm. The weighted least square
estimator makes our algorithm perform with high accuracy.
In addition, the optimized parameters for features improve
346
2014
347