Skin-based Face

Detection
Generic Framework for Face Detection

Skin Morphological Template Face

Detection Processing Matching Selection

Dilation Convolving the image with some template

Filling The average of the image being tested must
be subtracted to eliminate biasing toward
Erosion brighter areas
 Why to detect skin?
Person Detection
Face Detection and Face Tracking
Hand Tracking for
Gesture Recognition
Robotic Control
Other Human Computer Interaction
A filter for adult content on the internet
Other uses in video applications
 Methods of Skin Detection
Pixel-Based Methods
Classify each pixel as skin or non-skin individually,
independently from its neighbors.
Color Based Methods fall in this category
Region Based Methods
Try to take the spatial arrangement of skin pixels
into account during the detection stage to enhance
the methods performance.
Additional knowledge (in terms of texture etc)
are required
 Skin Color based methods - Advantages

Allows fast processing

Robust to geometric variations of the skin patterns
Robust under partial occlusion
Robust to resolution changes
Eliminate the need of cumbersome tracking devices or
artificially places color cues
Experience suggests that human skin has a characteristic
color, which is easily recognized by humans.
 Issues with skin color
Are Skin and Non-skin colors seperable?
Illumination changes over time.
Skin tones vary dramatically within and across individuals.

Different cameras have different output for the identical image.

Movement of objects cause blurring of colours.
Ambient light, shadows change the apparent colour of the image.

Issue 1: What colour space to be used?

Issue 2: How exactly the colour distribution has to be
modelled?
 Issue 1: What colour space to be used?

Increased separability between skin and non skin

classes
Decreased separability among skin tones
Cost of conversion for real time applications
What is the color distribution model used
Keeping the Illumination component 2D
color space vs. 3D color space
Stability of color space (at extreme values)
 Issue 2: How exactly the colour
distribution has to be modelled?
Explicit definition of skin region rule-based classifier
defining the boundaries of skin cluster in some colorspace.

Non parametric Estimate skin color distribution from

the training data without deriving an explicit model of the
skin.
Look up table or Histogram Model
Kernel density, parazan window

Parametric Deriving a parametric model from the

training set
Gaussian Model
Mixture of Gaussians
What colour space to be used?
 Different Color Models
RGB
Normalized RGB
HIS, HSV, HSL
Fleck HSV
TSL
YCrCb
Perceptually uniform colors
CIELAB, CIELUV
Others
YES, YUV, YIQ, CIE-xyz
 RGB Red, Green, Blue
Most common color space used to represent
images.
Was developed with CRT as an additive color
space
[1] Rehg and Jones have used this color space
to study the separability of the color space
 Normalized RGB rg space

2D color space as b component is redundant

b=1gr
Invariant to changes of surface orientation
relatively to the light source
 HSV, HSI, HSL (hue, saturation,
value/intensity/luminance)

High cost of conversion

Based on intuitive values
Invariant to highlight at white light sources
Pixel with large and small intensities are discarded as HS becomes
unstable.
Can be 2D by removing the illumination component
 Y Cr Cb

YCrCb is an encoded nonlinear RGB signal,

commonly used by European television studios
and for image compression work.
Y Luminance component, C Chorminance
 Perceptually uniform colors
skin color is not a physical property of an object,
rather a perceptual phenomenon and therefore a
subjective human concept.
Color representation similar to the color sensitivity
of human vision system should be taken into
consideration.
Complex transformation functions from and to
RGB space, demanding far more computation than
most other colorspaces
Skin Classifier
 Skin detection

skin

Skin pixels have a distinctive range of colors

Corresponds to region(s) in RGB color space
for visualization, only R and G components are shown above
Skin classifier
A pixel X = (R,G,B) is skin if it is in the skin region
But how to find this region?
 Skin detection

Learn the skin region from examples

Manually label pixels in one or more training images as skin or not skin
Plot the training data in RGB space
skin pixels shown in orange, non-skin pixels shown in blue
some skin pixels may be outside the region, non-skin pixels inside. Why?
Skin classifier
Given X = (R,G,B): how to determine if it is skin or not?
 Skin classification techniques

Skin classifier
Given X = (R,G,B): how to determine if it is skin or not?
Nearest neighbor
find labeled pixel closest to X
choose the label for that pixel
Data modeling
fit a model (curve, surface, or volume) to each class
Probabilistic data modeling
fit a probability model to each class
 Probabilistic skin classification

Modeling uncertainty
Each pixel has a probability of being skin or not skin

Skin classifier
Given X = (R,G,B): how to determine if it is skin or not?
Choose interpretation of highest probability
set X to be a skin pixel if and only if

Where do we get and ?

 Learning conditional PDFs

We can calculate P(R | skin) from a set of training images

It is simply a histogram over the pixels in the training images
each bin Ri contains the proportion of skin pixels with color Ri

This doesnt work as well in higher-dimensional spaces.

Approach: fit parametric PDF functions
common choice is rotated Gaussian
center
covariance

orientation, size defined by eigenvecs, eigenvals

 Learning conditional PDFs

We can calculate P(R | skin) from a set of training images

It is simply a histogram over the pixels in the training images
each bin Ri contains the proportion of skin pixels with color Ri
But this isnt quite what we want
Why not? How to determine if a pixel is skin?
We want P(skin | R), not P(R | skin)
How can we get it?
 Bayes rule

In terms of our problem:

what we measure domain knowledge
(likelihood) (prior)

what we want normalization term

(posterior)

The prior: P(skin)

Could use domain knowledge
P(skin) may be larger if we know the image contains a person
for a portrait, P(skin) may be higher for pixels in the center
Could learn the prior from the training set. How?
P(skin) could be the proportion of skin pixels in training set
 General classification
This same procedure applies in more general circumstances
More than two classes
More than one dimension

Example: face detection

Here, X is an image region
dimension = # pixels
each face can be thought
of as a point in a high
dimensional space
H. Schneiderman, T. Kanade. "A Statistical Method for 3D Object
Detection Applied to Faces and Cars". IEEE Conference on
Computer Vision and Pattern Recognition (CVPR 2000) H. Schneiderman and T.Kanade
http://www-2.cs.cmu.edu/afs/cs.cmu.edu/user/hws/www/CVPR00.pdf
 References
Some slides were taken from presentations by;
Nikhil Rasiwasia, Color Space for Skin Detection A Review,
University of Trento, (TN) Italy
Ashutosh Saxena, Object Detection, Cornell University

[1]Michael J Jones & James R Rehg, Statistical Color Models with

Application to Skin Detection
[2]D.Zarit, Comparison of five color models in skin pixel classification
[3]Albiol, optimum color spaces for skin detection
Other papers
[4]Min C. Shin Does colorspace transformation make any difference on
skin detection
[5]Vezhnevets, A survey on Pixel-Based skin color detection techniques
 Next
Illustration of skin detection using Bayes
classifier.
Performance evaluation using ROC curves.