Face Detection and Neural Networks

http://www.math.umn.edu/~wittman/faces/main.html Todd Wittman Math 8600: Image Analysis Prof. Jianhong Shen Fall 200 ! S"ring 2002 These are Matlab m-files that describe a neural network which will (hopefully) identify if an image contains a human face. The output value is a number which represents the probability that the image is a face: 1 for face ! for not a face. "ou need to download all the re#uired m-files for the programs to run.

Documentation
$n the fall semester of %!!1 $ tried to develop an artificial neural network that could detect fleshtones. $ tried working in the &'( and ")* color spaces getting slightly better results in the ")* space.

M* +ord &eport: ,ace -etection . /eural /etworks M* 0owerpoint 0resentation: ,ace -etection . /eural /etworks

$n the following spring semester of %!!% $ tried to refine the network by using more training data. This network seemed to do a better 1ob detecting flesh tones. $ also tried unsuccessfully to link the network to a segmentation program. 2nfortunately it is very difficult to segment natural images.

M* +ord &eport: ,ace -etection . *egmentation M* 0owerpoint 0resentation: ,ace -etection . *egmentation

RGB Histogram pproach

This neural network takes the three & ' ( histograms (appended as one vector) as input. *elect any color 1peg image. Matlab will not read gif files. $f the image contains a face it should be a full frontal view of the head (mugshot). To run the network type in Matlab the command: y 3 rgb4forward(5your4image.1pg5) The returned value y is the probability that the image represents a face. "ou will need all

the files below to run the program. The result should include the original image the color histogram and the y value. rgb4forward.m: ,orward neural network for &'( approach. &e#uired files: forward.m: 6btains output from general neural network. image%rgbhist.m: 7onverts image to vector of & ' ( histograms. resi8e4matri9.m: $nterpolates pi9el values from image. sigmoid.m: Threshold function for firing of neurons. rgb4weights.mat: 7ontains the weights calculated from the training set.

!"# Histogram pproach

This neural network uses the histograms under the ")* scheme rather than the conventional &'( scheme. 0erformance seems to have increased slightly under this scheme. *elect any color 1peg image. To find the output type in Matlab: y3yes4forward(5your4image.1pg5) The returned y value should be : !.; for a human face < !.; if not a face. The plot should look include the original image the color histogram and the y value. yes4forward.m: ,orward neural network for ")* approach. &e#uired files: forward.m: 6btains output from general neural network. image%yeshist.m: 7onverts image to vector of " ) * histograms. resi8e4matri9.m: $nterpolates pi9el values from image. sigmoid.m: Threshold function for firing of neurons. yes4weights.mat: 7ontains the weights calculated from the training set.

$raining
The following m-files were used to train the neural network. The =evenberg-Mar#uad code and the related files eone.m etot.m were written by -r. ,adil *antosa. The trainall routine takes about ; minutes to run but it only runs 1! iterations at a time. To start a new set of weights set the toggle to ! initially. 6therwise the algorithm will start with the weights contained in the .mat file. =evenberg-Mar#uad 7ode levmar.m: The driver for the training process. etot.m: 7omputes gradient for overall system of images. eone.m: 7omputes gradient for one image. sigmoid.m: Threshold function for firing of neurons. &'( >istogram ?pproach rgb4trainall.m: 7omputes weights for %! images in training set. rgb4weights.mat: 7ontains weights calculated after 1!! iterations.

image%rgbhist.m: 7onverts image to vector of & ' ( histograms. resi8e4matri9.m: $nterpolates pi9el values from image. ")* >istogram ?pproach yes4trainall.m: 7omputes weights for %! images in training set. yes4weights.mat: 7ontains weights calculated after 1!! iterations. image%yeshist.m: 7onverts image to vector of " ) * histograms. resi8e4matri9.m: $nterpolates pi9el values from image.

$raining %mages
$n ,all %!!1 $ uesed a small %! image training set to generate the weights in the .mat files. ?ll faces were fed in with output y31 while non-faces had output y3!. The 1! faces were chosen to represent different age groups skin tones and genders. The non-faces were random ob1ects (toasters landscapes cats etc.). $n *pring %!!% $ used a larger training set of 1!! faces. The faces were chosen by race and gender to match proportionately to the %!!! 2* 7ensus statistics. ? picture of all the faces used is given below. The non-faces were again some random scenes plus some randomly generated @gibberish@ color matrices.

?ll Training ,aces

Results
$n general the ")* approach seems to outperform the &'( approach on the test images. >owever the ")* approach fails to classify A of the test images which is rather disturbing. *pecificaly ")* mistakenly classifies the images carter.1pg (i3;) hand.1pg (i31;) and glove.1pg (i31B). This can be seen in the graphs below which show the outputs on the training sets for both methods. The first 1! images (i31:1!) are faces so the output y should be : !.;. The last 1! images (i311:%!) are non-faces so the output should be < !.;. RGB $raining

The variations between the two methods are due to the differences between the &'( and ")* histograms. The ")* histograms tend to show more variation while the &'( histograms tend to cluster and look very similar. The picture below shows the &'( images and histograms compared to the ")* versions below.

#ittman$math.%mn.ed%