Eigenfaces face recognition (MATLAB) #deadbeef

Page 1

Skip to content Follow: RSS Twitter #deadbeef the rants of a technocrat Home Code Computer science Code snippets Data mining Graph theory Image processing Language processing Machine learning Interesting reading In the news Random thoughts Languages C# JavaScript MATLAB Perl Python Software Coding tools Tags face recognition, matlab, pca

Eigenfaces face recognition (MATLAB)

2 December 2010 10 Votes Eigenfaces is a well studied method of face recognition based on principal component analysis (PCA), popularised by the seminal work of Turk & Pentland. Although the approach has now largely been superseded, it is still often used as a benchmark to compare the performance of other algorithms against, and serves as a good introduction to subspace-based approaches to face recognition. In this post, Ill provide a very simple implementation of eigenfaces face recognition using MATLAB. PCA is a method of transforming a number of correlated variables into a smaller number of uncorrelated variables. Similar to how Fourier analysis is used to decompose a signal into a set of additive orthogonal sinusoids of varying frequencies, PCA decomposes a signal (or image) into a set of additive orthogonal basis vectors or eigenvectors . The main difference is that, while Fourier analysis uses a fixed set of basis functions , the PCA basis vectors are learnt from the data set via unsupervised training. PCA can be applied to the task of face recognition by converting the pixels of an image into a number of eigenface feature vectors, which can then be compared to measure the similarity of two face images. Note: This code requires the Statistics Toolbox . If you dont have this, you could take a look at this excellent article by Matthew Dailey, which I discovered while writing this post. He implements the PCA functions manually, so his code doesnt require any toolboxes.

Loading the images

The first step is to load the training images. You can obtain faces from a variety of publicly available face databases. In these examples, I have used a cropped version of the Caltech 1999 face database. The main requirements are that the faces images must be: Greyscale images with a consistent resolution. If using colour images, convert them to greyscale first with rgb2gray. I used a resolution of 64 48 pixels . Cropped to only show the face . If the images include background, the face recognition will not work properly, as the background will be incorporated into the classifier. I also usually try to avoid hair, since a persons hair style can change significantly (or they could wear a hat).

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef

Page 2

Aligned based on facial features. Because PCA is translation variant, the faces must be frontal and well aligned on facial features such as the eyes, nose and mouth . Most face databases have ground truth available so you dont need to label these features by hand. The Image Processing Toolbox provides some handy functions for image registration. Each image is converted into a column vector and then the images are loaded into a matrix of size n m , where n is the number of pixels in each image and m is the total number of images. The following code reads in all of the PNG images from the directory specified by input_dir and scales all of the images to the size specified by image_dims :
01 02 03 04 05 06 07 08 09 10 11 12 13 14 input _dir = '/path /to/my /images'; image _dims = [48 , 64]; filenames = dir (fullfile(input _dir, '* .png')); num _images = numel (filenames); images = []; for n = 1:num _images filename = fullfile(input_dir , filenames(n).name); img = imread(filename ); if n == 1 images = zeros(prod (image_dims), num _images); end images(:, n) = img(:); end

Training
Training the face detector requires the following steps (compare to the steps to perform PCA): 1. 2. 3. 4. 5. 6. Calculate the mean of the input face images Subtract the mean from the input images to obtain the mean-shifted images Calculate the eigenvectors and eigenvalues of the mean-shifted images Order the eigenvectors by their corresponding eigenvalues , in decreasing order Retain only the eigenvectors with the largest eigenvalues (the principal components) Project the mean-shifted images into the eigenspace using the retained eigenvectors

The code is shown below:

01 02 03 04 05 06 07 08 09 10 11 12 13 % steps 1 and 2: find the mean image and the mean -shifted input images mean _face = mean (images, 2); shifted _images = images - repmat(mean_face , 1, num_images); % steps 3 and 4: calculate the ordered eigenvectors and eigenvalues [evectors , score, evalues] = princomp(images'); % step 5: only retain the top 'num_eigenfaces' eigenvectors (i.e. the principal components) num _eigenfaces = 20 ; evectors = evectors(:, 1:num_eigenfaces); % step 6: project the images into the subspace to generate the feature vectors features = evectors' * shifted_images;

Steps 1 and 2 allow us to obtain zero-mean face images. Calculating the eigenvectors and eigenvalues in steps 3 and 4 can be achieved using the princomp function. This function also takes care of mean-shifting the input , so you do not need to perform this manually before calling the function. However, I have still performed the mean-shifting in steps 1 and 2 since it is required for step 6, and the eigenvalues are still calculated as they will be used later to investigate the eigenvectors. The output from step 4 is a matrix of eigenvectors. Since the princomp function already sorts the eigenvectors by their eigenvalues , step 5 is accomplished simply by truncating the number of columns in the eigenvector matrix. Here we will truncate it to 20 principal components, which is set by the variable num_eigenfaces; this number was selected somewhat arbitrarily, but I will show you later how you can perform some analysis to make a more educated choice for this value. Step 6 is achieved by projecting the mean-shifted input images into the subspace defined by our truncated set of eigenvectors . For each input image, this projection will generate a feature vector of num_eigenfaces elements.

Classification
Once the face images have been projected into the eigenspace , the similarity between any pair of face images can be calculated by finding the Euclidean between their corresponding feature vectors and ; the smaller the distance between the feature vectors, the more similar the distance faces . We can define a simple similarity score based on the inverse Euclidean distance:

To perform face recognition , the similarity score is calculated between an input face image and each of the training images. The matched face is the one with the highest similarity, and the magnitude of the similarity score indicates the confidence of the match (with a unit value indicating an exact match ).

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef

Page 3

Given an input image input_image with the same dimensions image_dims as your training images, the following code will calculate the similarity score to each training image and display the best match :
01 02 03 04 05 06 07 08 09 10 % calculate the similarity of the input to each training image feature _vec = evectors' * (input _image(:) - mean _face); similarity_score = arrayfun(@ (n) 1 / (1 + norm(features(:,n) - feature_vec )), 1:num _images); % find the image with the highest similarity [match _score, match _ix] = max (similarity_score); % display the result figure, imshow([input _image reshape(images(:,match_ix ), image_dims)]); title (sprintf('matches %s, score %f', filenames(match_ix ).name, match_score ));

Below is an example of a true positive match that was found on my training set with a score of 0.4425:

To detect cases where no matching face exists in the training set, you can set a minimum threshold for the similarity score and ignore any matches below this score.

Further analysis
It can be useful to take a look at the eigenvectors or eigenfaces that are generated during training:
1 2 3 4 5 6 7 % display the eigenvectors figure; for n = 1:num _eigenfaces subplot (2, ceil(num _eigenfaces/2), n); evector = reshape(evectors(:,n), image_dims ); imshow(evector); end

Above are the 20 eigenfaces that my training set generated. The subspace projection we performed in the final step of training generated a feature vector of 20 coefficients for each image. The feature vectors represent each image as a linear combination of the eigenfaces defined by the coefficients in the feature vector; if we multiply each eigenface by its corresponding coefficient and then sum these weighted eigenfaces together, we can roughly reconstruct the input image. The feature vectors can be thought of as a type of compressed representation of the input images. Notice that the different eigenfaces shown above seem to accentuate different features of the face. Some focus more on the eyes, others on the nose or mouth, and some a combination of them. If we generated more eigenfaces, they would slowly begin to accentuate noise and high frequency features. I mentioned earlier that our choice of 20 principal components was somewhat arbitrary. Increasing this number would mean that we would retain a larger set of eigenvectors that capture more of the variance within the data set. We can make a more informed choice for this number by examining how much variability each eigenvector accounts for. This variability is given by the eigenvalues . The plot below shows the cumulative eigenvalues for the first 30 principal components:
1 2 3 4 5 % display the eigenvalues normalised_evalues = evalues / sum(evalues ); figure, plot(cumsum(normalised_evalues)); xlabel('No. of eigenvectors '), ylabel('Variance accounted for '); xlim ([1 30]), ylim ([0 1]), grid on ;

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef

Page 4

We can see that the first eigenvector accounts for 50% of the variance in the data set, while the first 20 eigenvectors together account for just over 85%, and the first 30 eigenvectors for 90%. Increasing the number of eigenvectors generally increases recognition accuracy but also increases computational cost. Note, however, that using too many principal components does not necessarily always lead to higher accuracy, since we eventually reach a point of diminishing returns where the low-eigenvalue components begin to capture unwanted within-class scatter. The ideal number of eigenvectors to retain will depend on the application and the data set, but in general a size that captures around 90% of the variance is usually a reasonable trade-off.

Closing remarks
The eigenfaces approach is now largely superceded in the face recognition literature. However, it serves as a good introduction to the many other similar subspace-base face recognition algorithms. Usually these algorithms differ in the objective function that is used to select the subspace projection . Some subspace-based techniques that are quite popular include: Independent component analysis (ICA) selects subspace projections that maximise the statistical independence of the dimensions Fisher s linear discriminant (FLD) - selects subspace projections that maximise the ratio of between-class to within-class scatter. Non-negative matrix factorisation (NMF) selects subspace projections that generate non-negative basis vectors

Share this:

Like this:

Be the first to like this post .

From Image processing, Machine learning, MATLAB 37 Comments

1.

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef Ayesha Hakim permalink In step: shifted _images = images repmat (mean_face, 1, num_images); it gives the error that matrix dimension must agree while subtracting. What may be the problem? Reply

Page 5

alister permalink Its hard to say it could be the images were loaded incorrectly into the matrix, the mean face vector was calculated along the wrong dimension, the number of input images was miscounted, etc. When you run it, what are the dimensions of images and mean_face? And what is the value of num_images and what dimensions in pixels are the images you are using? Reply 2. Ayesha Hakim permalink Thanks, I solved the problem. I am looking for a tutorial for Fishers linear discriminant (FLD) and how to implement it with PCA. Your blog helps me a lot , can you please also publish similar blog for fisherfaces as well? Reply

alister permalink Glad it helped you. I plan to write an article on FLD and NMF in the near future keep an eye out. Reply 3. nilufar permalink i have the same problem in line shifted _images = images repmat (mean_face, 1, num_images); can you give some help , Reply

alister permalink I have updated the code, on the previous line it was calculating mean_face along the wrong dimension. It should work now . Reply 4. Naufal permalink Its good tutorial but Im stuck. It gives error ??? Subscripted assignment dimension mismatch. Error in ==> eig at 13 images(:, n) = img(:); May I know why? Is it OK I put input _dir = C:\Program Files\MATLAB\R2009b\work \s 1; Reply

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef alister permalink

Page 6

The most likely cause is that your input images have inconsistent dimensions or you are not specifying the dimensions correctly. Check that: 1. all of your input images are the same size and that they are greyscale images (use rgb2gray after loading if they are colour images) 2. you are correctly setting image_dims to specify the dimensions of your input images in the format [height, width] To answer your other question, using Windows paths with spaces and backslashes is fine in MATLAB. Reply 5. fafa permalink i have a problem in line evectors = evectors(:, 1:num_eigenfaces); can you give me some help Reply

alister permalink You ll need to explain the problem youre having. Reply 6. weasels permalink Hi, I have a doubt: do we have to take all the training images at once to generate the eigen vectors(say e) or do we take the 1st individuals set of images and find eigen vectors(say e1) then find the 2nd individuals sets eigen vectors(say e2) and so on and then include all the eigen vectors into one eigen vector variable ( say e = [e1;e2;e3...] ) because when i try to get first 20 eigen vectors, they are not the same as yours, instead i am getting the eigen faces as a collection of all and is not giving 20 unique faces as yourshope u understood my problem (cook1234567@gmail.com ). i put all 449 images in 1 folder and am using it as a training database input _dir= c:\myimages\ conatain all 449 caltech images input _image = c:\image_0450.jpg; now my output is not either of image_0430 to image_0449 rather it mathes image_0278 Reply

alister permalink You should be training a single PCA classifier on all of your face images, not one per individual person. Did you test on more than one face? Did it match any faces correctly? Reply 7. weasels permalink Hi Alister , Yes, in fact i have tried using single pca classifier considering the whole database at once(one folder containing 20 face images each of 20 individuals).I could only get a 28% success. I tried with other databases like clouds ,leaves,MPEG 7 CE shapes images database but not much difference . So i tried improving it by transforming the images to wavelets and then subject to PCA and it worked well.Now i can get a proper classification with a 70% to 84% success depending upon the database . Thanks for your blog, it helped me a lot . Reply

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef alister permalink

Page 7

Possibly it could be related to the data set. Its just a guess but because eigenfaces use absolute pixel intensities as their features, they are highly susceptible to lighting variations, whereas differential features such as wavelets are not. Do you have significant lighting variations in your face images? If so, you could try normalising the images first using a technique such as histogram equalisation. Anyway, glad to hear you got it working with wavelets anyway. Reply 8. jyotsna permalink if i want to use these eigen vectors as a input to neural network how cn i use it.cn u plz help me Reply

alister permalink Just use feature_vec as the input to your neural network. I used a simple classifier based on Euclidean distance, but neural networks are just another form of classifier and work basically the same way. Reply 9. irlmaks permalink Hi. I have used pics with background. I tried to do everything you described in the way it is. For the input image i have used one of the test images. But the result is rather horrible which gives out some arbitrary image number with a very low similarity score of about 0.0002. Also when displaying the result it is not the image corresponding to the detected number , rather a black and white image with very few black image pixels . Can you point out the possible mistake that i am making? Note: I am not decreasing the number of eigen vectors . As a precautionary measure to test , i have used all of them. I am posting the code which i have compiled %% Loading the Images clear all input _dir = path of the training image files; image_dims = [48 64]; filenames = dir(fullfile (input _dir, *.jpg)); num_images = numel(filenames ); images = zeros(prod(image_dims),num_images); for n = 1:num_images filename = fullfile (input_dir, filenames(n).name); img = imread(filename); img = rgb2gray(img); img = imresize(img ,image_dims); images(:, n) = img(:); end %% Training % steps 1 and 2: find the mean image and the mean-shifted input images mean_face = mean(images, 2); shifted _images = images repmat (mean_face, 1, num_images); % steps 3 and 4: calculate the ordered eigenvectors and eigenvalues [evectors, score, evalues] = princomp(images); % step 5: only retain the top num_eigenfaces eigenvectors (i.e. the principal components) %num_eigenfaces = 1500; %evectors = evectors(:, 1:num_eigenfaces);

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef % step 6: project the images into the subspace to generate the feature vectors features = evectors*shifted _images; % calculate the similarity of the input to each training image input _image = imread (input.jpg); input _image = rgb2gray(input _image); input _image = imresize (input_image,image_dims); input _image = im 2double (input_image); feature_vec = evectors * (input_image(:) mean_face); similarity_score = arrayfun(@(n) 1 / (1 + norm(features(:,n) feature _vec )), 1:num_images); % find the image with the highest similarity [match_score, match_ix] = min(similarity_score); % display the result figure , imshow([input _image reshape (images(:,match_ix), image_dims)]); title(sprintf(matches %s , score %f, filenames(match_ix).name, match _score)); Reply

Page 8

rahul satija permalink hiii i m facing the same problem.If u have got the solution of above problem then plzz share.That will be a great help . Reply 10. Vanush Vee permalink I have a question . If I associate a parameter with each image in the training set, would I be able to use this technique for regression rather than classification with an unknown test set? Reply

alister permalink What parameter do you have in mind? It may be possible if the parameter has a direct relationship to the face image (e.g. the subjects age ). Its effectiveness would depend entirely on what the parameter represents , what its relationship is to the principal components/facial features and what type of regression you are using (simple linear regression vs. more complex non-linear methods like neural networks). Reply

Vanush Vee permalink I am not using it for facial recognition. I would like to adapt this technique for another application . I have a bunch of images of galaxies, grouped into a test and training set. Each galaxy as a 2-tuple associated with it , which describes its ellipticity (e1, e2). Using the training images, I would like to predict the ellipticity (e1, e2) of the test set, for which it is unknown . To reduce dimension, i would use PCA to get a feature vector using the same method described above . However, I cannot find any literature about using feature vectors for regression. Im very much new to Machine learning.

11. hgg permalink I am doing comp engg in mumbai i have to complete this project in the coming months .. can any o u guys help much appreciated .

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef Reply 12. roni permalink hi, when I look at the eigenfaces I get from the program, I get nothing like yours. I very dark pictures, almost all black with a bit of gray lines. I am using the same database u used and I cropped the background out. so I have pics of faces without the background . Reply

Page 9

alister permalink If you increase the contrast of the images can you make out anything? Reply

roni permalink hi I think I found my mistake. The images in the database were without background but the faces were not aligned to the center, so I had different positions of black background giving me much more variance , which was represented by black spots . I believe if I try with center-alligned images it will work . Thanks.

13. rahul satija permalink Hi.i am facing the same problem as Irlmaks. I have used pics with background . I tried to do everything you described in the way it is. For the input image i have used one of the test images. But the result is rather horrible which gives out some arbitrary image number with a very low similarity score of about 0.0002. Also when displaying the result it is not the image corresponding to the detected number, rather a black and white image with very few black image pixels. Can you point out the possible mistake that i am making? Reply 14. shashank permalink could u plz explain the significance of calculating the eigen vector in this algo ???? Reply 15. dshivkiran87 permalink >> input_dir = path; >> image_dims = [200,100]; >> filenames = dir(fullfile (input_dir,*.jpg)); >> num_images = numel(filenames); >> images = []; >> for n = 1:num_images filename = fullfile (input_dir, filenames(n).name); img = imread(filename); if n == 1 images = zeros(prod(image_dims),num_images); end images(:,n)=img (:) end

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef ??? Subscripted assignment dimension mismatch. can u help me to solve this error :) Reply

Page 10

rahul permalink hi dshivkiran87, the problem u r facing is that u r using rgb images..first convert it into grayscale and then use them to make the training set.I m posting a part of my code .hope u will get help from it. n= 1:no_images filename = fullfile (a, fn(n).name); img = imread(filename); img=rgb2gray(img ); img = imresize(img ,image_dims); Reply 16. radhika permalink hi, I have some problem in the PCA phase of the code. It is causing some memory error. The error is : ??? Error using ==> svd Out of memory. Type HELP MEMORY for your options. Error in ==> princomp at 85 [U ,sigma,coeff] = svd(x0,econFlag); % put in 1/sqrt(n-1) later Error in ==> bestofluck at 6 [evectors, score, evalues] = princomp(images); what is this error related to ? could you help please ? Reply

richa permalink let the image_dim to be [48 64] not the default image size Reply 17. prajakta permalink at the step 6-[evectors, score, evalues] = princomp(images);, it is giving an err Out of MEMORY. Can u plz help me to solve this err??? Reply 18. mohamed alahmady permalink hi i have aproplem in this code, please quickly >> >> >> >> >> input_dir=C:\Documents and Settings\m\My Documents \My Pictures\faces ; image_dims = [48, 64]; filenames = dir(fullfile (input_dir, *.png)); filenames = dir(fullfile (iput_dir, *.png)); num_images = numel(filenames);

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef >> images = []; >> for n = 1:num_images filename = fullfile (input_dir, filenames(n).name); img = imread(filename); if n == 1 images = zeros(prod(image_dims), num_images); end images(:, n) = img(:); end >> mean_face = mean(images, 2); >> shifted_images = images repmat(mean_face, 1, num_images); >> [evectors, score, evalues] = princomp(images); >> num_eigenfaces = 20; >> evectors = evectors(:, 1:num_eigenfaces ); ??? Index exceeds matrix dimensions . Reply

Page 11

tidusx permalink Hi I see your havn the same problem as me with this code. Just wondering did you manage to find a solution? Reply 19. tidusx permalink Hi, I am currently doing something similar in my college project and just testing out your code so that i can get some practice results. For some reason there is a problem with evectors = evectors(:, 1:num_eigenfaces); It keeps giving me a error saying index exceeds matrix dimensions. Can you help me out with this Reply 20. evya permalink it meens that num_eigenfaces is more than the number of columns in evectors Matrix Reply 21. Nicolas permalink Why do you use princomp(images) and not princomp(shifted _images*shifted_images) ? I thought PCA was using the eigenvectors of the symmetric matrix : shifted _images*shifted_images ? Reply

Leave a Reply

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef

Page 12

Enter your comment here...

Notify me of follow-up comments via email . Notify me of new posts via email .

Post Comment

Exploring your Gmail social network (Python) Calculating variance and mean with MapReduce (Python)

Email Subscription
Enter your email address to subscribe to this blog and receive notifications of new posts by email. Join 17 other followers

Sign me up!

Category Cloud
C# Code snippets Coding tools

Data mining Graph theory Image processing In the news JavaScript

Language processing Machine learning

MATLAB Perl Python Random thoughts

Top Posts & Pages

Eigenfaces face recognition (MATLAB) Object tracking using a Kalman filter (MATLAB) OpenCV Viola & Jones object detection in MATLAB Wake-on-LAN (C#) Accessing your Gmail messages in MATLAB Parsing English numbers with Perl Configuring TortoiseCVS to use TortoiseMerge Calculating variance and mean with MapReduce (Python) Code

Links Pages
Code

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM

Eigenfaces face recognition (MATLAB) #deadbeef

Page 13

Search
Search

Blog at WordPress.com . | Theme: Titan by The Theme Foundry.

http://blog.cordiner.net/2010/12/02/eigenfaces-face-recognition-matlab/

3/13/2012 3:57:25 PM