Assignment 2: Face Recognition using PCA

Berk Gokberk
Introduction to Biometrics

In this assignment, you are going to design a classical face identication system using 1) standard Principal Component Analysis (PCA) method and 2) the Eigenface approach. PCA and Eigenface [1] techniques will be used to extract features from face images. For the PCA method, the steps of the feature extraction method are as follows: 1. Given a set of M face images: I1 , I2 , . . . IM , vectorize pixel values to obtain pixel column vectors i . If the row and column size of an image Ii is P Q, then the size of i is (P Q) 1. Let R be R = P Q, for simplicity. You can vectorize a face image by simply concatenating each pixel row side by side. i will also be referred to as a face image from now on. 2. Form a training matrix A = [1 , 2 , . . . , M ] containing a face image at every column. The size of A is R M . 3. Compute the covariance matrix,C, of the training data A. You can use cov function of Matlab. Note that cov function assumes each row contains a sample which is dierent (columnwise) for the A matrix. The size of C is R R. 4. Compute the eigenvectors, uk , and eigenvalues k of C. The size of an eigenvector uk is R 1. If you de-vectorize uk , to obtain P Q image (the reverse operation you perform in Step.1), you get an Eigenface. 5. Select K eigenvectors, u1 , u1 , u3 , . . . uK , having the biggest corresponding eigenvalues: 1 > 2 > 3 > . . . > K and form the PCA transformation matrix U = [u1 , u2 , . . . uK ]. The size of U is R K. 6. PCA Projection: Project (or extract the feature of) a given face image, i , into the PCA space by: i = U T i . The size of i is K 1. Above, steps from 2 to 5 is called PCA training. The output of PCA training is the PCA transformation matrix U . Step 6 explains how to use U to extract PCA features from a given face image. For the Eigenface approach, Step 3 and 4 are dierent at the training phase. In the Eigenface approach, eigenvectors and eigenvalues of the matrix AT A are computed, instead of using the covariance matrix C. Therefore maximum feature dimensionality of reduced features is bounded by the number of images in the training set. Additionally, the training matrix A is slightly dierent for the Eigenface approach: A is formed by subtracting the mean face from every face image. Therefore A = [1 a , 2 a , . . . , M a ] in the Eigenface approach where a is the mean face of the training images. a is computed by averaging pixel values of the training face images. During projection phase, Step 6, mean face is subtracted from the face image before multiplication operation. For further details on the

Eigenface technique, please refer to Calculating Eigenfaces section (pages 73-75) in [1]. It is possible to reconstruct the original face image, up to a certain quality, from the PCA/Eigenface features. To reconstruct the face image, i from PCA i = U i . The size of feature vectors, i , (or, say PCA coecients), compute i is R 1. By, reversing the row concatenation operation, you can convert back to original image size to obtain Ii . In this assignment, you are a given a face database, the ORL face database, containing 40 subjects with 10 face images per subject. You will use the rst ve images of each person to form training matrix A in order to compute U . Using U , you will compute PCA and Eigenface features for all the images in the ORL face database. The remaining ve images of every subject will be used as a test set. You are going to nd the identity of images in the test set. Each subject will have the rst ve images as their enrolled images. Therefore, in our system, each subject is represented by their ve PCA/Eigenface feature vectors. In order to compute the dissimilarity of two feature vectors, Euclidean distance will be used: d(i , j ) = i , j , where . denotes Euclidean distance. In your report, youre going to provide the following: 1. Plot the rst 20 Eigenfaces (Eigenvectors) for the standard PCA and Eigenface approach. Example Eigenfaces for the PCA method is shown Figure 1.

Fig. 1. First 20 Eigenfaces/Eigenvectors computed by the standard PCA method.

2. Reconstruct a sample face image from the test set using PCA features of size: [2, 5, 10, 20, 40, 60, 100, 150, 200, 400, 1000, 2000] and plot the reconstructed

images together with the original face image. See Figure 2 for an example. Do the same thing for a non-face image. See Figure 3. Comment on the dierence between these two cases in terms of the reconstruction ability of the PCA method.

Fig. 2. Reconstructed face image.

3. Perform Step 2 for the Eigenface approach. Compare the quality of reconstructions of the standard PCA and Eigenface approaches. 4. Using your test set, plot the rank-1 identication accuracy (y-axis) vs PCA dimensionality (x-axis) plot. As PCA dimensionality (i.e., PCA feature vector size), you can choose [5, 10, 15, 20, 25, . . . , 2500]. As a subset of this plot, provide some rank-1 recognition rates together with PCA feature vector size (i.e., selected Eigenface count) and the total variance explained as a table. See the format of Table 1 as an instance. The total variance explained
K

by selecting the biggest K eigenvectors is computed by

i=1 R i=1

i i

. Note that

eigenvalues are sorted in descending order: i > i+1 . 5. Perform Step 4 for the Eigenface approach. Compare and comment on the the identication accuracies of the Eigenface and standard PCA approach. Your are going to deliver your Matlab source code. The main script of your assignment should be named as assignment2.m. When assignment2.m runs, it should produce the 1) Eigenface plots, 2) Reconstruction face example plot (only one face image is OK), 3) a rank-1 identication rate for a specic PCA dimensionality (Your code is not expected to produce PCA dimensionality vs Rank-1 identication rate), and 4) a rank-1 identication rate for a specic Eigenface dimensionality. You can have separate functions that are called from assignment2.m as well. In that case, please submit all your .m les.

Fig. 3. Reconstructed non-face image.

The ORL face database is provided as a Matlab .mat le, FaceData_56_46.mat. It contains a Matlab struct, FaceData. For instance, the third image of subject 39 can be accessed as: I = FaceData(39,3).Image. Since images are stored as unsigned integers (uint8), you might nd it useful to convert them to double format by: I = double(I)/255; (This code also maps the image pixel values from [0 255] to [0 1]). Tips For performance plots (accuracy vs PCA/Eigenface dimensionality) required at Step 4 and 5, you do not need to re-compute eigenvectors/eigenvalues at each dimensionality. Compute them once and use the required part of the eigenvector matrix for feature extraction. This saves a lot of time. When plotting images/eigenvectors, use imshow() function. If the displayed image is not meaningful, try imshow(Image,[]).

Table 1. The performance of PCA based face recognizer with respect to feature dimensionality.
Number of Variance Rank-1 Recognition Eigenfaces Explained Rate 10 0.31 34.77% 50 0.53 67.23% . . . 1000 0.99 78.03%

References
1. Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cognitive Neuroscience 3(1), 7186 (1991)