Question

Face recognition using eigenfaces Collect a set of facial photographs and then build a recognition system to re-recognize the same people.1. Take several photos of each of your classmates and store them. 2\. Align the images by automatically or manually detecting the corners of the eyes and using a similarity transform to stretch and rotate each image to a canonical position. 3\. Compute the average image and a PCA subspace for the face images 4\. Take a new set of photographs a week later and use them as your test set. 5\. Compare each new image to each database image and select the nearest one as the recognized identity. Verify that the distance in PCA space is close to the distance computed with a full SSD (sum of squared difference) measure. 6\. (Optional) Compute different principal components for identity and expression, and use them to improve your recognition results.

Short answer

This exercise asked to create a face recognition system using eigenfaces and PCA. The steps involved collecting and aligning facial photos, computing an average image and PCA subspace, testing the system with new photos, and comparing each new photo to database images.

Step-by-step solution

Collect and Store Photographs

Take numerous photos of each of your classmates and store them for future use. This will form the dataset for the face recognition system.

Align the Images

Detect the corners of the eyes either manually or using an automatic algorithm. Then, using a similarity transform, adjust each image to place it in a canonical position. This ensures that the images are standardized for the face recognition process.

Compute Average Image and PCA Subspace

Compute the average image, and then compute the PCA subspace for the face images. The PCA subspace will capture the most significant variations within the face images.

Collect New Photographs

A week later, repeat step 1 by taking new photographs of your classmates. These will serve as your test set.

Compare Images and Select Identity

Compare each new photograph from the test set to each photograph in the initial database. Select the closest image as the recognized identity. Then, verify that the distance in PCA subspace is close to the distance computed with a full SSD measure. This will ensure that the recognition algorithm is accurate.

(Optional) Compute Principal Components for Identity and Expression

If necessary, further improve your recognition results by calculating different principal components for identity and expression. This will help distinguish between different facial expressions as well as identify individual faces.

Key concepts

PCA Subspace for Face Images

Principal Component Analysis (PCA) is a statistical method used to reduce the dimensionality of data while preserving as much variability as possible. In the context of face recognition using eigenfaces, PCA is employed to create a subspace where faces are represented more compactly compared to the high dimensionality of raw image pixels.

When we compute the PCA subspace, we start by aligning a set of face images and then calculate the average face image. This average image captures the common features across all images, such as the general position of eyes, nose, and mouth. From there, we subtract this average from each face image to center the dataset around zero.

Next, we calculate the eigenvectors and eigenvalues of the covariance matrix of the dataset. The eigenvectors, known as eigenfaces, represent the directions of maximum variance in the data. The eigenfaces with the largest eigenvalues carry the most significant information about the variations between faces. We keep only those eigenfaces that correspond to the largest eigenvalues, creating a subspace. Any new or test face image can now be projected into this subspace, and its position can be compared to existing faces within that subspace to perform recognition.

Similarity Transform in Image Alignment

To ensure that facial recognition systems operate accurately, it's crucial to align face images so that corresponding features of different faces coincide. This process is known as image alignment, and it typically involves a similarity transform.

A similarity transform includes translation, rotation, scaling, and sometimes flipping, to ensure that the faces in different images are oriented and sized similarly. For face images, a common approach is to take key facial landmarks, such as the corners of the eyes and the tip of the nose, as reference points. The goal is to align these landmarks across all images.

In practice, once the key landmarks are identified either manually or using an automatic detection algorithm, the similarity transform calculates the parameters needed to align the images. This ensures that the eyes, nose, and mouth are in the same position across all images. When images are consistently aligned, the PCA can be more effective because it reduces the variability due to misalignment, which is not relevant for recognizing identity.

Sum of Squared Difference (SSD)

The Sum of Squared Difference (SSD) is a simple yet powerful measure of similarity between two images. In the domain of face recognition, SSD is often used to quantify the dissimilarity between the test image and each image in the database.

To calculate SSD, we start by taking corresponding pixels from two images, subtract one from the other, square the result, and then sum up all those squared values. Mathematically, if we have two images, I and J, the SSD is calculated as follows: \[ SSD = \sum_{x,y} (I(x,y) - J(x,y))^2 \]

Where \((x,y)\) are pixel coordinates. The lower the SSD, the more similar the images are. When comparing the PCA-based face recognition approach to a full SSD measure, checking whether the distances in PCA space are close to the SSD results can confirm the effectiveness of the PCA approach. If distances in PCA space correlate well with those given by SSD, it's an indication that the PCA subspace retains the essential characteristics necessary to differentiate between individuals' faces.