3. Establishing the Eigenface Basis
First of all, we have to obtain a training set of <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">M</nobr> grayscale face images <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">I1,I2,...,IM</nobr>. They should be:
- face-wise aligned, with eyes in the same level and faces of the same scale,
- normalized so that every pixel has a value between 0 and 255 (i.e. one byte per pixel encoding), and
- of the same <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">N×N</nobr> size.
So just capturing everything formally, we want to obtain a set <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">{I1,I2,...,IM}</nobr>, where
<nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">Ik=⎡⎣⎢⎢⎢⎢⎢⎢pk1,1pk2,1⋮pkN,1pk1,2pk2,2pkN,2.........pk1,Npk2,NpkN,N⎤⎦⎥⎥⎥⎥⎥⎥N×N</nobr>
and
<nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">0≤pki,j≤255.</nobr>Once we have that, we should change the representation of a face image <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">Ik</nobr> from a <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">N×N</nobr> matrix, to a <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">Γk</nobr> point in <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">N2</nobr>-dimensional space. Now here is how we do it: we concatenate all the rows of the matrix <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">Ik</nobr> into one big vector of dimension <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">N2</nobr>. Can it get any more simpler than that?
This is how it looks formally:
<center style="color: rgb(0, 0, 0); font-family: Georgia, Times, "Times New Roman", serif; font-size: 18px;">
<nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">Γk=⎡⎣⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢pk1,1pk1,2⋮pk1,Npk2,1pk2,2⋮pk2,N⋮pkN,1pkN,2⋮pkN,N⎤⎦⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥N×1</nobr>
, where
<nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">k=1,...,M</nobr>
and
<nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">pki,j∈Ik</nobr>
</center>
Since we are much more interested in the characteristic features of those faces, let's subtract everything what is common between them, i.e. the average face.
The average face of the previous mean-adjusted images can be defined as <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">Ψ=1M∑Mi=1Γi</nobr>, then each face differs from the average by the vector <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">Φi=Γi−Ψ</nobr>.
Now we should attempt to find a set of orthonormal vectors which best describe the distribution of our data. The necessary steps in this at a first glance daunting task would seem to be:
- Obtain a covariance matrix
<nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">C=1M∑Mi=1ΦiΦTi=AAT</nobr>, where <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">A=[Φ1Φ2...ΦM]</nobr>. - Find the eigenvectors <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">uk</nobr> and eigenvalues <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">λk</nobr> of <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">C</nobr>.
However, note two things here: <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">A</nobr> is of the size <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">N2×M</nobr> and hence the matrix <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">C</nobr> is of the size <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">N2×N2</nobr>. To put things into perspective - if your image size is <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">128×128</nobr>, then the size of the matrix <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">C</nobr> would be <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">16384×16384</nobr>. Determining eigenvectors and eigenvalues for a matrix this size would be an absolutely intractable task!
So how do we go about it? A simple mathematical trick: first let's calculate the inner product matrix <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">L=ATA</nobr>, of the size <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">M×M</nobr>. Then let's find it's eigenvectors <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">vi,i=1,...,M</nobr> of <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">L</nobr> (of the <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">M</nobr>-th dimension). Now observe, that if <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">Lvi=λivi</nobr>, then
<center style="color: rgb(0, 0, 0); font-family: Georgia, Times, "Times New Roman", serif; font-size: 18px;">
<nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">ALviAATAviCAvi===λiAvi⇒λiAvi⇒λiAvi,</nobr>
</center>
and hence <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">ui=Avi</nobr> and <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">λi</nobr> are respectively the <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">M</nobr> eigenvectors (of <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">N2</nobr>-th dimension) and eigenvalues of <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">C</nobr>. Make sure to normalize <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">ui</nobr>, such that <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">∥ui∥=1</nobr>.
We will call these eigenvectors <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">ui</nobr> the eigenfaces. Scale them to 255 and render on the screen, to see why.
It turns out that quite a few eigenfaces with the smallest eigenvalues can be discarded, so leave only the <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">R≤M</nobr> ones with the largest eigenvalues (i.e. only the ones making the greatest contribution to the variance of the original image set) and chuck them into the matrix <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">U=[u1u2...uR]N2×R</nobr>
After you have done that - congratulations! We won't need anything else, but the matrix <nobr style="transition: none; border: 0px; padding: 0px; margin: 0px; max-width: none; max-height: none; min-width: 0px; min-height: 0px; vertical-align: 0px; line-height: normal;">U</nobr>
for the next steps - face detection and classification.