Principal Component Analysis of Scalar and Vector Fields

Mirza Faisal Beg

August 25, 2004

Consider a set of images Ii : Ω → [0, 255] ⊆ R where the index i ∈ [1, M ]. The size of the domain Ω is
usually a 2D square, and it is helpful to imagine the images as a column vector of gray-scale entries ignoring
the spatial correlations between the positions. Let N = |Ω| be the size of each column vector image. The given
set of M images is a small point cloud in the space RN of all possible size N images. The mean image vector
of this set is
M
1 X
I¯ = Ii . (1)
M i=1
√
Consider forming the mean subtracted images Ji√= (1/ M )(Ii − I). ¯ Now construct the matrix J whose ith
column is the mean subtracted (and scaled by 1/ M ) image Ji . The matrix J is of size N × M . The goal is
to find the decomposition
J = U ΣV t . (2)
where U is a matrix of size N × N , Σ is a matrix of size N × M of eigenvalues or singular values, and V is a
matrix of size M × M .
It turns out that
J t J = V Σt ΣV (3)
is of size M × M , where M is the size of the given image database, and non-zero entries of Σt Σ are the squares
of the singular values of J. In comparison,
JJ t = U ΣΣt U t (4)
is matrix of size N × N where N is usually very large, ( 100 × 100 pixel images give N = 10000), with the
non-zero eigenvalues again being the square of the singular values of J. It also turns out that JJ t is nothing
but the covariance matrix of the image database
M
X
K= Ji Jit = JJ t . (5)
i=1

The N eigenvectors of the covariance K (same as those of JJ t ) are given to be say uj , j ∈ [1, N ]. These
are the columns of the matrix U in equation 4 and therefore the covariance is written in the basis of these
eigenvectors to be
K = JJ t = U ΣΣt U t (Mercer’s Theorem) (6)
giving the familiar Mercer’s theorem. Given the columns uj , j ∈ [1, N ] of U , one can write any image I in this
basis to be
N
X
I = I¯ + ¯ uj iRN uj
h(I − I), (7)
j=1

¯ PN
where at any point x ∈ Ω of the domain, the value of the reconstructed image is I(x) = I(x) + j=1 h(I −
¯ uj iRN uj (x). In many cases, the size N of the discrete grid Ω may be too large making it a practical issue to
I),
compute SVD of N × N matrix K = JJ t . In this case, one can then resort to factoring the equation K̃ = J t J
of size M × M where M is the number of images in the database and use the eigenvectors of V from equation
3 for representing any image in the database. Alternatively, given the eigenvectors, (say v˜j , j = 1, . . . , M of size
M × 1) of K̃, then the corresponding eigenvectors uj (size N × 1) of K = JJ t are given by uj = Jvj where

1
J is of size N × M and vj is of size M × 1. The eigenvalues corresponding to both uj and vj are the same.
This is the idea (Sirovich, Kirby 1987) that given M images, the best one can do is find M “good” directions
for representing this database and that the directions themselves will be in the span of the given M images. In
this case, the representation requires the M eigenvectors vj , j ∈ [1, M ] and can be written as
M
X
I = I¯ + ¯ (Jvj )iRN (Jvj ).
h(I − I), (8)
j=1

Another option is to use the iterative method for calculating the first M eigenvectors of K = (1/M )JJ t using
Hotelling’s iterative method (1933).
For vector fields in Rn , the same calculations apply, except that size of each ”image” is of size n × N giving
the covariance to be nN × nN matrix.