Some Basics of Matrix Calculation

Let us consider a (n × k) matrix A = {aij }n,k

i,j=1 from IR
(n×k)
. First, let us consider
quadratic matrices, i.e. n = k.
Pn
We define the trace of A to be the sum of the diagonal elements tr(A) = i=1 aii .

The determinant of a matrix is defined by

n
X
det(A) = |A| = (−1)i+j aij · det(Aij ),
j=1

where Aij is matrix A with row i and column j canceled.

Example 1  
a b 
1. det  = ad − cb
c d
 
0 1 2 ¯ ¯ ¯ ¯ ¯ ¯
¯ ¯ ¯ ¯ ¯ ¯
  ¯ 2 1 ¯ ¯ 3 1 ¯ ¯ 3 2 ¯
2. det 
 3 2 1  = 0¯
 ¯
¯ −1· ¯
¯ ¯
¯ +2· ¯
¯ ¯
¯ = 0·(−1)−1·(−1)+2·1 = 3
¯
¯ 1 0 ¯ ¯ 1 0 ¯ ¯ 1 1 ¯
1 1 0

Note: For n = k = 3 there is the general rule

 
a11 a12 a13
 
det  
 a21 a22 a23  = a11 a22 a33 −a11 a23 a32 −a12 a21 a33 +a12 a23 a31 +a13 a21 a32 −a13 a22 a31
a31 a32 a33

There exist many important and interesting rules for calculating the determinant as
well as in which cases det(A) = 0. Repeat them!

We call AT the transposed of A (make graphic if necessary). The rule of calculation

is
(ABC)T = C T B T AT with A, B, C compatible matrices.
If A = AT we call A symmetric.

A matrix is called regular if it has full rank rank(A) = n = k, this is that all
column (or row) vectors are linearly independent and thus the space spanned by A
has dimension n = k, = span(A). If they are not linear independent (i.e. one of the
variables is a linear combination of the others) it is called singular.

We call a matrix invertible if there exist a matrix B so that AB = I = BA with I

being the identity matrix. We set B = A−1 , called the inverse of A.

1
We have the basic rules

(A−1 )−1 = A , (AB)−1 = B −1 A−1

(AT )−1 = (A−1 )T , (A−1 )T AT = (AA−1 )T = I

and further, the following statements are equivalent

a) matrix A is invertible, i.e. A−1 exists
b) matrix A is regular (nonsingular), i.e. has full rank
c) the determinant is unequal zero, det(A) 6= 0
d) the transposed (AT ) is invertible
How to calculate the inverse ?
In general, the question is to solve the following equation system
X X
AB = I , i.e. aij bji = 1 ∀i and aij bjk = 0 ∀i 6= k
j j

This problem can be solved in several ways; the two most common are

1. Write on one side A, on the other I. Now perform trivial row or column operations
on A, and the same on I until A has been converted to I =⇒ and I to A−1 . Check by
multiplying A with the obtained result.

2. The mathematical formula is

1
A−1 = CT where cij = (−1)i+j det(Aij )
det(A)
see definition of Aij above.

Example 2 Recall Example 1

 −1  

a b  1 
d −b 
1. =
c d ad − cb −c a
 −1  T
0 1 2 −1 1 1
  1 

2.  3 2 1 


=  2 −2 1   = ···
3
1 1 0 −3 6 −3

We call a matrix positive definite if xT Ax > 0 for all x 6= 0, positive semidefinite

if xT Ax ≥ 0, negative definite if xT Ax < 0, and negative semidefinite if xT Ax ≤ 0
for all x 6= 0. (Say also non-· · ·.)

Note: A symmetric positive or symmetric negative definite matrix is always invert-

ible. To know this is important when later looking on the variance of the β estimates.

2
For regression in statistics projections are of special interest as our fitted values Ŷ
are projections of Y into the model space, i.e. Xβ, respectively into span(X). A
projection matrix PA , or often also called V , that projects into span(A), is constructed
by
PA = A(AT A)−1 AT
and thus symmetric and idempotent, i.e. PA2 = PA (explain and discuss intuitively!).
It holds that rank(PA ) = rank(A) but notice that if n 6= k, A and PA do not have
the same dimension. The orthogonal projection is defined as (I − PA ), i.e. this is the
projection into the space orthogonal to span(A). For consequences see next subsection!

Finally we define Eigenvalues and Eigenvectors: λ ∈ IR is called eigenvalue of A

with eigenvector v if v exists so that Av = λv. Certainly, the eigenvectors of A are a
basis of span(A). If there even exists an invertible matrix S so that

SAS −1 = Diag(λ1 , . . . , λk )

then we say that we can diagonalize A.

But how to find the eigenvalues?

Construct the characteristic polynomial of A, i.e. P olc = det(A − t · I) (the ’−t’
affects only the diagonal elements) and search for the roots of this polynomial P olc . If
Q
we can write P olc = l (γl − t) then we can diagonalize A and the roots (the γl ) are
the eigenvalues, i.e. γl = λl .