Why matrix?

Advanced Statistics
W2: Matrix Algebra Basics

This meeting introduces the basic elements of

matrix algebra used in multivariate statistics.
l
l

Titis Wijayanto
Industrial Engineering, UGM

Why matrices? Lots of data and calculation

Why algebra? Allows you to find unknowns.
To do matrix algebra you need to use the matrix manipulations youve
will lear, e.g.
l
l
l
l

Why matrix?
l

Matrix algebra is the fundamental notational technique used in

multiple correlation, MANOVA, factor analysis, structural
equation modeling (SEM), etc.

Although it is possible to use statistical programs without

understanding matrix algebra, it is much harder to do so.

Matrix algebra is a convenient notational system that allows us

to think about data at a higher (broader) level rather than data
point by data point.

Addition and subtraction

Multiplication
Division
Powers

e.g. Inner / outer

e.g. Inverse
e.g. Transpose

Notation and basic definitions:

Matrix
A matrix is a rectangular or square array of
numbers/variables arranged in rows and
columns.

a11 ,, a1n
a 21 ,, a 2n
A=
= {Aij}

am1 ,, amn

Notation and basic definitions:

Column Vector

Notation and basic definitions:

Row Vector

[m x 1] matrix

a1
a 2
a = = {ai}


am

Notation and basic definitions:

Square Matrix
Same number of rows and columns

5 4 7
B = 3 6 1
2 1 3

[1 x n] matrix

a ' = [a1 a 2 , , an ] = {aj}

We use lowercase letter with to
indicate row vector as the transpose of
column vector

Notation and basic definitions:

Equal Matrix
If A = [aij ]mn , B = [bij ]mn
Then A = B if and only if aij = bij 1 i m, 1 j n
Ex 1: (Equal matrix)

1 2
A=

3 4

a b
B=

c d

If A = B

Then a = 1, b = 2, c = 3, d = 4

Notation and basic definitions:

Symmetric Matrix

Notation and basic definitions:

Symmetric Matrix

Symmetric matrix:
Ex:

A square matrix A is symmetric if A = AT

Skew-symmetric matrix:
A square matrix A is skew-symmetric if AT = A

is a skew-symmetric, find a, b, c?

0 1 2
A = a 0 3
b c 0

0 a b
A = 1 0 c

2 3 0

Sol:

Ex:

Sol:

0 1 2
If A = a 0 3

b c 0

1 2 3
If A = a 4 5

b c 6

is symmetric, find a, b, c?

1 2 3
1 a b

T
A = a 4 5 A = 2 4 c

b c 6
3 5 6

A = AT a = 1, b = 2, c = 3
A = AT
a = 2, b = 3, c = 5

Notation and basic definitions:

Diagonal matrix and trace
If a matrix contains zeros in all off diagonal positions, it
is said to be a diagonal matrix.

d1
0
A = diag (d1 , d 2 ,!, d n ) =
"
0

Trace:

If A = [aij ]nn

Then Tr ( A) = a11 + a22 + ! + ann

0
d2
"
0

0
! 0
M nn
# "
! d n

Notation and basic definitions:

Identity Matrix
Square matrix with ones on the
diagonal and zeros elsewhere.

1
0
I =
0

0 0 0
1 0 0
0 1 0

0 0 1

Notation and basic definitions:

Transpose Matrix
Rows become columns and columns become rows

Notation and basic definitions:

Transpose Matrix
Properties of transposes:

a11 a 21 ,, am1
a12 a 22 ,, am 2
A' =

a1n a 2n ,, amn

(1) ( AT )T = A

(2) ( A + B)T = AT + BT
(3) (cA)T = c( AT )
(4) ( AB)T = BT AT

The transpose of a matrix A, denoted by A, is

obtained from A by interchanging rows and
columns.

Matrix operations

Matrix operations
Scalar multiplication:

Matrix addition:

If A = [aij ]mn , B = [bij ]mn

Then A + B = [aij ]mn + [bij ]mn = [aij + bij ]mn
Ex 2: (Matrix addition)

1 2 1 3 1 + 1 2 + 3 0 5
0 1 + 1 2 = 0 1 1 + 2 = 1 3

1 1
3 + 3 =

2 2

1 1 0
3 + 3 = 0

2 + 2 0

If A = [aij ]mn , c : scalar

Then cA = [ca ij ]mn
Matrix subtraction:

A B = A + (1) B
Ex 3: (Scalar multiplication and matrix subtraction)

1 2 4
A = 3 0 1

2 1 2

0 0
2

B = 1 4 3

3 2
1

Find (a) 3A, (b) B, (c) 3A B

Matrix operations

Matrix operations

Ex 4: (Find AB)

Matrix multiplication:

3
1

A = 4 2

0
5

If A = [aij ]mn , B = [bij ]n p

Then AB = [aij ]mn [bij ]n p = [cij ]m p
Size of AB
where

Sol:

cij = aik bkj = ai1b1 j + ai 2b2 j + ! + ainbnj

k =1

a11 a12 ! a1n b ! b

! b1n
11

1j
"
"
"

b
"
b
!
b
21
2j
2n

=
ai1 ai 2 ! ain
"
"
"
" ci1 ci 2 ! cij ! cin
"

"
"

b ! bnj ! bnn

an1 an 2 ! ann n1
Notes: (1) A+B = B+A, (2)

3 2
B=

4 1

AB BA

(1)(3) + (3)(4) (1)(2) + (3)(1)

AB = (4)(3) + (2)(4) (4)(2) + (2)(1)

(5)(2) + (0)(1)
(5)(3) + (0)(4)
9 1
= 4 6

15 10

Matrix operations
Determinants

Matrix form of a system of linear equations:

a11 x1 + a12 x2 + ! + a1n xn = b1

a x + a x +!+ a x = b
21 1 22 2
2n n
2

"

am1 x1 + am 2 x2 + ! + amn xn = bm

m linear equations

The determinant of an nn matrix A is defined as the sum of

all n! possible products of n elements such that
1. each product contains one element from every row and
every column, and

a11 a12 " a1n x1 b1

a

21 a22 " a2 n x2 = b2
!
!
!
! ! !

am1 am 2 " amn xn bm
=

Single matrix equation

Ax =b

m n n 1

m 1

2. the factors in each product are written so that the column

subscripts appear in order of magnitude and each product
is then preceded by a plus or minus sign according to
whether the number of inversions in the row subscripts is
even or odd.

The Inverse of a Matrix

Determinants
Inverse matrix:

The determinant of A is a scalar denoted by |A| or by

det(A).

Consider

A M nn

If there exists a matrix B M nn such that AB = BA = I n ,

Then (1) A is invertible (or nonsingular)
(2) B is the inverse of A

Note:
A matrix that does not have an inverse is called noninvertible (or
Positive

Negative

singular).

The Inverse of a Matrix

Eigenvalues and eigenvectors

The inverse of a matrix is unique)

If B and C are both inverses of the matrix A, then B = C.
Pf:

AB = I

Let A be an nxn matrix. A scalar is called an eigenvalue

of A if there is a nonzero vector x such that
Ax = x.

C ( AB ) = CI
(CA) B = C
IB = C
B=C

such vector x is called a eigenvector A corresponding to

Consequently, the inverse of a matrix is unique.

Notes:
(1) The inverse of A is denoted by

(2) AA1 = A1 A = I

A1

Eigenvalues and eigenvectors

2
3
Show that x = 1 is an eigenvector of A =

Corresponding to

3 2

Eigenvalues and eigenvectors

When A is squared, the eigenvectors stay the same.
The eigenvalues are squared

=4

Is it true that
?

Ax = x
If is an eigenvalue of A and x is an eigenvector
belonging to , any nonzero multiple of x will be an
eigenvector

Eigenvalues and eigenvectors

to find and x, we use the following equation:

(A I)x = 0
The product of the n eigenvalues equals the determinant.
The sum of the n eigenvalues equals the sum of the n diagonal
entries (trace).

Eigenvalues and eigenvectors

Example:
Find the eigenvalues and eigenvectors of

2 1
A =

1 2

Eigenvalues and eigenvectors

Try next:
Find the eigenvalues and eigenvectors of A2, A-1, and (A+4I)

Quiz
1. Find the eigenvalues of these matrices. All powers have the same
eigenvectors.

2. Find the eigenvalues of A and B and AB and BA

Matrix in R

Matrix in R

Now open your R software!

Matrix operation

Making a Matrix in R
A <matrix(data=c(7,18,-2,22,-16,3,55,1,9,-4,0,31),byrow=
TRUE,nrow=3,ncol=4)
#check the dimensions
dim(A)
Making an Identity Matrix (I) in R
id <- function(n)diag(c(1),nrow=n,ncol=n)
# To create an identity matrix of order 12
I12 <- id(12)

Transposition
At <- t(A)
# t() is the transpose function
Diagonals
diag(A)
Addition/Substraction of matrix
A+A
#addition of matrix A and A
Multiplication of matrices
A%*$At
#matrix At multiplied by A