Ch# 8: Matrix Algebra

Matrix
An array of numbers.
Size is indicated by number of rows
and columns.
A matrix with k rows and n columns is
a k n matrix (memory trick, since
convention is for row number to come
before column number, think RC
Matrices are e!ual if they both have the
same size and the same corres"ondin#
entries.
Matrix Addition
Matrices must be of same size.
$heir sum is a new matrix of the same
size.
$he (i, j entry of the new matrix is
the sum of the (i, j entries of the
matrices bein# added.
%.#.

,
_

+ +
+
+ +

,
_

,
_

kn kn k1 k1
ij ij
1n 1n 11 11
kn k1
ij
1n 11
a a
a
a a
a a
a
a a
b b
b
b b
b b
b
b b
kn k
ij
n

&
& &&
'r

,
_

,
_

,
_

( &) *
& &* &*
( + *
) , &
& - .
, ) &
( / &
) , .
& + /

0ut the followin# addition is
undefined1

,
_

+
,
_

+ &
. /
* * )
/ * &
2o. 'f rows and columns should be the same
$he zero matrix is an additive identity1

,
_

,
_

,
_

kn k1
ij
1n 11
kn k1
ij
1n 11
a a
a
a a
a a
a
a a

) )
)
) )
$hus, A 3 0 4 A for all matrices A
Subtraction
5irst, note that

,
_

,
_

kn k1
ij
1n 11
kn k1
ij
1n 11
a a
a
a a
a a
a
a a

Since A 6 B 4 A
3 (6B1

,
_

,
_

,
_

kn kn k1 k1
ij ij
1n 1n 11 11
kn k1
ij
1n 11
a a
a
a a
a a
a
a a
b b
b
b b
b b
b
b b
kn k
ij
n

&
& &&
%.#.

,
_

,
_

,
_

( + )
& * )
. + +
) , &
& - .
, ) &
( / &
) , .
& + /
0ut the followin# subtraction is
undefined1

,
_

,
_

+ &
. /
* * )
/ * &
Scalar: A one7by7one matrix. 8n other
words, an ordinary number.
Scalar multiplication
8f r is a scalar, rA is the matrix created
by multi"lyin# each entry of A by r.

,
_

,
_

kn k1
ij
1n 11
kn k1
ij
1n 11
a a
a
a a
a a
a
a a
r r
r
r r
r

Matrix Multiplication
AB only exists if the number of
columns of A is e!ual to the number
of rows of B.
$o find the (i, j
th
entry of AB,
multi"ly the i
th
row of A and the j
th

column of B as follows1
( )

,
_

mj
j
j
im i i
b
b
b
a a a

*
&
* &

4 a
i&
b
&j
3 a
i*
b
*9
3 : 3 a
im
b
m9
4

m
h
hi ih
b a
&
%.#.

,
_

+ +
+ +
+ +

,
_

,
_

fD eB fC eA
dD cB dC cA
bD aB bC aA
D C
B A
f e
d c
b a
;owever, if we reverse the order of
multi"lication

,
_

,
_

f e
d c
b a
D C
B A
$he "roduct is undefined.
8f A is k m and B is m n, the
"roduct AB will be k n1
< of rows of AB 4 < of rows of A
< of columns of AB 4 < of columns
of A
Consider the n n identity matrix.

,
_

& ) )
) & )
) ) &

I
5or any m n matrix A1
AI 4 A
5or any m n matrix B1
IB 4 B
Sort of like multi"lyin# real numbers
by &.
a!" o# Matrix Algebra:
Associative =aws1
(A3B 3 C 4 A 3 (B3C
(ABC 4 A(BC
Cummutative =aw for Addition1
A 3 B 4 B 3 A
>istributive =aws1
A(B3C 4 AB 3 AC
(A3BC 4 AC 3 BC
Cummutative law for multi"lication
(ab 4 ba doe" not hold.
8n matrices A0 4 ) can be true
without A or 0 bein# zero
%.#.

,
_

,
_

+ +
+ +

,
_

,
_

& &
) *
* & ) &
* * ) *
* )
& &
& &
& *

,
_

,
_

+ +


,
_

,
_

* *
) &
* ) * )
& & & *
& &
& *
* )
& &
$otation: 8f C is a matrix, we refer
to the (i, j element as C
ij
.
%ran"po"e"
$he trans"ose of a k n matrix A is
obtained by interchan#in# the rows
and columns of A. $hus, the (i, j
th

entry becomes the (j, i
th
entry of
the trans"osed matrix.
$he book writes the trans"ose of A
as A
%
.
Another common notation is A&.
%.#.

,
_

,
_

*/ &/
** &*
*& &&
*/ ** *&
&/ &* &&
a a
a a
a a
a a a
a a a
T
( )
&* &&
*&
&&
a a
a
a
T

,
_

,
_

,
_

? . /
( - *
, + &
? ( ,
. - +
/ * &
T

,
_

,
_

. - +
/ * &
. /
- *
+ &
T
( ) + / * &
+
/
*
&

,
_

T
( )

,
_

/
*
&
/ * &
T
2ote1 8f A and B are k n matrices
and r is a scalar
(A 3 B
$
4 A
$
3 B
$
(A 6 B
$
4 A
$
6 B
$
(A
$

$
4 A
(rA
$
4 rA
$
(AB
$
4 B
$
A
$
@roof (not re!uired1 (of the last
relationshi"1
((AB
$

ij
4 (AB
ji
(def. of trans"ose
4 A
h
A
jh
BB
hi
(def. of matrix
multi"lication
4 A
h
(A
$

hj
B(B
$

ih
(def. of trans"oses
4 A
h
(B
$

ih
B(A
$

hj
(commutative law
of multi"lication
for scalars
4 (B
$
A
$

ij
(>ef. of matrix
multi"lication
C%>
S'"tem" o# e(uation" in Matrix #orm
a
&&
x
&
3 : 3 a
&n
x
n
4 b
&
a
*&
x
&
3 : 3 a
*n
x
n
4 b
*
) ) ) )
) ) ) )
) ) ) )
a
k&
x
&
3 : 3 a
kn
x
n
4 b
k
=et

,
_

kn k1
ij
1n 11
a a
a
a a

A
,

,
_

n
x
x
x
&
, and

,
_

k
b
b
b
&
x and b are column matrices,
sometimes referred to as column
vectors. $his allows the system of
e!uations to be written as

,
_

,
_

,
_

k n
b
b
x
x

& &
kn k1
ij
1n 11
a a
a
a a
'r more com"actly as
Ax 4 b
%.#.
x
&
6 ).+x
*
6 )./x
/
4 &/)
6).*x
&
3 ).((x
*
6 ).&+x
/
4 ,+
6).-x
&
6 ).*x
*
3 ).?-x
/
4 ?-
Can be written as

,
_

,
_

,
_

?-
,+
&/)
?- . ) * . ) - . )
&+ . (( . * . )
/ . ) + . ) &
/
*
&
x
x
x
Additional Special Matrice"
S(uare matrix: 2umber of columns
is e!ual to number of rows. %.#.

,
_

** *&
&* &&
a a
a a
or

,
_

?- . ) * . ) - . )
&+ . (( . * . )
/ . ) + . ) &
Column Matrix or *ector
A & n matrix. %.#.

,
_

c
b
a
and

,
_

&
)
+o! Matrix
A k & matrix. %.#.
( ) ) & *
and
( ) / *
,iagonal Matrix or *ector
A s!uare matrix with a
ij
4 ) iDj
%.#.

,
_

b
a
)
)
and

,
_

/ ) )
) * )
) ) &
-pper.%riangular Matrix
a
ij
4 ) i E j
Fsually a s!uare matrix.
%.#. a matrix in row echelon form.
%.#.

,
_

d
b a
)
and

,
_

. ) )
- + )
/ * &
o!er.%riangular Matrix
a
ij
4 ) i G j
Fsually a s!uare matrix.
%.#.

,
_

d c
a )
and

,
_

. - +
) / *
) ) &
S'mmetric Matrix
A
$
4 A, or e!uivalently a
ij
4 a
ji
i and j
%.#.

,
_

d b
b a
and

,
_

. - /
- + *
/ * &
Idempotent Matrix
A s!uare matrix B for which BBB 4
B.
%.#.

,
_

+ +
- -

,
_

,
_

+
+

,
_

,
_

+ +
- -
&+ *) &. *)
*) *- *) *-
+ +
- -
+ +
- -
/ermutation Matrix
A s!uare matrix of )s and &s in
which each row and column
contains exactly one &.
%.#.

,
_

& ) )
) ) &
) & )
0lementar' Matrice"
Recall the elementary row o"erations1
&. 8nterchan#in# two rows of a matrix.
*. Chan#e a row by addin# to it a
multi"le of another row.
/. Multi"ly each element in a row by the
same nonzero number.
$hese o"erations can be "erformed on
matrix A by "remulti"lyin# it by an
elementary matrix.
$heorem (.*1 5orm the "ermutation
matrix 0
ij
by interchan#in# the i
th

and j
th
rows of the identity matrix I.
=eft7multi"lication of a #iven
matrix A by 0
ij
has the effect of
interchan#in# the i
th
and j
th
rows of
A.
@roof1 =et e
hk
denote a #eneric element
of 0
ij
1
e
ij
4 e
ji
4 &
e
ii
4 e
jj
4 )
e
hh
4 & h D i, j
e
hk
4 ), otherwise.
$he element in row k and column n of
0
ij
A is

'

j i k a
j k a
i k a
a e
kn
in
jn
m
mn km
,

$herefore, 0
ij
A is A with rows i and j
interchan#ed.
C%>
%.#.
=et

,
_

. - /
- + *
/ * &
A
$o interchan#e the first and second
rows, form

,
_

& ) )
) ) &
) & )
&*
E

,
_

,
_

+ + + + + +
+ + + + + +
+ + + + + +

,
_

,
_

. - /
/ * &
- + *
. ) ) - ) ) / ) )
) ) / ) ) * ) ) &
) - ) ) + ) ) * )
. - /
- + *
/ * &
& ) )
) ) &
) & )
A E
ij
$o multi"ly the i
th
row of a matrix by a
nonzero scalar r, form the elementary
matrix by multi"lyin# the i
th
row of the
identity matrix by the scalar r. %.#. in
the case of multi"lyin# the second row
of #eneral / / matrix by -1

,
_

,
_

,
_

// /* /&
*/ ** *&
&/ &* &&
// /* /&
*/ ** *&
&/ &* &&
*
- - -
& ) )
) - )
) ) &
- (
a a a
a a a
a a a
a a a
a a a
a a a
A E
5or the addition of r times the i
th
row
of A to the j
th
row of A, form the
elementary matrix %
i9
(r by addin# r
times row i to row j in the identity
matrix I. %.#. to add - times row * to
row / in the #eneral / / matrix.

,
_

+ + +

,
_

,
_

// ** /* ** /& *&
*/ ** *&
&/ &* &&
// /* /&
*/ ** *&
&/ &* &&
*/
- - -
& - )
) & )
) ) &
- (
a a a a a a
a a a
a a a
a a a
a a a
a a a
A E
$heorem (./1 =et 0 be an elementary
n n matrix formed by "erformin#
a "articular row o"eration on the n
n identity matrix. 5or any n m
matrix A, 0A is the matrix
obtained by "erformin# the same
two o"erations on A.
$heorem (.+1 5or any k n matrix
A, there exist elementary row
matrices 0
1
, 0
1
, ..., 0
m
such that
the matrix "roduct
0
m
B0
m21
BBB0
&
BA 4 - where - is
in reduced echelon form.
%xam"le1

,
_

& + /
/ * &*
& & &
A
$o #et A into row echelon form,
first add 6&* times row & to row *1

,
_

& ) )
) & &*
) ) &
&* (
&*
E
$he add 6/ times row & to row /
and ).& times row * to row /1

,
_

& ) /
) & )
) ) &
/ (
&/
E

,
_

& & . ) )
) & )
) ) &
& . ) (
/ , *
E
$o verify that this #ives us a matrix
in row echelon form1
%
*/
().&B %
&/
(6/B %
&*
(6&*BA

,
_

,
_

,
_

,
_

& + /
/ * &*
& & &
& ) )
) & &*
) ) &
& ) /
) & )
) ) &
& &) H & )
) & )
) ) &

,
_

,
_

,
_

& + /
/ * &*
& & &
& ) )
) & &*
) ) &
& &) H & /
) & )
) ) &

,
_

,
_

& + /
/ * &*
& & &
& &) H & * . +
) & &*
) ) &

,
_

- . / ) )
&- &) )
& & &
In3er"e"
'nly a""lies to s!uare matrices.
=et M
n
be the class of n n
matrices.
$he matrix B in M
n
is an inverse
for A if AB 4 BA 4 I.
8f matrix B exists, we say that A is
invertible.
$heorem (.-1 An n n matrix can
have at most one inverse.
@roof1 Su""ose not. =et B and C
both be inverses of A. $hen
C 4 CI 4 C(AB 4 (CAB 4 IB 4 B.
C%>
2otation1 Ie write the inverse of A
as A
6&
.
>efinition1 =et A be a k n matrix.
$he n k matrix B is a ri#ht
inverse for A4 if AB 4 I. $he n k
matrix C is a left inverse for A if
CA 4 I.
%.#. $he matrix

,
_

* &
& )
& )
is a ri#ht
inverse of

,
_

) & *
& / &
. $o see this

,
_

,
_

,
_

& )
) &
* &
& )
& )
) & *
& / &
0ut it is not a left inverse as

,
_

,
_

,
_

& - -
) & *
) & *
) & *
& / &
* &
& )
& )
Conversely,

,
_

* & &
& ) )
is the left inverse of

,
_

) &
& /
* &
. $o see this

,
_

,
_

,
_

& )
) &
) &
& /
* &
* & &
& ) )
0ut it is not a ri#ht inverse as

,
_

,
_

,
_

& ) )
- & &
- * *
* & &
& ) )
) &
& /
* &
=emma (..1 8f A has a ri#ht inverse
B and a left inverse C, then A is
invertible and B 4 C 4 A
6&
@roof1 See "roof of theorem (.-.
$heorem (..1 8f an n n matrix A is
invertible, then it is nonsin#ular
and the uni!ue solution to the
system of linear e!uations Ax 4 b
is x 4 A
6&
b.
@roof1 Ie want to show that if A is
invertible, we can solve any system
of e!uations Ax 4 b. Multi"ly each
side of this system by A
6&
to solve
for x, as follows1
Ax 4 b
A
6&
(Ax 4 A
6&
b
(A
6&
Ax 4 A
6&
b
Ix 5 A
6&
b
x 5 A
6&
b C%>
$heorem (.,1 8f a n n matrix A is
nonsin#ular, then it is invertible.
%xam"le1 5ind the inverse of

,
_

& + /
/ * &*
& & &
A
5irst au#ment A with the identity
matrix1
[ ]

,
_

& ) ) J & + /
) & ) J / * &*
) ) & J & & &
J I A
Ie want to use elementary row
o"erations until we #et the identity
matrix on the left side of the
"artition. 8n an earlier exam"le, we
#ot to the followin# "oint1

,
_

& & * . + J - . / ) )
) & &* J &- &) )
) ) & J & & &
2ow multi"ly row / by 6*H, and
row * by 6&H&)1

,
_

, H * /- H & * . & J & ) )

) & . ) * . & J * H / & )
) ) & J & & &
Add 6/H* times row / to row * and
6& times row / to row &1

,
_

, H * /- H & * . & J & ) )

, H / /- H * . . ) J ) & )
, H * /- H & * . ) J ) & &
Add 6& times row * to row &1

,
_

, H * /- H & * . & J & ) )

, H / /- H * . . ) J ) & )
, H & /- H / + . ) J ) ) &
$he ri#ht half of this au#mented
matrix is the inverse of A.

,
_

, H * /- H & * . &
, H / /- H * . . )
, H & /- H / + . )
&
A
=et

,
_

d c
b a
A
Irite the au#mented matrix1
[ ]
,
_

& ) J
) & J
J
d c
b a
I A
8f a 4 b 4 ), the matrix is clearly
sin#ular. $o make thin#s more
interestin#, assume a D ). 5irst, add,
6cHa times row & to row *1

,
_

& H J )
) & J
a c
a
bc ad
b a
$his is in row echelon form. $hus,
when a D ), A is invertible if and
only if ad 6 cd D ).
2ow multi"ly the first row by &Ha
and the second row by
bc ad
a

,
_

bc ad
a
bc ad
c
a a b
J & )
) H & J H &
Add KbHa times row * to row &1

,
_

bc ad
a
bc ad
c
bc ad
b
bc ad
d
J & )
J ) &
$hus,

,
_

a c
b d
bc ad
A
&
&
$heorem (.?1 5or any s!uare matrix
A, the followin# statements are
e!uivalent1
(a A is invertable.
(b A has a ri#ht inverse.
(c A has a left inverse.
(d %very system Ax 4 b has at least
one solution for every b.
(e %very system Ax 4 b has at most
one solution for every b.
(f A is nonsin#ular.
(# A has maximal rank n (where A
is a n n matrix.
$heorem (.&)1 =et A and B be s!uare
invertible matrices. $hen,
(a(A
6&

6&
4 A.
(b(A
$

6&
4 (A
6&

$
(cAB is invertible, and
(AB
6&
4 B
6&
A
6&.
As with ordinary numbers, (A3B
6&

is #enerally not A
6&
3 B
6&
.
Integral po!er"
A
m
4 AB ABBBA (m times
%xam"le1

,
_

& &
& *
A
$hen,

,
_

,
_

,
_

* /
/ -
& &
& *
& &
& *
*
A
8f A is invertible, we can define
ne#ative "owers of A as well1
A
6m
4 A
6&
B A
6&
BBBA
6&
(m times
$heorem (.&&1 8f A is invertible1
(aA
m
is invertible for any inte#er
m and (A
m
4 (A
6&

m
.
(b 5or any inte#ers r and s, A
r
A
s
4
A
r3s
.
(c 5or any scalar r D ), rA is
invertible and (rA
6&
4 (&HrA
6&
.
$heorem (.&*1 Any matrix A can be
written as a "roduct
A 4 6
1
B6
1
BBB6
m
B-
Ihere the 6
i
Ls are elementary
matrices and - is in reduced row
echelon form. Ihen A is nonsin#ular,
- 4 I and A 4 6
1
B6
1
BBB6
m