4

Elementary Matrix Theory

We will be using matrices:
1. For describing change of basis formulas.
2. For describing howto compute the result of any given linear operation acting on a vector
(e.g., nding the components with respect to some basis of the force acting on an object
having some velocity.)
These are two completely different things. Do not confuse them even though the same computa-
tional apparatus (i.e., matrices) is used for both. For example, if you confuse rotating a vector
with using a basis constructed by rotating the original basis, you are likely to discover that your
computations have everything spinning backwards.
Throughout this set of notes, K , L , M and N are positive integers.
4.1 Basics
Our Basic Notation
A matrix A of size M N is simply a rectangular array with M rows and N columns of
things,
A =
_
_
_
_
_
_
_
_
a
11
a
12
a
13
a
1N
a
21
a
22
a
23
a
2N
a
31
a
32
a
33
a
3N
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
a
M1
a
M2
a
M3
a
MN
_

_
.
As indicated, I will try to use bold face, upper case letters to denote matrices. We will use two
notations for the (i, j )
th
entry of A (i.e., the thing in the i
th
row and j
th
column):
(i, j )
th
entry of A = a
i j
= [A]
i j
Until further notice, assume the thing in each entry is a scalar (i.e., a real or complex number).
Later, well use such things as functions, operators, and even other vectors and matrices.
11/28/2011
Elementary Matrix Theory Chapter & Page: 42
The matrix A is a row matrix if and only if it consists of just one row, and B is a column
matrix if and only if it consists of just one column. In such cases we will normally simplify the
indices in the obvious way,
A =
_
a
1
a
2
a
3
a
N
_
and B =
_
_
_
_
_
_
_
_
b
1
b
2
b
3
.
.
.
b
N
_

_
.
Basic Algebra
Presumably, you are already acquainted with matrix equality, addition and multiplication. So all
well do here is express those concepts using our []
i j
notation:
Matrix Equality Two matrices A and B are equal (and we write A = B) if and only if both
of the following hold:
(a) A is the same size as B.
(b) Letting MN be the size of A and B,
[A]
j k
= [B]
j k
for j = 1 . . . M and k = 1 . . . N .
Matrix Addition and Scalar Multiplication Assuming A and B are both MN matrices,
and and are scalars, then A + B is the MN matrix with entries
[A + B]
j k
= [A]
j k
+ [B]
j k
for j = 1 . . . M and k = 1 . . . N .
Matrix Multiplication Assuming A is a L M matrix and B is a M N matrix, their
product AB is the LN matrix with entries
[AB]
j k
= j
th
row of A times k
th
column of B
=
_
a
j 1
a
j 2
a
j 3
a
j M
_
_
_
_
_
_
_
_
_
b
1k
b
2k
b
3k
.
.
.
b
Mk
_

_
= a
j 1
b
1k
+ a
j 2
b
2k
+ a
j 3
b
3k
+ + a
j M
b
Mk
=
M

m=1
a
j m
b
mk
=
M

m=1
[A]
j m
[B]
mk
.
version: 11/28/2011
Elementary Matrix Theory Chapter & Page: 43
If A is a row matrix, then so is AB and the above formula reduces to
[AB]
k
=
M

m=1
[A]
m
[B]
mk
.
If, instead, B is a column matrix, then so is AB and the above formula reduces to
[AB]
j
=
M

m=1
[A]
j m
[B]
m
.
Do remember that matrix multiplication is not commutative in general, but is associative.
That is, except for a few special cases,
AB = BA ,
but, as long as the sizes are appropriate, we can assume
A(BC) = (AB)C .
Also, you cannot assume that if the product AB is a matrix of just zeros, then either A or B
must all be zeros.
?

Exercise 4.1 a: Give an example where AB = BA.

b: Give an example of two nonzero matrices A and B for which the product AB is a matrix
of just zeros. ( A and B can have some zeros, just make sure that at least one entry in each
is nonzero.)
Inner Products of Matrices
Using the above denitions for addition and scalar multiplication, the set of all MN matrices
form a vector space of dimension MN . The default inner product for this vector space is the
natural extension of the inner product for
N
,
A | B =
M

j =1
N

k=1
a
j k

b
j k
=
M

j =1
N

k=1
[A]
j k

[B]
j k
.
Note that, if A and B are both row matrices
A =
_
a
1
a
2
a
3
a
N
_
and B =
_
b
1
b
2
b
3
b
N
_
or are both column matrices
A =
_
_
_
_
_
_
_
_
a
1
a
2
a
3
.
.
.
a
M
_

_
and B =
_
_
_
_
_
_
_
_
b
1
b
2
b
3
.
.
.
b
M
_

_
then
A | B =

j
a
j

b
j
=

J
[A]
j

[B]
j
.
version: 11/28/2011
Elementary Matrix Theory Chapter & Page: 44
Conjugates, Transposes and Adjoints
In the following, assume A is a MN matrix and B is a NL matrix.
Complex Conjugates
The complex conjugate of A, denoted by A

is simply the matrix obtained taking the complex

conjugate of each entry,
_
A

_
mn
=
_
[A]
mn
_

.
!

Example 4.1:
If A =
_
2 + 3i 5 7 2i
6i 4 + 4i 8
_
then A

=
_
2 3i 5 7 + 2i
6i 4 4i 8
_
.
We will say A is real if and only if all the entries of A are real, and we will say A is
imaginary if and only if all the entries of A are imaginary.
The following are easily veried, if not obvious:
1. (AB)

= A

.
2. (A

= A.
3. A is real A

= A.
4. A is imaginary A

= A.
Transposes
The transpose of the MN matrix A, denoted by either A
T
(my notation) or

A (Arfken &
Webers notation), is the N M matrix whose rows are the columns of A (or, equivalently,
whose columns are the rows of A),
_
A
T
_
mn
= [A]
nm
.
!

Example 4.2:
If A =
_
2 + 3i 5 7 2i
6i 4 + 4i 8
_
then A
T
=
_
_
_
2 + 3i 6i
5 4 + 4i
7 2i 8
_

_
.
Also,
If |a =
_
_
_
1 + 2i
3i
5
_

_
then |a
T
=
_
1 + 2i 3i 5
_
.
If you just think about what transposing a transpose means, it should be pretty obvious
that
_
A
T
_
T
= A .
version: 11/28/2011
Elementary Matrix Theory Chapter & Page: 45
If you think a little bit about the role of rows and columns in matrix multiplication, then you may
not be surprised by the fact that
(AB)
T
= B
T
A
T
.
This is easily proven using our []
j k
notation:
_
(AB)
T
_
j k
= [AB]
kj
=
M

m=1
[A]
km
[B]
mj
=
M

m=1
_
A
T
_
mk
_
B
T
_
j m
=
M

m=1
_
B
T
_
j m
_
A
T
_
mk
=
_
B
T
A
T
_
j k
,
showing that
_
(AB)
T
_
j k
=
_
B
T
A
T
_
j k
for every ( j, k) .
Hence, (AB)
T
= B
T
A
T
as claimed above.
Two more terms you may recall:
A is symmetric A
T
= A [A]
j k
= [A]
kj
.
And
A is antisymmetric A
T
= A [A]
j k
= [A]
kj
.
Note that, to be symmetric or antisymmetric, the matrix must have the same number of rows as
it has columns (i.e., it must be square).
You may recall from your undergraduate linear algebra days that there is a nice theory
concerning the eigenvalues and eigenvectors of real symmetric matrices. Well extend that
theory later.
Adjoints
Combining the transpose with complex conjugation yields the adjoint.
1
The adjoint of the MN matrix A, denoted by either adj(A) or A

, is the NM matrix
A

=
_
A
T
_

=
_
A

_
T
.
Using the []
j k
notation,
_
A

_
j k
=
_
[A]
kj
_

.
!

Example 4.3:
If A =
_
2 + 3i 5 7 2i
6i 4 + 4i 8
_
then A

=
_
_
_
2 3i 6i
5 4 4i
7 + 2i 8
_

_
.
1
The adjoint discussed here is sometimes called the operator adjoint. You may also nd reference to the classical
adjoint, which is something totally different, and is of no interest to us at all.
version: 11/28/2011
Elementary Matrix Theory Chapter & Page: 46
Also,
If |a =
_
_
_
1 + 2i
3i
5
_

_
then |a

=
_
1 2i 3i 5
_
.
From what weve already discussed regarding complex conjugates and transposes, we im-
mediately get the following facts:
1.
_
A

= A.
2. (AB)

= B

.
3. A

= A
T
A is real .
4. A

= A
T
A is imaginary .
Any matrix that satises A

= A is said to be either self adjoint or Hermitian, depending

on the mood of the speaker. Such matrices are the complex analogs of symmetric real matrices
and will be of great interest later.
Along the same lines, any matrix that satises A

= A is said to be antiHermitian. I
suppose you could also call them anti-self adjoint (self anti-adjoint?) though that is not
commonly done.
4.2 Extending the Bra Ket Notation
Using an Orthonormal Basis
Assume we have an N-dimensional vector space V with orthonormal basis
B = { e
1
, e
2
, . . . , e
N
} .
Remember, for any vector v in this space,
|v = |v
B
= column matrix of components of v w.r.t. B
=
_
_
_
_
_
v
1
v
2
.
.
.
v
N
_

_
where v =
N

j =1
v
j
e
j
.
We now dene v| to be the adjoint of |v ,
v| = v|
B
= |v

=
_
v
1

v
2

v
N

_
.
Observe that
_
A|v
_

= v| A

.
version: 11/28/2011
Elementary Matrix Theory Chapter & Page: 47
Also note that
v| |w =
_
v
1

v
2

v
N

_
_
_
_
_
_
w
1
w
2
.
.
.
w
N
_

_
=
N

j =1
v
j

w
j
which, since we are assuming an orthonormal basis, reduces to
v| |w
. ,, .
matrix
product
= v | w
. ,, .
vector
inner
product
.
The | is the bra in the (Dirac) bra-ket notation.
Using an Arbitrary Basis

Now let
B = { b
1
, b
2
, . . . , b
N
} .
be any basis for our vector space V , and let
D = { d
1
, d
2
, . . . , d
N
} .
be the corresponding reciprocal basis. Recall from section 2.6 that
v | w =
N

k=1
v

k
w
k
where
_
_
_
_
_
v
1
v
2
.
.
.
v
N
_

_
= |v
D
and
_
_
_
_
_
w
1
w
2
.
.
.
w
N
_

_
= |w
B
.
In terms of matrix operations, the above is
v | w =
N

k=1
v

k
w
k
=
_
v

1
v

2
v

N
_
_
_
_
_
_
w
1
w
2
.
.
.
w
N
_

_
= |v
D

|w
B
.
Because of this, the appropriate general denition of bra v is
v| = v|
B
= |v
D

where D is the reciprocal basis corresponding to B. That way, we still have

v| |w
. ,, .
matrix
product
= v | w
. ,, .
vector
inner
product
.

This material is optional. It requires the reciprocal basis from section 2.6.
version: 11/28/2011
Elementary Matrix Theory Chapter & Page: 48
4.3 Square Matrices
For the most part, the only matrices well have much to do with will be either row or column or
square matrices.
The Zero, Identity and Inverse Matrices
A square matrix is any matrix having the same number of rows as columns. Two important
NN matrices are
The zero matrix, denoted by O or O
N
, which is simply the NN matrix whose entries are
all zero,
[O]
j k
= 0 for all ( j, k) .
The identity matrix, denoted by I or I
N
or 1 (Afkin & Webers notation), which is the matrix
whose entries are all zero except for 1s on the major diagonal,
[I]
j k
=
j k
.
Recall that, if A is any NN matrix, then
A + O = A , AO = OA = O and AI = IA = A .
!

Example 4.4: The 33 zero and identity matrices are

O =
_
_
_
0 0 0
0 0 0
0 0 0
_

_
and I =
_
_
_
1 0 0
0 1 0
0 0 1
_

_
.
The inverse of a (square) matrix A is the matrix, denoted by A
1
, such that
AA
1
= A
1
A = I .
Not all square matrices have inverses. If A has an inverse, it is said to be invertible or nonsingular.
If A does not have an inverse, we call it noninvertible or singular.
It may later be worth noting that, while the denition of the inverse requires that
AA
1
= I and A
1
A = I ,
it can easily be shown that, if B is any square matrix such that
AB = I or BA = I ,
then
AB = I and BA = I ,
and thus, B = A
1
.
Here are fewother things you should recall about inverses (assume A is an invertible NN
matrix):
version: 11/28/2011
Elementary Matrix Theory Chapter & Page: 49
1. AB = C B = A
1
C.
2. BA = C B = CA
1
.
3.
_
A
1
_
1
= A.
4. If B is also an invertible NN matrix, then
(AB)
1
= B
1
A
1
.
?

Exercise 4.2: Verify anything above that is not immediately obvious to you.
4.4 Determinants
Denition, Formulas and Cramers Rule
Recall that each N N matrix has an associated scalar called the determinant of that matrix.
We will denote the determinant of
A =
_
_
_
_
_
_
a
11
a
12
a
1N
a
21
a
22
a
2N
.
.
.
.
.
.
.
.
.
.
.
.
a
N1
a
N2
a
NN
_

_
by
det(A) or det
_
_
_
_
_
_
a
11
a
12
a
1N
a
21
a
22
a
2N
.
.
.
.
.
.
.
.
.
.
.
.
a
N1
a
N2
a
NN
_

_
or

a
11
a
12
a
1N
a
21
a
22
a
2N
.
.
.
.
.
.
.
.
.
.
.
.
a
N1
a
N2
a
NN

as seems convenient.
The determinant naturally arises when solving a system of N linear equations for N
unknowns
a
11
x
1
+ a
12
x
2
+ + a
1N
x
N
= b
1
a
21
x
1
+ a
22
x
2
+ + a
2N
x
N
= b
2
.
.
.
a
N1
x
1
+ a
N2
x
2
+ + a
NN
x
N
= b
N
(4.1a)
where the a
j k
s and b
k
s are known values and the x
j
s are the unknowns. Do observe that we
can write this system more concisely in matrix form,
Ax = b (4.1b)
version: 11/28/2011
Elementary Matrix Theory Chapter & Page: 410
where A is the NN matrix of coefcients, and x and b are the column matrices
x =
_
_
_
_
_
_
x
1
x
2
.
.
.
x
N
_

_
and b =
_
_
_
_
_
_
b
1
b
2
.
.
.
b
N
_

_
.
In solving this, you get (trust me)
det(A) x
k
= det(B
k
) for k = 1, 2, . . . N (4.2)
where B
k
is the matrix obtained by replacing the k
th
column in A with b. We can then nd
each x
k
by dividing through by det(A) , provided det(A) = 0 . (Equation (4.2) is Cramers
rule for solving system (4.1a) or (4.1b), written the way it should be but almost never is
written.)
?

Exercise 4.3: What can you say about the possible values for the x
k
s if det(A) = 0 in
the above?
In theory, the denition of and formula for the determinant of any square matrix can be based
on equation (4.2) as a solution to system (4.1a) or (4.1b). (Clever choices for the b vectors may
help.) Instead, we will follow standard practice: Ill give you the formulas and tell you to trust
me. (And if you remember how to compute determinants as I rather expect just skip ahead
to Properties and Applications of Determinants.)
For N = 1 and N = 2 , we have the well-known formulas
det[a
11
] = a
11
and

a
11
a
12
a
21
a
22

= a
11
a
22
a
12
a
21
.
For N > 2 , the formula for the determinant of an NN matrix
A =
_
_
_
_
_
_
a
11
a
12
a
1N
a
21
a
22
a
2N
.
.
.
.
.
.
.
.
.
.
.
.
a
N1
a
N2
a
NN
_

_
can be given in terms of the determinants of (N 1)(N 1) submatrices via either the
expansion by minors of the rst row
det A =
N

k=1
(1)
k1
a
1k
det(A
1k
)
or the expansion by minors of the rst column
det A =
N

j =1
(1)
j 1
a
j 1
det
_
A
j 1
_
where A
j k
is the (N 1)(N 1) matrix obtained from A by deleting the j
th
row and k
th
column.
version: 11/28/2011
Elementary Matrix Theory Chapter & Page: 411
Either of the above formulas gives you the determinant of A. If you are careful with your
signs, you can nd the determinant via an expansion using any row or column. If you are
interested
2
, here is a most general formula for the determinant for the above NN matrix A:
det(A) =
N

i
1
=1
N

i
2
=1

N

i
N
=1
(i
1
, i
2
, . . . , i
N
) a
i
1
1
a
i
2
2
a
i
N
N
where is a function of all N-tuples of the integers from 1 to N satisfying all the following:
1. If any two of the entries in (i
1
, i
2
, . . . , i
N
) are the same, then
(i
1
, i
2
, . . . , i
N
) = 0 .
2. If i
1
= 1 , i
2
= 2 , i
3
= 3 , etc., then
(i
1
, i
2
, i
3
. . . , i
N
) = (1, 2, 3, . . . , N) = 1 .
3. Suppose ( j
1
, j
2
, . . . , j
N
) is obtained from (i
1
, i
2
, . . . , i
N
) by simply switching just two
adjacent entries in (i
1
, i
2
, . . . , i
N
) . In other words, for some integer K with 1 K <
N ,
j
k
=
_

_
i
k
if k = K or k = K + 1
i
K+1
if k = K
i
K
if k = K + 1
.
Then
( j
1
, j
2
, j
3
. . . , j
N
) = (i
1
, i
2
, i
3
. . . , i
N
) .
Properties and Applications of Determinants
Let me remind you of many of the properties and a few of the applications of determinants.
In many ways, you can consider Cramers rule (equation (4.2)) for solving system (4.1a) or,
equivalently, matrix equation (4.1b) as a basic dening equation for the determinant. Cramers
rule tells us that system(4.1a) canbe solved for every choice of the b
j
s if and only if det(A) = 0 .
But the ability to solve that system is equivalent to the ability to solve the matrix equation
Ax = b
for every choice of b, and since this matrix equation is solvable for every choice of b if and
only if A is invertible, we must have that:
An NN matrix A is invertible det(A) = 0 .
This is doubtlessly our favorite test for determining if A
1
exists. It will also probably be the
most important fact about determinants that we will use.
Another property that can be derived fromCramers rule is that, if A and B are both NN
matrices, then
det(AB) = det(A) det(B) . (4.3)
2
If you are not interested, skip to Properties and Applications of Determinants below.
version: 11/28/2011
Elementary Matrix Theory Chapter & Page: 412
(Just consider using Cramers rule to solve (AB)x = b, rewritten as A(Bx) = b.)
Now it is real easy to use the formulas (or even Cramers rule) to verify that
det(I) = det
_
_
_
_
_
_
_
_
1 0 0 0
0 1 0 0
0 0 1 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 0 1
_

_
= 1 .
From this and the observation regarding determinants of products you can easily do the next
exercise.
?

Exercise 4.4: Assume A is invertible, and show that

det
_
A
1
_
=
1
det(A)
.
A few other properties of the determinant are listed here. Assume A is an NN matrix
and is some scalar.
1. det(A) =
N
det(A) .
2. det
_
A

_
= det(A)

.
3. det
_
A
T
_
= det(A) .
4. det
_
A

_
= det(A)

.
?

Exercise 4.5: Convince yourself of the validity of each of the above statements. (For the
rst, det(A) =
N
det(A) , you might rst show that det(I) =
N
, and then consider
rewriting A as IA and taking the determinant of that.)
Also remind yourself of the computational facts regarding determinants listed at the bottom
of page 168 and top of page 169 of Arfken & Weber.
Finally, let me comment on the importance of Cramers rule: It can be viewed as being
very important, theoretically, for the way it relates determinants to to solving of linear systems.
However, as a practical tool for solving these systems, it is quite unimportant. The number of
computations required to nd all the determinants is horrendous, especially if the systemis large.
Other methods, such as the Gauss elimination/rowreduction learned in your undergraduate linear
algebra course (and discussed in example 3.1.1 on page 170 of Arfken & Weber) are much faster
and easier to use.
version: 11/28/2011