CHAPTER 1

Vector Spaces
We saw dierent types of vectors last session with very similar algebraic properties.
Other mathematical objects share these properties, and we will investigate these:
functions,
nite vector spaces,

polynomials,
matrices.

Because they have very similar structures, techniques useful for dealing with one of these
may be useful for others.
1. Spaces of functions
Let I be an interval, for example, [0, 1], and write C(I, R) for the set of all continuous
real-valued functions on I. We say that functions f and g are equal, and we write f = g,
if and only if f (x) = g(x) for all x I. Given functions f and g in C(I, R) and R,
we dene new functions f + g and f in C(I, R) as follows: (f + g)(x) = f (x) + g(x)
for all x I and (f )(x) = f (x) for all x I. We write f for (1)f , that is,
(f )(x) = f (x) for all x in I, and 0 for the zero function, i.e., 0(x) = 0 for all x in I.
Proposition 1. The set C(I, R) of all continuous real-valued functions on the interval
I has the following properties:
(1) for all f, g C(I, R),
f + g C(I, R)
(closure under addition)
(2) for all f C(I, R) and R,
f C(I, R)
(closure under scalar multiplication)
(3) for all f C(I, R),
f +0= 0+f =f
(existence of zero)
(4) for all f in C(I, R),
f + (f ) = (f ) + f = 0
(existence of additive inverses)
(5) for all f, g, h C(I, R),
(f + g) + h = f + (g + h)
(addition is associative)
1

1. VECTOR SPACES

(6) for all f, g C(I, R),

f +g =g+f
(addition is commutative)
(7) for all f C(I, R),
1f = f
(1 is an identity for scalar multiplication)
(8) for all f C(I, R) and all , R,
()f = (f )
(scalar multiplication is associative)
(9) for all f, g C(I, R) and all , R,
( + )f = f + f
(f + g) = f + g
(scalar multiplication distributes over addition).
These are essentially the same properties enjoyed by geometric vectors and algebraic
or coordinate vectors. Actually, functions have more properties: you can multiply them,
dierentiate them, and so on. But many properties of functions just rely on addition and
scalar multiplication. Polynomials behave in a similar way to functionsindeed, they are
special case of functions.
Definition 2. We write Pd (R) and Pd (C) for the sets of all real polynomials and all
complex polynomials of degree at most d.
These are both examples of vector spaces (dened below); the eld of scalars for Pd (R)
is taken to be R, but the eld of scalars for Pd (C) may be taken to be R or C.
We observed last session that matrices can be added and multiplied by scalars in
the same way as vectors. Matrices also have a multiplicative structure, which is not
commutative.
2. Finite vector spaces
Finite vector spaces require nite elds.
Problem 3. Write Z2 for the integers modulo 2. This is the set {0, 1}, with addition
and multiplication dened thus:
0
1

0 1
0 1
1 0
Addition

Show that Z2 is a eld.

0
1

0 1
0 0
0 1

Multiplication

3. VECTOR SPACE AXIOMS

Problem 4. Write Z3 for the integers modulo 3. This is the set {0, 1, 2}, with addition
and multiplication dened thus:
0
1
2

0
0
1
2

1
1
2
0

2
2
0
1

0
1
2

Addition

0
0
0
0

1
0
1
2

2
0
2
1

Multiplication

Show that Z3 is a eld.

Challenge Problem. Draw up similar tables for Zn , the integers modulo n. Show
that Zn is a eld if and only if n is a prime.
In particular Z4 is not a eld.
Challenge Problem. Find rules for adding and multiplying four symbols to get a
eld.
If F is any eld, we may form Fn , the set of vectors of the form
(x1 , x2 , . . . , xn )T ,
where all xi F. This is a vector space over F.
Finite elds are used in coding. For example, 128-bit encoding works as follows: 128
bits of computer information corresponds to a vector with 128 places, each of which is
to Z128
which is one-to-one and onto. Unless
0 or 1. A code is a function from Z128
2
2
we know this function, we cannot decode the message. Constructing and inverting these
functions often involves vector space ideas, such as multiplying by a matrix to encode and
multiplying by its inverse to invert.
3. Vector space axioms
Let F be a eld and V be a set. Suppose that there are operations + : V V V
and : F V V . Then V is called a vector space over F if it satises the following
conditions:
(1) for all v, w V , their sum v + w is dened and is in V
(V is closed under addition)
(2) for all F and v V , their product v is dened and is in V
(V is closed under scalar multiplication)
(3) for all u, v, w V ,
(u + v) + w = u + (v + w)
(addition is associative)
(4) for all u, v, w V ,

u+v =v+u

(addition is commutative)
(5) there exists a vector 0 such that for all v V ,
v+0=v
(there exists a zero vector)

1. VECTOR SPACES

(6) for all v V , there exists a vector v such that

(v) + v = 0
(existence of additive inverses)
(7) for all v V ,

1v = v

(1 is a multiplicative identity)
(8) for all v V and all , F,
()v = (v)
(multiplication is associative)
(9) for all u, v V and all , R,
( + )u = u + u
(u + v) = u + v
(scalar multiplication distributes over addition).
Usually we just write v, rather than v. From these dening properties of a vector
space, called axioms, we can deduce other properties.
Proposition 5. Suppose that V is a vector space over F. Then
(1) for all u V , there is only one z V (the vector 0) such that u + z = u
(uniqueness of the zero vector)
(2) for all u, v, w V ,
if u + v = u + w, then v = w
(cancellation property)
(3) for all u V , there is only one v V such that u + v = 0
(uniqueness of inverses)
(4) for all F,
0 = 0
(5) for all v V ,
0v = 0
(6) for all v V ,
(1)v + v = 0 = v + (1)v
(7) for all v V and F,
if v = 0 then = 0 or v = 0
(8) for all v, w V and all , F,
if v = v and v = 0, then =
if v = w and = 0, then v = w.
Proof. These follow from the vector space axioms. We consider only (7) and (8).
To prove (7), note that if v = 0 and = 0, then v = 1 v = 1 0 = 0.
To prove (8), observe that if v = v, then ( )v = 0, so = or v = 0 by (7).


Similarly, if v = w and = 0, then 1 v = 1 w, so v = w.
Note that (1)v = v.

4. SUBSPACES

4. Subspaces
A subspace of a vector space V over a eld F is a nonempty subset of V which is a
vector space in its own right; in particular, V is a subspace of itself. We are often asked
to decide when a subset is a subspace, and this might require us to check up to ten items.
This is quite tedious. Consequently, the following theorem is convenient.
Theorem 6 (Subspace theorem). If S is a subset of a vector space V over a field F,
then S is a subspace if and only if S has all three of the following properties:
(1) S is not empty
(2) S is closed under addition
(3) S is closed under scalar multiplication.
Further, S is not a subspace if 0
/ S or if any of the above properties fails to hold.
Proof. First, note that vector space axioms (3), (4), (7), (8) and (9) automatically
hold for subsets. Axioms (1) and (2) might not hold, but are ensured by hypotheses (2)
and (3) of this proposition. Finally, 0 = 0v and v = (1)v, so axioms (5) and (6) follow
from hypothesis (3). Thus if all the hypotheses hold, so do all the vector space axioms.
Conversely, if hypothesis (1) is false or if 0
/ S, then vector space axiom (5) cannot
hold; if hypothesis (2) or (3) fails, then axiom (1) or (2) fails.


Problem 7. Show that the set of vectors (x1 , x2 )T in R2 such that
x2 = 2x1 + c
is a subspace if and only if c = 0.
Answer. Suppose that c = 0. Clearly (0, 0)T lies in S, so that S is not empty. If
x, y S, then x2 = 2x1 and y2 = 2y1, so x2 + y2 = 2(x1 + y1 ), and x + y S. Finally, if
x S and R, then x2 = 2x1 , so x2 = 2x1 , and x S.
x2
x1 = 2x2

x1

x1 = 2x2 4

1. VECTOR SPACES

Suppose that c = 0. Then (0, 0)T

/ S. Therefore S does not contain the zero vector,
and so S fails to satisfy the vector space axiom on the existence of the zero vector; thus
S is not a subspace.

Problem 8. Show that the set of vectors (x1 , x2 , x3 )T in R3 such that
x1 + 2x2 + 3x3 = d
is a subspace if and only if d = 0.
Problem 9. Show that the unit circle in R2
{(x1 , x2 )T R2 : x21 + x22 = 1}
is not a subspace.
Problem 10. Show that the sets of vectors H1 and H2 in R2 given by
H1 = {(x, y) : y x}
H2 = {(x, y) : y x}
are not subspaces, but H1 H2 and H1 H2 are subspaces.

H1

H2

Problem 11. Suppose that A Mm,n (R), and dene S = {x Rn : Ax = b}. Show
that S is a subspace if and only if b = 0.
Answer. Suppose rst that b = 0. Then A0 = b, so 0
/ S. Consequently, S is not
a subspace.
Now suppose that b = 0. Then A0 = b, so that 0 S, and S is not empty.
Next, suppose that x, y S. Then Ax = 0 and Ay = 0, so
A(x + y) = Ax + Ay = 0,
and x + y S, i.e., S is closed under addition.
Finally, suppose that R and x S. Then Ax = 0, so Ax = 0, and A(x) = 0.
Hence x S, i.e., S is closed under scalar multiplication.

By the subspace theorem, S is a subspace of Rn .

5. LINEAR COMBINATIONS AND SPANS

Problem 12. Suppose A Mm,n (R). Show that the set of b for which Ax = b has
at least one solution is a subspace.
Problem 13. Let S denote the set {p P3 (C) : p(1) = 0}. Show that S is a subspace
of P3 (C).
Answer. First, 0 S, so S is not empty.
Next, if p, q S, then p(1) = 0 and q(1) = 0, so
(p + q)(1) = p(1) + q(1) = 0,
and p + q S, i.e., S is closed under addition.
Further, if C and p S, then (p)(1) = p(1) = 0, so p S, i.e., S is closed
under scalar multiplication.
By the subspace theorem, S is a subspace of P3 (C).

5. Linear combinations and spans
Definition 14. A linear combination of vectors v 1 , v 2 , . . . , v n is a vector of the form
1 v 1 + 2 v 2 + + n v n ,
where 1 , 2 , . . . , n are scalars.
Definition 15. The span of the vectors v 1 , v 2 , . . . , v n is the set of all linear combinations of v 1 , v 2 , . . . , v n : it is written span{v1 , v 2 , . . . , v n }.
In a vector space, all nite sums of the form
1 v 1 + 2 v 2 + + n v n
are well-defined, i.e., have an unambiguous meaning. We could put brackets round the
sum in many dierent ways, but it can be shown that all give the same result.
Challenge Problem. In how many dierent ways can we bracket the sum of n
vectors? Prove (by induction) that they are all equivalent.
Theorem 16. The smallest subspace of a vector space which contains the vectors v 1 ,
v 2 , . . . , v n is their span.
This theorem says two things: the span is a subspace, and it is the smallest subspace.
Proof. By the subspace theorem, to show that span{v 1 , v 2 , . . . , v n } is a subspace,
we must show it is nonempty and closed under addition and scalar multiplication.
First, if 1 = 2 = = n = 0, then
1 v 1 + 2 v 2 + + n v n = 0,
so that 0 span{v 1 , v 2 , . . . , v n }, and the span is not empty.
Next, if w, w
span{v1 , . . . , v n }, then there exist scalars 1 , . . . , n and
1 , . . . ,
n
such that
w = 1 v 1 + 2 v 2 + + n v n
w
=
1 v 1 +
2 v 2 + +
n v n .

1. VECTOR SPACES

Then, adding these, we see that

w + w
= (1 +
1 )v 1 + + (n +
n )v n ,
so w + w
span{v 1 , v 2 , . . . , v n }.
Finally, suppose that w span{v 1 , . . . , v n } and is a scalar. Then there exist scalars
1 , . . . , n such that
w = 1 v 1 + 2 v 2 + + n v n ,
so that
w = 1 v 1 + 2 v 2 + + n v n ,
and
w span{v 1 , v 2 , . . . , v n }.
By the Subspace Theorem, span{v 1 , v 2 , . . . , v n } is a subspace.
To show that span{v 1 , v 2 , . . . , v n } is the smallest subspace containing v 1 , v 2 , . . . , v n ,
suppose that W is any subspace such that
v 1 , v 2 , . . . , v n W.
We need to show that span{v 1 , . . . , v n } W , that is, every vector in span{v 1 , . . . , v n } is
also in W . Since W is closed under scalar multiplication and addition, given any scalars
1 , . . . , n ,
1 v 1 + 2 v 2 + + n v n W.


Thus span{v 1 , v 2 , . . . , v n } W , as required.
We say that a subset {v 1 , v 2 , . . . , v n } of a vector space V is a spanning set for V , or
just spans V , if
span{v 1 , v2 , . . . , vn } = V.
Problem 17. Show that {(1, 0)T , (0, 1)T } is a spanning set for R2 .
Answer. Take (x, y)T in R2 . Then
(x, y)T = x(1, 0)T + y(0, 1)T .
Therefore every vector in R2 is a linear combination of the vectors (1, 0)T and (0, 1)T ;
hence they span R2 .

Problem 18. Show that {(1, 0, 0)T , (0, 1, 0)T , (0, 0, 1)T } spans R3 .
Problem 19. Do the vectors (0, 1, 1)T , (2, 1, 3)T and (1, 2, 3)T span R3 ? If not,
describe their span.
Answer. Suppose (c1 , c2 , c3 )T R3 . Then
(c1 , c2 , c3 )T span{(0, 1, 1)T , (2, 1, 3)T , (1, 2, 3)T }
if and only if there exist scalars 1 , 2 , 3 R such that



0
2
1
c1
1 1 + 2 1 + 3 2 = c2 .
c3
1
3
3

5. LINEAR COMBINATIONS AND SPANS

This is equivalent to the system

01 + 22 + 13 = c1
11 + 12 + 23 = c2
11 32 33 = c3 ,
which in turn can be represented by the augmented matrix

0
2
1
c1
1
1
2
c2 .
1 3 3 c3
Note that the columns of the augmented matrix are the vectors in question.
We solve this by row-reductions:
R1 R2 , R3 = R3 + R1

1 1
2
0 2
1
0 2 1
R3 = R2 + R3

1 1 2
0 2 1
0 0 0

c2
c1
c2 + c3

c2
.
c1
c1 + c2 + c3

This has a solution if and only if c1 + c2 + c3 = 0. This means that

(c1 , c2 , c3 )T span{(0, 1, 1)T , (2, 1, 3)T , (1, 2, 3)T }
if and only if c1 + c2 + c3 = 0. Since there are vectors in R3 , such as (1, 0, 0)T , which do
not satisfy this,
span{(0, 1, 1)T , (2, 1, 3)T , (1, 2, 3)T }
is a subspace of R3 dierent from R3 ; we say that it is a proper subspace of R3 .

Recall that we say that the real or complex polynomials p(x) and q(x) are the same
when the coecients are the same, i.e., when the values p(x) and q(x) are the same for
all x. In this case we often just write p = q. We call x a dummy variable. The same
convention applies to functions.
Problem 20. Suppose that
p1 (t) = t3 3t
p2 (t) = t3 t2 t
p3 (t) = t2 t 1
q(t) = t + 1.
Is it true that q span{p1 , p2 , p3 }?
Answer. If this were true, then there would exist 1 , 2 , 3 R such that
q = 1 p1 + 2 p2 + 3 p3 ,

10

1. VECTOR SPACES

i.e.,
t + 1 = 1 (t3 3t) + 2 (t3 t2 t) + 3 (t2 t 1)
= (1 + 2 )t3 + (2 + 3 )t2 (31 + 2 + 3 )t 3 .
So, we would have
1 + 2
2 + 3
31 + 2 + 3
3

=0
=0
= 1
= 1.

Back-substituting, 3 = 1 and 2 = 1, but then 1 = 1 and 31 = 1, which is

impossible. Since this system has no solutions, q
/ span{p1 , p2 , p3 }.

Problem 21. Is sin(2x + 3) in span{sin(2x), cos(2x)}?
Answer. By the well-known addition formula,
sin(2x + 3) = sin(2x) cos(3) + cos(2x) sin(3)
= cos(3) sin(2x) + sin(3) cos(2x),
and sin(3) and cos(3) are scalars, so sin(2x + 3) span{(sin(2x), cos(2x)}.

Problem 22. Is sin(2x) in span{sin x, cos x}?

Answer. Suppose that there exist numbers 1 and 2 such that
sin(2x) = 1 sin x + 2 cos x
for all x. Then, taking x = 0 and /2, we see that
0 = 0 + 2
0 = 1 + 0
so we must have 1 = 2 = 0. But now taking x = /4, we get 1 = 0 + 0 which is
impossible. Thus sin 2x
/ span{sin x, cos x}.

To summarise the last two problems, the function sin(2x + 3) is a linear combination
of the functions sin(2x) and cos(2x), because the formula
sin(2x + 3) = cos 3 sin 2x + sin 3 cos 2x
is true for all x, and the function sin(2x) is not a linear combination of the functions
sin(x) and cos(x), because we cannot nd numbers and such that
sin(2x) = sin(x) + cos(x)
for all x, though for a xed x this is possible. In these examples, it is hard to nd an
x-free way to write things.

5. LINEAR COMBINATIONS AND SPANS

11

Proposition 23. Suppose that a1 , a2 , . . . , an Rm . Let A be the matrix with

columns a1 , a2 , . . . , an . Then b span{a1 , a2 , . . . , an } if and only if there exists x Rn
such that b = Ax.
Proof. Observe that
Ax = = x1 a1 + + xn an .

The result follows.

We dene the column space of a matrix A to be the subspace spanned by its columns.
Problem 24. Is (1, 0, 5)T in the column space of the matrix

1 1
2 1 ?
3 1
Answer. It is equivalent to ask whether the system represented by the augmented
matrix

1
1 1
2 1
0
5
3 1
has a solution. We proceed to row-reduce the augmented matrix.
R2 = R2 2R1 , R3 = R3 3R1

R3 = R3 4R2

1 1
0 1
0 4

1
2
8

1 1
0 1
0 0

1
2 .
0

There is a (unique) solution, so

1
1
1
0 span 2 , 1 .

5
3
1

Problem 25. Let S denote span{x3 x + 1, x3 x2 1, x2 x + 2} in P3 (R). Which
polynomials q(x) belong to S?
Answer. Suppose that q is a linear combination of these polynomials. Then q is of
degree at most 3. Write q(x) = a3 x3 + a2 x2 + a1 x + a0 . Then
q(x) = 1 (x3 x + 1) + 2 (x3 x2 1) + 3 (x2 x + 2)
= (1 + 2 )x3 + (3 2 )x2 (1 + 3 )x + (1 2 + 23 )

12

1. VECTOR SPACES

precisely when
1 + 2
3 2
1 3
1 2 + 23

= a3
= a2
= a1
= a0 .

This system of equations is represented by the augmented matrix

a3
1
1
0
0 1 1
a2

1 0 1 a1 .
a0
1 1 2
We reduce this to row-echelon form:
R3 = R3 + R1 , R4 = R4 R1

1 1
0 1

0 1
0 2
R3 = R3 + R2 , R4 = R4 2R2

1 1
0 1

0 0
0 0

0
1
1
2
0
1
0
0

a3
a2

a1 + a3
a0 a3

a3

a2
.
a1 + a2 + a3
a0 2a2 a3

Therefore
a3 x3 + a2 x2 + a1 x + a0 span{x3 x 1, x3 x2 1, x2 x + 2}
if and only if a1 + a2 + a3 = 0 and a0 2a2 a3 = 0.

Problem 26. What is the dierence between

span{(3, 0, 1)T , (1, 1, 1)T , (0, 3, 2)T } and

span{(3, 0, 1)T , (1, 1, 1)T }?

Answer. These are two subspaces of R3 . The rst subspace clearly contains every
vector the second subspace. The question is whether the rst subspace is larger.
Let b = (b1 , b2 , b3 )T . Then
b span{(3, 0, 1)T , (1, 1, 1)T , (0, 3, 2)T }
if and only if the system represented by the augmented matrix

b1
3
1
0
0
1 3 b2
b3
1 1 2
has a solution, which is when b1 + 2b2 + 3b3 = 0.

6. LINEAR DEPENDENCE

13

Similarly, b span{(3, 0, 1)T , (1, 1, 1)T } if and only if b1 + 2b2 + 3b3 = 0. So the
two spans are the same.

The answer to the question why the extra vector does not change the span will be our
next concern.
6. Linear Dependence
Definition 27. Vectors v 1 , . . . , v n are linearly independent if the only possible choice
of scalars 1 , . . . , n for which
1 v 1 + + n v n = 0

(1)

is 1 = = n = 0. They are linearly dependent if there exists a choice of 1 , . . . , n ,

not all 0, so that (1) holds, that is, if they are not linearly independent.
For example, the vectors (1, 0)T and (2, 0)T in R2 are linearly dependent, since
2(1, 0)T + (1)(2, 0)T = (0, 0)T .
The vectors (2, 0)T and (1, 1)T are linearly independent: indeed, if
(2, 0)T + (1, 1)T = (0, 0)T ,
then 2 + = 0 and = 0, so
= = 0.
Things get trickier as the number of vectors increases.
Problem 28. Are the vectors (1, 0, 1)T , (1, 2, 0)T , and (0, 2, 1)T linearly dependent
or independent?
Answer. Suppose that



1
1
0
0

1 0 + 2 2 + 3 2
= 0 .
1
0
1
0
Equivalently,
11 + 12 + 03 = 0
01 + 22 + 23 = 0
11 + 02 13 = 0.
This can be represented by the augmented matrix

1 1 0
0
0 2 2
0 .
1 0 1 0
We row-reduce this to

1 1 0
0 2 2
0 0 0

0
0 .
0

(2)

14

1. VECTOR SPACES

Thus the original system (2) has innitely many solutions, and in particular has nonzero
solutions, e.g., 1 = 1, 2 = 1, 3 = 1. Since



1
1
0
0
1 0 1 2 + 1 2 = 0 ,
1
0
1
0


the vectors are linearly dependent.

Problem 29. Are the polynomials
p1 (t) = 1 + t,
2

p3 (t) = 1 + t ,

p2 (t) = 1 t
p4 (t) = 1 t3

linearly dependent or independent?

Answer. Suppose that
1 p1 + 2 p2 + 3 p3 + 4 p4 = 0.
Then
1 (1 + x) + 2 (1 x) + 3 (1 + x2 ) + 4 (1 x3 ) = 0
for all x, i.e.,
(1 + 2 + 3 + 4 ) + (1 2 )x + 3 x2 4 x3 = 0
for all x, so
1 + 2 + 3 + 4
1 2
3
4

=0
=0
=0
= 0.

Solving, the only solution is

1 = 2 = 3 = 4 = 0,
so the polynomials p1 , p2 , p3 and p4 are linearly independent.

Note that the rows of the system of equations correspond to the x0 terms, the x1
terms, the x2 terms and the x3 terms. This is not an accidentas far as questions of
spans and linear dependence are concerned, the polynomial a0 + a1 x + a2 x2 + + an xn
behaves like the vector (a0 , a1 , a2 , . . . , an )T .
Problem 30. Are the functions 1, ex and e2x linearly independent?
Answer. Suppose that

(3)
1 + 2 ex + 3 e2x = 0
x
2x
for all x. If this implies that 1 = 2 = 3 = 0, then 1, e and e are linearly independent.
One way to show this is via limits: if (3) holds, then
lim (1 + 2 ex + 3 e2x ) = lim 0,

6. LINEAR DEPENDENCE

so 1 = 0. Now

15

2 ex + 3 e2x = 0

so dividing by ex ,
2 + 3 ex = 0.
Another limit argument shows that 2 = 0, and then 3 ex = 0, so 3 = 0. Thus 1, ex and

e2x are linearly independent.
Challenge Problem. Suppose that 1 < 2 < < n . Show that e1 x , e2 x , . . . ,
en x are linearly independent functions.
Challenge Problem. Suppose that 0 < 1 < 2 < < n . Show that sin 1 x,
sin 2 x, . . . , sin n x are linearly independent functions.
Theorem 31. Suppose that v 1 , . . . , v n are vectors, and that v span{v 1 , . . . , vn }.
If v 1 , . . . , v n are linearly independent, there is only one choice of scalars 1 , . . . , n such
that
v = 1 v 1 + + n v n .
(4)
If v 1 , . . . , v n are linearly dependent, there is more than one choice of scalars 1 , . . . , n
such that (4) holds.
Proof. Suppose that the vectors v 1 , . . . , v n are linearly independent. Because v is
in span{v 1 , . . . , v n }, there exist 1 , . . . , n so that
v = 1 v 1 + + n v n .
Suppose that 1 , . . . , n are scalars (possibly dierent from 1 , . . . , n ) so that
v = 1 v 1 + + n v n .
Subtracting,
0 = (1 1 )v 1 + + (n n )v n .
By linear independence, 1 1 = = n n = 0, so i = i , and there is only one
representation of v in terms of the v i .
Suppose now that v 1 , . . . , v n are linearly dependent. Then there exist 1 , . . . , n , not
all zero, so
1 v 1 + + n v n = 0.
Since v span{v 1 , . . . v n }, there exist 1 , . . . , n so that
v = 1 v 1 + + n v n .
Adding,
v = (1 + 1 )v 1 + + (n + n )v n .
Since not all i are zero, this is a dierent representation of v in terms of the v i .

Theorem 32. The vectors v 1 , . . . , v n are linearly dependent if and only if at least
one of them may be written as a linear combination of the others.

16

1. VECTOR SPACES

Proof. Suppose that v 1 , . . . , v n are linearly dependent. Then there exist 1 , . . . ,

n , not all zero (say i = 0) so that
1 v 1 + + i v i + + n v n = 0.
Rearranging to make v i the subject,
1
i1
i+1
n
v1
v i1
v i+1 v n ,
i
i
i
i
so v i is a linear combination of the other vectors, i.e., v i lies in their span.
Conversely, suppose that v i may be written as a linear combination of the other
vectors, i.e., for some scalars 1 , . . . , i1 , i+1 , . . . , n , we have
vi =

v i = 1 v 1 + + i1 v i1 + i+1 v i+1 + + n v n .
Put i = 1. Then
1 v 1 + + i1 v i1 + i v i + i+1 v i+1 + + n v n = 0,
i.e., v 1 , . . . , v n are linearly dependent.

There are several other results whose proofs are similar.

Theorem 33. Suppose that v 0 , v 1 , . . . , v n are vectors. Then
span{v 0 , v 1 , . . . , v n } = span{v 1 , . . . , v n }
if and only if v 0 span{v 1 , . . . , v n }.
Proof. Suppose rst that span{v 0 , v 1 , . . . , v n } = span{v 1 , . . . , v n }. Because v 0 is in
span{v0 , v 1 , . . . , v n }, it follows that
v 0 span{v 1 , . . . , v n }.
Conversely, suppose that
v 0 span{v 1 , . . . , v n };
then there exist 1 , . . . , n so that
v 0 = 1 v 1 + + n v n .
Now, if w span{v 0 , v 1 , . . . , v n }, then there are scalars 0 , 1 , . . . , n so that
w = 0 v 0 + 1 v 1 + + n v n
= 0 (1 v 1 + + n v n ) + 1 v 1 + + n v n
= (0 1 + 1 )v 1 + + (0 n + n )v n ,
and so w span1 , . . . , vn }, i.e.,
span{v 0 , v1 , . . . , v n } span{v 1 , . . . , v n }.
Since the opposite inclusion
span{v 0 , v 1 , . . . , v n } span{v 1 , . . . , vn }
holds (by denition), the two spans coincide.

6. LINEAR DEPENDENCE

17

Problem 34. Show that the vectors

v 1 = (1, 2, 0, 3)T ,

v 2 = (2, 1, 0, 3)T ,

v 3 = (0, 1, 1, 0)T ,

v 4 = (1, 1, 1, 1)T

are linearly dependent. Find a proper subset S of {v 1 , v 2 , v 3 , v4 } so that span(S) is the

same as span{v 1 , v 2 , v3 , v 4 }.
Answer. Consider the expression
1 v 1 + 2 v 2 + 3 v 3 + 4 v 4 = 0.
This gives rise to a system of equations which is represented by the augmented matrix

1 2 0 4 0
2 1 1 1
0
,

0 0 1 0
0
3 3 0 3 0
which we row-reduce to

1 2 0 4
0 3 1 9

0 0 1 0
0 0 0 0

0
0
.
0
0

Since C4 is a nonleading column, the general solution is 4 = , 3 = 0, 2 = 3,

1 = 2. Thus
2v 1 3v 2 v 4 = 0.
In particular,
v 4 span{v 1 , v 2 , v 3 }
so that
span{v 1 , v 2 , v 3 , v 4 } = span{v 1 , v2 , v 3 }.
However, we also have
v 1 span{v 2 , v 3 , v4 },
so that
span{v 1 , v 2 , v 3 , v 4 } = span{v 2 , v3 , v 4 }.
and
v 2 span{v 1 , v 3 , v 4 }
so that
span{v 1 , v 2 , v 3 , v 4 } = span{v 1 , v3 , v 4 }.

Note that it is not true that v 3 span{v 1 , v2 , v 4 }. In fact, the position of the zeros
in the augmented matrix tell us that C4 is a nonleading column, and this translates into
the fact that v 4 is a linear combination of v 1 , v 2 and v 3 , but without explicitly nding
the solution, we cannot tell whether v 3 span{v 1 , v 2 , v 4 }, or v 2 span{v 1 , v3 , v 4 }. In
practical terms, this means that the natural vector to choose to eliminate is v 4 , because
we can see immediately that this is a linear combination of the others.

18

1. VECTOR SPACES

The preceding example is a particular case of the following theorem, whose proof
follows from the previous results.
Theorem 35. If S is a linearly independent set of vectors, then for any proper subset
S
of S,
span(S
) span(S).
However, if S is a linearly dependent set of vectors, then there is at least one proper subset
S
of S so that
span(S
) = span(S).
For example, consider the vectors (0, 0)T and (1, 0)T . These are linearly dependent
because
2(0, 0)T + 0(1, 0)T = (0, 0)T .
Also, span{(0, 0)T , (1, 0)T } = span{(1, 0)T }, but span{(0, 0)T , (1, 0)T } = span{(0, 0)T }.
For later use, we need one more theorem.
Theorem 36. Suppose that v 1 , . . . , v n are linearly independent vectors.
If v n+1 span{v 1 , . . . , v n }, then v 1 , . . . , v n , v n+1 are linearly dependent.
If v n+1
/ span{v 1 , . . . , v n }, then v 1 , . . . , v n , v n+1 are linearly independent.
Proof. The rst half is already proved. Suppose that v 1 , . . . , v n are linearly independent and that v n+1
/ span{v 1 , . . . , v n }. If 1 , . . . , n+1 are scalars so that
1 v 1 + + n v n + n+1 v n+1 = 0,
then n+1 = 0, for otherwise we could make v n+1 the subject, and this would show that
v n+1 span{v 1 , . . . , v n }, which is not allowed. But now
1 v 1 + + n v n = 0,
so 1 = = n = 0 since v 1 , . . . , v n are linearly independent.
Since we have now shown that 1 = = n = n+1 = 0, it follows that v 1 , . . . , v n+1
are linearly independent, as required.


7. Bases and dimensions
A basis (plural bases) for a vector space V is a linearly independent spanning set
for V . The number of elements in a basis for V does not depend on the basis. This
number is called the dimension of V , and is written dim(V ). This result is found by
comparing the number of elements in spanning sets and in linearly independent sets.
Theorem 37. If {v 1 , . . . , v m } is a set of linearly independent vectors in the vector
space V , and {w1 , . . . , w n } is a spanning set for V , then m n.
Proof. We suppose that {v 1 , . . . , v m } is linearly independent and that {w1 , . . . , w n }
is a spanning set.

7. BASES AND DIMENSIONS

19

Since {w 1 , . . . , wn } spans V , there are scalars aij (where i = 1, . . . , m and j = 1, . . . , n)

so that
v 1 = a11 w1 + + a1n wn
v 2 = a21 w1 + + a2n wn
(5)
...
v m = am1 w1 + + amn w n .
Suppose that 1 , . . . , m solve the equations
a11 1 + a21 2 + + am1 m = 0
a12 1 + a22 2 + + am2 m = 0
...
a1n 1 + a2n 2 + + amn m = 0.

(6)

(Note that the aij have been transposed, going from the array (5) to the array (6)). Then
(a11 1 + a21 2 + + am1 m )w 1
+ (a12 1 + a22 2 + + am2 m )w 2
+
...
+ (a1n 1 + a2n 2 + + amn m )wn = 0,
or equivalently,
1 (a11 w 1 + a12 w 2 + + a1n w n )
+ 2 (a21 w 1 + a22 w 2 + + a2n w n )
+
...
+ m (am1 w1 + am2 w2 + + amn w n ) = 0,
i.e., 1 v 1 + 2 v 2 + + m v m = 0, and so 1 = 2 = = m = 0, as the vectors v 1 , v 2 ,
. . . , v m are linearly independent.
We have just shown that the only solution to the equations (6) is 1 = 2 = =
m = 0. This implies that, if the augmented matrix corresponding to these equations,
namely

a11 a21 am1 0

a12 a22 am2 0
,
.
..
..

..
.
.
a1n a2n amn 0
is row-reduced, then all columns are leading columns. This in turn implies that m < n,
as required.


Corollary 38. If {v 1 , . . . , v m } and {w1 , . . . , wn } are both bases of the vector space
V , then m = n.
Proof. Since {v 1 , . . . , v m } spans V and {w1 , . . . , w n } is linearly independent, m n.
Conversely, since {v 1 , . . . , vm } is linearly independent and {w 1 , . . . , wn } spans V , n m.
Combining these inequalities, m = n.

20

1. VECTOR SPACES

Definition 39. The dimension of a vector space V is the number of elements in a

basis for V .
Problem 40. Show that the dimension of Rn is n.
Answer. The vectors e1 , e2 , . . . , en form a basis. Indeed, for any x Rn ,
x = x1 e1 + + xn en ,
so they span, and they are linear independent because if x1 e1 + + xn en = 0, then
x1 = x2 = = xn = 0.

In particular, the dimension of a line is 1 and of a plane is 2.
Problem 41. Let V be the set of all x in R4 such that
2x1 + 3x2 x3 = 0.
Show that V is a subspace of R4 and nd dim(V ).
Answer. We do not show here that V is a subspace. Consider the vectors



1
0
0
0
1
0

v2 =
v3 =
v1 =
2
3
0 .
0
0
1
If 1 v 1 + 2 v 2 + 3 v 3 = 0, then

1
0

= 0 ,
21 + 32 0
3
0

so 1 = 2 = 3 = 0, i.e., the vectors v 1 , v 2 and v 3 are linearly independent. Conversely,

if x V , then x3 = 2x1 + 3x2 so that

x1
x1
1
0
0
x2

x2
=
= x1 0 + x2 1 + x4 0 ,
x3 2x1 + 3x2
2
3
0
x4
x4
0
0
1
i.e.,
x = x1 v 1 + x2 v 2 + x4 v 3 ,
and {v 1 , v 2 , v 3 } spans V . Thus {v 1 , v 2 , v 3 } is a basis, and so dim(V ) = 3.

The obvious question to ask about this solution is where the vectors v 1 , v 2 and v 3
came from. We shall see the answer to this later.
Suppose that V is a vector space of dimension n, and that {v 1 , . . . , v m } V . If
{v 1 , . . . , v m } spans V , then m n. If m = n, then {v 1 , . . . , v m } is a basis, while if
m > n, then it is possible to remove m n of the vectors to get a basis. Similarly, if
{v 1 , . . . , v m } is linearly independent, then m n. If m = n, then {v 1 , . . . , v m } is a basis,
while if m < n, then it is possible to add n m vectors to get a basis.

7. BASES AND DIMENSIONS

21

Theorem 42. Suppose that {v 1 , . . . , v n } is a spanning set for V . Then there is a

subset of {v 1 , . . . , v n } which is a basis of V .
Proof. First, remove any v i which is zero. Having done this, and relabelled the
vectors if necessary, we may assume each v i is nonzero.
Dene subsets S1 , S2 , . . . , Sn of {v 1 , . . . , v n } as follows:
S1 = {v 1 }

if v 2 span(S1 )
S1
S2 =
/ span(S1 )
S1 {v 2 } if v 2

Sn =

...
if v n span(Sn1 )
Sn1
.
Sn1 {v n } if v n
/ span(Sn1 )

Using Theorem 33, we can show by induction that

span{v 1 , . . . , v i } = span(Si )
for all i. In particular, when i = n, we see that Sn is a spanning set. On the other hand,
it is linearly independent by construction, so is a basis.


Problem 43. Suppose that


2
2
1
2

v1 =
3 , v 2 = 2 ,
0
1

0
1

v3 =
1 ,
0

2
4

v4 =
2 ,
2

0
1

v5 =
2 .
2

Show that {v 1 , v 2 , . . . , v 5 } spans R4 , and nd a subset which is a basis.

Answer. Consider a = (a1 , a2 , a3 , a4 )T in R4 . We can write a as 1 v 1 + + 5 v 5
if and only if that the system represented by the augmented matrix

2 2 0 2 0 a1
1 2 1 4 1 a2

3 2 1 2 2 a3
0 1 0 2 2 a4
has a solution. Reducing

1
0

0
0

to row-echelon form, we obtain the augmented matrix

2 1
4 1
a2

1 0
2 2
a4
.
0 2 2 2
a1 2a2 + 2a4
0 0
0 5 a2 + 2a4 a1 + a3

We can deduce two things from this.

First, for any a, there is a solution. Therefore the vectors v 1 , v 2 , . . . , v 5 span R4 .
Next, the solution is not unique, as the column corresponding to v 4 is not a leading
column. If we take a = 0, then there is a solution for which the coecient of v 4 is nonzero,
so we can write v 4 as a linear combination of the other vectors. This means that we can

22

1. VECTOR SPACES

eliminate v 4 without changing the span. As the span is R4 , which is 4-dimensional, we

need at least 4 vectors in a basis. Consequently {v 1 , v 2 , v 3 , v5 } is a basis.

We can say more about this problem: in fact we can show that v 1 2v 2 v 3 + v 4 = 0,
and we can make any one of v 1 , . . . , v 4 the subject of this formula, for instance
v 4 = 2v 2 + v 3 v 1 .
This means that we could omit any one of these vectors to get a basis. If we had omitted
v 4 from the beginning, we would have got the system

2 2 0 0 a1
1 2 1 1 a2

3 2 1 2 a3 ,
0 1 0 2 a4
which row-reduces to

1
0

0
0

2 1 1
1 0 2
0 2 2
0 0 5

a2

a4
.
a1 2a2 + 2a4
a1 a2 + a3 + 2a4

From this, we see that it is possible to write all vectors a as a linear combination of v 1 , v 2 ,
v 3 and v 5 uniquely. The uniqueness implies that these vectors are linearly independent,
by Theorem 31. If we omitted a dierent vector, we cannot see immediately what we
would end up with in the row-reduction process.
The method of solution of Problem 43 works for coordinate vectors in general. Given
vectors v 1 , . . . , v n in Rm , to get a linearly independent set, we write the vectors as a
matrix, row-reduce it, and remove the vectors corresponding to nonleading columns.
This method also works for polynomials: as we have seen, they behave like coordinate
vectors. But for other kinds of vectors, like functions, we cannot use this method. The
point of Theorem 42 is that it works for all vector spacesall we need is a way to decide
whether vectors are in the span of other vectors.
How do we go about enlarging sets of linearly independent vectors to get bases?
Theorem 44. Suppose that {w1 , . . . , wm } is a spanning set for a vector space V , and
that {v 1 , . . . , v k } is a linearly independent set. Then we can find vectors v k+1 , . . . , v n so
that {v 1 , . . . , v k , v k+1 , . . . , v n } is a basis of V .
Proof. Consider the set {v 1 , . . . , v k , w1 , . . . , wm }. Since it contains a spanning set,
it is a spanning set. We apply the algorithm of Theorem 42 to it. When we do this, none
of the vectors v 1 , . . . , v k will be discarded, because they are linearly independent. Indeed,
the algorithm adds vectors to the set until it reaches a vector which depends linearly on
the previously chosen vectors, and the rst vector which might depend on the previously
chosen vectors is w 1 .

7. BASES AND DIMENSIONS

23

We can formalise this in another algorithm. Let S0 = {v 1 , . . . , v k }. We construct

linearly independent sets S1 , S2 , . . . of vectors as follows:

S0 if w 1 span(S0 )
S1 =
/ span(S0 )
S0 {w1 } if w 1

Sm1 if w m span(Sm1 )
Sm =
/ span(Sm1 ).
Sm1 {wm } if wm
The proof that this works is similar to the proof of Theorem 42.

Problem 45. Show that the set {(1, 2, 0, 1)T , (1, 3, 0, 1)T } is linearly independent in
R and that


(1, 2, 0, 1)T , (1, 3, 0, 1)T , (1, 0, 0, 0)T , (0, 1, 0, 0)T , (0, 0, 1, 0)T , (0, 0, 0, 1)T
4

spans R4 . Find a subset of this last set which is a basis.

Answer. We rst consider the augmented matrix:

1 1 0
2 3 0

0 0 0 .
1 1 0
This row-reduces to

1
0

0
0

1
1
0
0

0
0
.
0
0

The vectors are linearly independent, because both columns are leading columns, but do
not span, because there are some all-zero rows.
For the second part, we consider the augmented matrix:

1 1 1 0 0 0 a1
2 3 0 1 0 0 a2

0 0 0 0 1 0 a3
1 1 0 0 0 1 a4
and row-reduce it:

a1
1 1 1 0 0 0
0 1 2 1 0 0 a2 2a1

0 0 0 0 1 0
a3
a4 a1
0 0 1 0 0 1

1 1 1 0 0 0
a1
0 1 2 1 0 0 a2 2a1

0 0 1 0 0 1
a4 a1
0 0 0 0 1 0
a3

24

1. VECTOR SPACES

This system is consistent, so has solutions for any (a1 , . . . , a4 )T in R4 , and hence the set of
vectors spans R4 . Alternatively, we could argue that the set of standard basis vectors {e1 ,
. . . , e4 } spans R4 , and so any collection of vectors containing these is certainly spanning.
Finally, we observe that columns 4 and 6 in the reduced matrix from the previous part
are non-leading. We omit the corresponding vectors. Then


(1, 2, 0, 1)T , (1, 3, 0, 1)T , (1, 0, 0, 0)T , (0, 0, 1, 0)T
is a basis for R4 , containing the initial vectors (1, 2, 0, 1)T and (1, 3, 0, 1)T .

Problem 46. Suppose that {v i : i = 1, . . . , m} is an orthonormal set in Rn , i.e.,

1 if i = j
vi vj =
0 if i = j.
(i) Show that {v i : i = 1, . . . , m} is linearly independent.
(ii) Show that, if there exists a nonzero vector v such that v v i = 0 for i = 1, . . . ,
m, then we can enlarge {v i : i = 1, . . . , m} to a bigger orthonormal set.
Challenge Problem. This is part (iii) of the previous example. Show that, if the
only vector v for which v v i = 0 when i = 1, . . . , m is 0, then {vi : i = 1, . . . , m} is a
basis.
Last, but not least, {0} satises all the denitions of a vector space, but in a trivial
way. All the discussion above about spans, linear dependence, and so on, is meaningless
and silly. We dene dim{0} = 0. This is a convenient convention.
8. Coordinates with respect to a basis
Let B = {v 1 , . . . , v n } be a basis for a vector space V . Any vector v in V may be
written in the form
v = 1 v 1 + + n v n ,
for appropriate scalars 1 , . . . , n , which are uniquely determined by v. The numbers 1 ,
. . . , n are called the coordinates of v relative to the basis B, and written [v]B ; this is
a column vector. The order of the scalars 1 , . . . , n is determined by the order of the
vectors v 1 , . . . , v n . The order of the vectors is important, because it determines the order
of the coecients, and we sometimes refer to an ordered basis to emphasize this.
For a xed vector v, choosing dierent bases leads to dierent coordinate vectors. It is
natural to deal with dierent bases. For example, in describing three-dimensional space,
we may want an axis pointing north, one up, and one east. Or we may want axes
with one in the direction of the axis of rotation of the earth. Or axes lying in the plane
of the solar system. Or axes in the plane of our galaxy. Choosing coordinates means
choosing a basis, and then writing vectors relative to that basis. We will investigate this
further later in this course.