Professional Documents
Culture Documents
linear spaces
complete
normed spaces
Banach space
inner product spaces
Hilbert space
metric spaces
If a normed space U contains a sequence {en } with the property that for
every u U there is a unique sequence of
scalars {n } such that ku(1 e1 +2 e2 +
+N eN k 0 as N , then {en } is
called a Schauder basis for U. A Schauder
basis is different from a Hamel basis in
that a countably infinite number of basis vectors and scalar coefficients may be
needed to uniquely represent a given vector.
X
h=
n en
n=1
with norm
Z
kf kp := sup
f (rei )p d
1/p
0r<1
dim
D(T)
=
N
<
=
dim
R(T)
khk =
|hh, en i| .
let B := {b1 , . . . , bK } be a basis for Y.
N.
n
The dimensions of the null space N (T),, Any linear operator T : X Y is
Let M be any linear subspace of H that
uniquely determined by the K images
the range space R(T) and the space X itcontains {en }; then M is dense in H.
of the N basis vectors yk = Ten .
self are related as dim N (T)+dim R(T) =
Any linear operator T on a finiteBanach Algebra [4, 5]. Strictly speakdim X .
dimensional linear space can be repreing, a Banach Algebra is an algebra B over
Let X1 , X2 , Y1 , Y2 be linear spaces over
sented by a matrix T with [T]k,n = tk,n ,
a scalar field F, where B is also a Banach
the same scalar field so that X1 and X2
where T depends on the bases E and B.
space under a norm kk that satisfies the
are isomorphic and Y1 and Y2 are isomorHence, the image of any vector x X
multiplicative inequality kxyk kxkkyk
phic. The linear operators T1 : X1 Y1
can be obtained as
for all x, y B.
and T2 : X2 Y2 are said to be isomorK X
N
In the following, though, an associative unit
X
phically equivalent if there exists isomorcomplex Banach algebra, i.e., a Banach aly
=
Tx
=
(tk,n n )bk
phisms U : X1 X2 and W : Y1 Y2
gebra over the complex field C that is assok=1 n=1
such that T1 = W1 T2 U and T2 =
PN
ciative and contains an identity element 1
WT1 U1 .
where x = n=1 n en .
with respect to vector multiplication such
Let X1 and X2 be isomorphic linear For given bases
E and B, the matrix T is
that k1k = 1 is simply referred to as a comspaces. The linear operators T1 : X1
uniquely determined by T.
plex Banach algebra.
X1 and T2 : X2 X2 are said to be sim- Conversely, any K N matrix T defines
ilar if there exists an isomorphism U :
a linear operator with respect to given
X1 X2 such that T1 = U1 T2 U
bases forX and Y.
and T2 = UT1 U1 .
Two matrices that represent a linear operSome Important Linear Spaces
Let T : V V be a linear operator
ator on a finite-dimensional normed space
and M V a linear subspace of V such
Euclidean Space [6]. The N -dimensional Lebesgue Space [3, 7]. Let X be an arbirelative to two different bases are similar.
that T(M) M; then M is called invaricomplex Euclidean space
trary set, F the -algebra of subsets of X ,
o
n
ant under T . In this case, the restriction Linear Functionals [1, 2]. A linear funcand a nonnegative measure on F . The
CN := x : x = x0 x1 xN 1 T, xn C Lebesgue space
of T to M is a mapping of M into itself. tional is a linear operator f : V F, defined
n
o
hx, yi :=
xn yn
p
or complex linear space, and let g be a
complement M are invariant under T,
measurable,
|f
|
d
<
n=0
real-valued functional on V that is subthen M is said to reduce T.
and corresponding induced norm is a finite- with norm
additive, i.e., g(u + v) g(u) + g(v) for
Any operator that maps a Banach space
Z
1/p
dimensional Hilbert space.
all u, v V, and that satisfies g(u) =
onto another Banach space is an open
p
kf
k
:=
|f
|
d
|| g(v) for every scalar . Let f be
mapping.
p
Sequence Space [3]. The sequence space
n
a linear functional, defined on a subInverse Operator. Let T : V Z be
is a Banach space for 1 p < .
space Z of V, that satisfies |f (z)| g(z)
lp := x : x = {xn }
n=0 ,
operator. Then, the inverse oper For p = , the space L (X , F , ) with a linear
for all z Z. Then, f has a linear exten1
o
X
-essential supremum norm kf k := ator T : R(T) D(T) exists iff Tv = 0
p
sion f from Z to V that satisfies |f (v)|
implies that v = 0.
xn C,
|xn | <
ess supt |f (t)| is also a Banach space.
g(v) for all v V.
1
n=0
If is finite and X = (a, b], the spaces If T exists, it is a linear operator.
The
codimension of N (f ) is 1.
1
p
p
with norm
L () := L ((a, b], F , ) are nested: If dim D(T) = N < and T exists, If A is any subspace of V with N (f ) A
!
1/p
then dim R(T) = dim D(T).
Lp () Lq () for p q.
X
and N (f ) 6= 0, then A = V.
p
2
lection F of all subsets of X , the space Von Neumann Theorem: Let T : B B a vector, i.e., a functional f V ? , is defined
xn yn
hx, yi :=
2
N
2
L (X , F ) reduces to C or l , respecn=0
be a bounded operator on a Banach for all v V ? as p(v) = (f )(v) := f (v).
tively.
1/2
and norm kxk2 := hx, xi
is an infinitespace B that satisfies kI P
Tk < 1. Then, Let V be an N -dimensional linear space,
1
n
dimensional Hilbert space, called the When is a probability measure, i.e.,
T
is
invertible,
and
T
=
and let E = {e1 , . . . , eN } be a basis
n=0 (I T) .
(X
)
=
1
for
arbitrary
X
,
then
Hilbert sequence space.
1
for V. Define the set of linear functionFurthermore,
kT
k
1/(1
kI
Tk).
2
L (X , F , ) is the space of all random
als B := {f1 , . . . , fN } with fk (en ) = kn .
Space of Continuous Functions.
variables with finite second moment.
Projections [2].
A linear operator P :
Then
B is a basis for the algebraic dual
Let C[a, b] denote the space of all complexX X that satisfies P2 = P is called a
space V ? of V, and dim E = dim B; B is
valued continuous functions f : [a, b] C Schwarz Space [7]. The Schwarz space S projection.
called the dual basis of E.
with pointwise addition and scalar multipli- is the space of all infinitely differentiable,
rapidly decaying functions of a real parame- Range R(P) and null space N (P) are
cation.
C [a, b], endowed with the supremum ter t:
n
dn f (t)
norm kf k := supatb |f (t)|, is a BaLinear Functionals on Normed Spaces
S := f : R C : lim tm
=0
nach space.
t
dtn
Linear Functionals [1, 2].
Let f :
Theorem implies that every bounded lin Endowed with the inner product hf, gi :=
m, n N
Rb
U
F
be
a
linear
functional
on
a
normed
ear functional f on a subspace S U has
f (t)g (t)dt and the induced norm, this
space U.
a
a linear extension f on U that has the
Paley-Wiener
Space.
to
write
space is an inner product space but not
same norm,
The norm kf k of a linear functional f
a Hilbert space.
Sobolev Space. to write
is the usual operator norm: kf k =
sup
f
(u)
= sup |f (s)| .
supuU ,u6=0 |f (u)|.
uU ,kuk=1
sS,ksk=1
A bounded linear functional is a linear
Let U be a normed space and let u
function f that satisfies kf k a for
U. Then, there exists a bounded linsome a R.
ear functional f on U such that kf k = 1
On a normed space U, the Hahn-Banach
and f (u) = kuk.
is written as fn f .
X
X
space, whose norm is the usual operator
T
n xn =
n T(xn )
norm kTk for all T G(U, Z).
Let B ba a Banach space; then, G(U, B)
n=1
n=1
P
is a Banach space.
for every convergent series n=1 n xn
Let H is a Hilbert space, then G(H, H) is
in X .
a Banach algebra.
If T is continuous at a single point, it is
continuous.
Convergence [1, 4.9]. Let U and Z be
The linear operator T is continuous iff it normed spaces. A sequence {Tn } of operais bounded.
tors Tn G(U, Z) is said to be
If a linear operator T is continuous, it is
uniformly operator convergent if {Tn }
uniformly continuous.
converges in the operator norm
If X is finite dimensional, then T is conon G(U, Z), i.e., kTn Tk 0;
tinuous.
strongly operator convergent if {Tn u} converges strongly in Z for every u U,
Operator Norm [1, 2.7]. Let T : U Z
i.e., kTn u Tuk 0 for all u U;
be a linear operator that maps a normed
space U into a normed space Z. The opera- weakly operator convergent if {Tn u} converges weakly in Z for every u U,
tor norm is defined as
i.e., |f (Tn u) f (Tu)| 0 for all u U
kTuk
kTk := sup
and all bounded linear functionals f on U,
uU kuk
0
u6=0
that is, for all f in the dual space U of U.
where the norms on the RHS are vec- Uniform convergence implies strong convertor norms in Z and U. If D(T) = {0}, gence, which in turn implies weak converthen kTk := 0.
gence, all with the same limit.
The operator norm kTk of T is equivalent Let Tn G(B, U), where B is a Banach
to
space and U a normed space. If {Tn } is
kTk = sup kTuk.
strongly operator convergent with limit T,
uU
then T G(B, U).
kuk=1
A sequence {Tn } of operators in G(B, Z),
Bounded Linear Operators [1, 2.7].
where B and Z are Banach spaces, is
The linear operator T : U Z that maps a
strongly operator convergent iff (i) the
normed space U into a normed space Z is
sequence {kTn k} is bounded, and (ii) the
said to be bounded if there is a real number a
sequence {Tn b} is Cauchy in Z for evsuch that kTk a.
ery b in a total subset of B.
A linear operator T is bounded iff it is
Adjoint Operator [1, 4.5]. Let X and Y
continuous.
If a normed space U is finite dimen- be normed spaces and let T : X Y be
for any
sional, then every linear operator on U is a bounded linear operator. Then,
0
bounded
linear
functionals
f
X
and
g
bounded.
0
0
0
T = 0 iff hTu, zi = 0 for all u U Y , the adjoint operator T : Y X of T
is defined by f (x) = (T g)(x) = g(Tx) for
and z Z.
all x X .
The null space N (T) of T is closed.
If {un } a sequence in D(T), then un u The adjoint operator T is linear and
implies Tun Tu.
bounded, and kT k = kTk.
For bounded linear operators T1 : X If T is represented by a matrix T, then
Y and T2 : Y Z on normed
the adjoint operator T is represented
spaces X , Y, and Z, it follows that
by TT .
kT1 T2 k kT1 kkT2 k, and for T : X X
Let S : X Y be another bounded linear
that kTn k kTkn .
operator. Then
Uniform
Boundedness
Theorem:
(S + T) = S + T .
Let {Tn } be a sequence of linear op (T) = T , F.
erators Tn : B U from a Banach
(ST) = T S .
space B into a normed space U such
1
exists and T1 B(X , Y),
that kTn bk cb < for every b B If T
1
then (T )
also exists, (T )1
and every n = 1, 2, . . . . Then, the se0
0
quence of norms {kTn k} is bounded, i.e.,
B(X , Y ), and (T )1 = (T1 ) .
there exists a c such that kTn k c for
Closed Linear Operators [1, 4.13].
all n = 1, 2, . . . .
A bounded linear operator T from a Ba- Let U and Z be normed spaces and let T :
nach space B onto a Banach space Z has D(T) Z be a linear operator with dothe property that the image T(B1 (0)) of main D(T) U. Then, T is called a
the open unit ball around the origin con- closed linear operator if its graph G(T) :=
{(u, z) : u D(T), z = Tu} is closed in the
tains an open ball around 0 Z.
Open mapping theorem: A bounded linear normed space U Z.
operator T from a Banach space onto an- Closed graph theorem: Let T be a closed
operator. If D(T) is closed in V, the opother Banach space is an open mapping.
erator T is bounded.
Hence, if T is bijective, T1 is continuous
Let T : D(T) Z be a linear operator,
and thus bounded.
where D(T) U and U, Z are normed
Operator Topologies [1, 2]. Let G(U, Z)
spaces. Then, T is closed iff it has the foldenote the set of all bounded linear operalowing property: If un u for un
tors from a normed space U into a normed
D(T), and Tun z, then u D(T)
space Z over the same scalar field. The
and Tu = z.
set G(U, Z) is a linear space under operator
for any u U.
Let {un } be a weakly convergent sew
u. Then {Tun }
quence in U with un
is strongly convergent in Z and has the
strong limit z = Tu.
If T is compact, so is its adjoint opera0
0
tor T : Z U .
Let {Tn } be a sequence of compact linear
operators from a normed space U into
a Banach space B. If {Tn } is uniformly
operator convergent, i.e., kTn Tk 0,
then the limit operator T is compact.
A compact linear operator T : U B
from a normed space U into a Banach
:
space B has a compact linear extension T
mM
as R (T) := T = (T I) on R(T ).
QQQ
QQQ no
QQQ
The resolvent set Q(T) of T is defined as the
yes
QQQ
Q(
set of all complex numbers such that the
range of T is dense in B and that T has
Q(T)
Sc (T)
a continuous inverse defined on its range.
The numbers Q(T) are called regular The four sets are pairwise disjoint
values. The set S(T) := Q(T)c is called the
and C = Q(T) Sp (T) Sc (T) Sr (T);
spectrum of T; a S(T) is called a specsome of the sets may be empty.
tral value of T. The spectrum S(T) can be If R (T) exists, it is a linear operator.
partitioned into three disjoint sets:
Let B be a complex Banach space, T :
B B a linear operator, and Q(T).
The point spectrum Sp (T) is the set such
If T is closed or bounded, then, R (T)
that T is not one-to-one. A Sp (T)
is defined on the whole space B and is
is called an eigenvalue of T.
bounded.
The continuous spectrum Sc (T) is the
set such that T is one-to-one, has its
Eigenvalues [1, 7]. Let U be a normed
range dense set in B, but R (T), defined
space over the complex field and T : D(T)
on R(T ), is not continuous and, thereU a linear operator with domain D(T) U.
fore, unbounded.
The residual spectrum Sr (T) is the set The resolvent R (T) exists iff Tu = 0
implies u = 0, i.e., the null space N (T)
such that T is one-to-one, but R(T ) is
is {0}.
not dense in B.
If T u = 0 for some u 6= 0, then
In summary:
Sp (T). The vector u is then called an
eigenvector of T with eigenvalue .
The subspace of D(T) that consists of 0
and all eigenvectors of T with eigenvalue is called the eigenspace of T corresponding to that eigenvalue.
Eigenvectors with different eigenvalues
constitute a linearly independent set.
X
(I T)1 =
Tn = I + T + T2 + . . .
n=0
X
R (T) =
( 0 )n Rn+1
0 .
n=0
A
complex number is in S(T) iff
TT = T T
there exists a sequence {hn } with hn
H, khn k = 1 for all n, such that k(T
I)hn k 0 as n ; in other words,
self-adjoint operators
the operator T I is not bounded below.
!
Let
a bounded linear operator H on
T=T
I
a Hilbert space H have the Cartesian
unitary
decomposition H = T + iS, where T
operators
and S are self-adjoint. Then, H is nororthogonal
mal iff T and S commute.
In that
!
1
projections U : U = U
case, max kTk2 , kSk2 kHk2 kTk2 +
P
kSk2 .
O
Let T and S be normal operators on
trace-class operators
a Hilbert space H such that one commutes with the adjoint of the other, i.e.,
TS? = S? T and T? S = ST? , or such
Hilbert-Schmidt operators
that the two operators commute, i.e.,
TS = ST; then, T + S, TS, and ST are
compact operators
normal.
Linear Operators
continuous
linear
operators
T:HH
Unitary Operators.
Let the operators U, V : H H be unitary, H a Hilbert
space. Then,
U is isometric, i.e., kUhk = khk for
all h H,
kUk = 1,
U1 is unitary,
UV is unitary.
A bounded linear operator on a Hilbert
space over the complex field is unitary iff
Hilbert Adjoint Operator [1, 2]. Let T :
it is isometric and onto.
H Z be a bounded linear operator
that maps the Hilbert space H into the Polar Decomposition [8, 30]. Let T :
Hilbert space Z. The Hilbert adjoint op- H H be a compact linear operator on a
erator T? of T is the operator T? : Z H separable complex Hilbert space H; let T?
such that hTh, zi = hh, T? zi for all h denote the Hilbert adjoint of T.
H and z Z. This operator exists, is The operator T can be factored as T =
unique, and is a bounded linear operator
UA, where A is a positive Hermitian opwith norm kT? k = kTk.
erator and U? U = I on the range of A.
Let S : H Z be another bounded linThe above factorization is called the poear operator, and let be any scalar. The
lar decomposition of T; the operator A is
Hilbert adjoint operator has the following
called the absolute value of T. The polar
properties:
decomposition exists even if T is bounded
I? = I.
instead of compact.
hT? h, zi = hz, Thi.
n (T).
2
n
0
0
H H and A2 : Z Z are bijective,
n=1
n=1
isometric, conjugate linear operators that
are uniquely defined by Rieszs theorem.
khk
uct ST is nonnegative.
If T is bounded and self adjoint, then T2
is nonnegative.
A monotone sequence {Tn } of bounded,
self-adjoint, linear operators is a sequence
that is either monotonically increasing, i.e.,
T1 T2 T3 . . . , or monotonically decreasing, T1 T2 T3 . . . .
Let {Tn } be a monotonically increasing
sequence of bounded, self-adjoint, linear
operators such that T1 T2 . . .
Tn . . . S, where S is also bounded
and self adjoint. Suppose that all elements of the sequence commute pairwise
and also commute with S. Then, {Tn } is
strongly operator convergent, Tn h Th
for all h H, and the limit operator T
is linear, bounded, self adjoint, and satisfies T S.
A bounded, self-adjoint linear operator S
is called a square root of another bounded,
self-adjoint, linear operator T if S2 = T.
If, in addition, S O, then S is called a
nonnegative square root of T and is denoted
by S = T1/2 .
Every nonnegative, bounded, self-adjoint,
linear operator T : H H on a complex
Hilbert space H has a nonnegative square
root S that is unique.
The square-root operator S of T commutes with every bounded linear operator
on H that commutes with T.
Compact Normal Operators [2].
Let T : H H be a normal operator on a
nontrivial Hilbert space H, and let T have
the Cartesian decomposition T = A + iB.
The operator T is compact iff both A
and B are compact.
The operator T is compact iff T? is compact.
If T is compact, it has an eigenvalue
with max{kAk, kBk} ||. If T is
self-adjoint, then it has an eigenvalue
with = kTk.
If T is compact and has no eigenvalues,
then H = {0}.
If H is not separable, then = 0 is necessarily an eigenvalue of any compact normal operator on H.
Hilbert-Schmidt Operators [9, 2, 8].
Let {xn } be an orthonormal basis for a
Hilbert space H. A bounded linear operator T : H H isPcalled a Hilbert
Schmidt(HS) operator if n=1 kTxn k2 < .
The number
X
1/2
kTkHS :=
kTxn k2
n=1
Trace Class Operators [8, 30]. A comas the product of two HS operators.
pact linear operator T : H H on a Hilbert Let {xn } be any orthonormal basis of H.
space H is said to be in trace class if
For a trace class operator T, the trace is
X
defined as the limitX
of the series
n (T) < .
tr T :=
hTxn , xn i.
n=1
The above sum defines the trace norm kTktr : This series converges absolutely. For trace
X
class operators
P T
kTktr :=
n (T).
tr T = n n (T), where n (T) are the
n=1
eigenvalues of T.
For T in trace class and any bounded oper- If T is a trace class operator that has no
ator B : H H
eigenvalues except = 0. Then, tr T = 0.
kTk kTktr ,
|tr T| kTktr .
kTktr = kT? ktr ,
tr T is a linear mapping of T.
kBTktr kBktr kTktr ,
tr T? = (tr T) .
kTBktr kBktr kTktr .
For any bounded operator B, tr(TB) =
For any pair of trace class operators T
tr(BT).
and S, T+S is trace class, and kT+Sktr Let T be a trace class operators, and
kTktr + kSktr .
let {Tn } be a sequence of degenerate op The trace class is a two-sided ideal in the erators, i.e., operators with finite range that
algebra of all bounded linear operators converge to T in trace norm. Then, the
on a complex Hilbert space.
determinant det(I + Tn ) of the matrix rep Trace class operators form a Banach resentation of I + T , I + T , tends to a
n
n
space with respect to the trace norm.
limit that is independent of the choice of
Every trace class operator is HS.
the sequence {Tn }. This limit is called the
The product of two HS operators T and S determinant of I + T:
is in trace class, and kSTktr kSkHS
det(I + T) := lim det(I + Tn ).
kTk .
HS
[
P(H) =
Pn (H).
n=1
\
N (P) =
N (Pn ).
n=1
Z MT
+
of T .
f (T) =
f ()dE ,
Let E with R be the projection of H
mT 0
+
onto the null space Y of T . Then, the
and for all x, y H,
Z MT
collection E (T) := {E : R} is the
unique spectral family associated with T
hf (T)x, yi =
f ()dw().
mT 0
on the interval [mT , MT ] R.
For < , the projection operator E
The operator f (T) is self adjoint.
E satisfies (E E ) T(E E )
If f () = f1 ()f2 (), then f (T) =
(E E ).
f1 (T)f2 (T).
The mapping E has a discontinuity
If f () 0 for all [mT , MT ], then
at 0 , i.e., E0 6= E+ , iff 0 is an eigenf (T) O.
0
value of T. In this case, the eigenspace
If f1 () f2 () for all [mT , MT ],
that corresponds to the eigenvalue 0
then f1 (T) f2 (T).
is N (T 0 I) = (E0 E+ )(H).
kf (T)k max[mT ,MT ] |f ()|.
0
If a bounded linear operator commutes
A real 0 belongs to the resolvent
with T, it also commutes with f (T).
set R (T) iff there is an > 0 such
that E (T) is constant on the interval [0
Unitary Operators [1, 10.5]. Let U :
, 0 + ].
H H be a unitary operator on a complex
A real 0 belongs to the continuous specHilbert space H.
trum Sc (T) iff the mapping E is
continuous at 0 (thus E0 = E+ ) and is The spectrum S(U) is a closed subset of
0
the unit circle. Consequently, || = 1 for
not constant in any neighborhood of 0 .
every S(U).
A bounded, self-adjoint, linear operator T on There exists a spectral family E = {E }
a complex Hilbert space H has the spectral
on [, ] such that
Z
representation
Z MT
Z MT
ei dE .
U=
T=
dE = mT EmT +
dE ,
mT 0
mT
for every continuous function f on the
unit circle,
where E = {E } is the spectral family asZ
sociated with T, and the integral is to be
f (U) =
f ei dE ,
understood in the sense of uniform operator
(g X given) (OE3)
T f f = 0.
(OE4)
Then,
Equation (OE1) has a solution x
iff f (y) = 0 for all solutions f of (OE4).
Hence, if f = 0 is the only solution
of (OE4), then (OE1) is solvable for every y.
Equation (OE3) has a solution f
iff g(x) = 0 for all solutions x of (OE2).
Hence, if x = 0 is the only solution
of (OE2), then (OE3) is solvable for every g.
Equation (OE1) has a solution x for every y U iff x = 0 is the only solution
of (OE2).
Equation (OE3) has a solution f for ev0
ery g U iff f = 0 is the only solution
of (OE4).
Equations (OE2) and (OE4) have the
same number of linearly independent solutions.
Linear Operator Equations [1, 8.5].
References
[1] E. Kreyszig, Introduction to Functional Analysis with Applications. New York, NY, [6]
U.S.A.: Wiley, 1989.
[7]
[2] A. W. Naylor and G. R. Sell, Linear Operator Theory in Engineering and Science.
New York, NY, U.S.A.: Springer, 1982.
[8]
[3] M. Pourahmadi, Foundations of Time Series Analysis and Prediction Theory. New
[9]
York, NY, U.S.A.: Wiley, 2001.
[4] W. Rudin, Real and Complex Analysis, 3rd ed.
1987.
N. Dunford and J. T. Schwarz, Linear Operators, New York, NY, U.S.A., 1958, vol. 1.
R. L. Allen and D. Mills, Signal Analysis: Time, Frequency, Scale, and Structure.
Piscataway, NJ, U.S.A.: IEEE Press, 2004.
P. D. Lax, Functional Analysis.