Time Series

Time Series Analysis and
Forecasting
Time Series Analysis Lecture Notes

MA(4030)Prepared By TMJA
Cooray
Introduction
A time
series is a set of observations generated
sequentially in time
Continuous
The
vs. discrete time series
observations from a discrete time series, made
at some fixed interval h, at times 1, 2,, N may

be denoted by x(1), x(2),, x(N)
Cooray
Introduction (cont.)
Discrete
time series may arise in two ways:
1- By sampling a continuous time series

2- By accumulating a variable over a period of time
Characteristics
of time series
Time periods are of equal length

No missing values

Cooray
Components of a time series

A t im e s e r ie s
p a tte rn c o m p o n e n t
r a n d o m (e r ro r ) c o m p o n e n t
tre n d p a tte rn
s e a s o n a l p a tte rn
c y c lic p a t t e r n
s ta tis tic a l p a tte r n
x =F +x

t
t
t
Cooray
Areas of application
Forecasting
Determination
Design
of a transfer function of a system
of simple feed-forward and feedback
control schemes

Cooray
Forecasting
Applications
Lead
Economic and business planning

Inventory and production control
Control and optimization of industrial processes
time of the forecasts
is the period over which forecasts are needed

Degree
of sophistication
Simple ideas
Moving averages
Simple regression techniques
Complex statistical concepts
Box-Jenkins methodology

Cooray
Approaches to forecasting
Self-projecting
approach
Cause-and-effect

Cooray
approach
Approaches to forecasting (cont.)

Self-projecting
approach
Advantages
Quickly and easily applied

A minimum of data is required
Reasonably short-to medium-term
forecasts
They provide a basis by which
forecasts developed through other
models can be measured against
Cause-and-effect
Advantages
Bring more information

More accurate medium-to
long-term forecasts
Disadvantages
Disadvantages
approach
Not useful for forecasting into the

far future
Do not take into account external
factors

Cooray
Forecasts of the explanatory

time series are required
Some traditional self-projecting models

Overall
trend models
The trend could be linear, exponential, parabolic, etc.

A linear Trend has the form
Trendt = A + Bt
Short-term changes are difficult to track
Smoothing
models
Respond to the most recent behavior of the series

Employ the idea of weighted averages
They range in the degree of sophistication
The simple exponential smoothing method:
Ft

Notes
z t Az t 1Time
MA(4030)Prepared
(Series
1 Analysis
A)FLecture
at
tBy
1 TMJA
Cooray
Some traditional self-projecting

models (cont.)
Seasonal
models
Very common
Most seasonal time series also contain long- and short-
term trend patterns

Decomposition
models
The series is decomposed into its separate patterns

Each pattern is modeled separately
Cooray
Drawbacks of the use of traditional

models
There
is no systematic approach for the identification

and selection of an appropriate model, and therefore,
the identification process is mainly trial-and-error
There is difficulty in verifying the validity of the
model
Most traditional methods were developed from intuitive
and practical considerations rather than from a statistical

foundation
Too narrow to
deal efficiently with all time series

Cooray
ARIMA models
Autoregressive
Integrated Moving-average
Can represent a wide range of time series
A stochastic
stochastic modeling approach that can be used
to calculate the probability of a future value lying
between two specified limits

Cooray
ARIMA models (Cont.)

In
the 1960s Box and Jenkins recognized the

importance of these models in the area of
economic forecasting
Time series analysis - forecasting and control
control
George E. P. Box
Gwilym M. Jenkins
1st edition was in 1976
Often
called The Box-Jenkins approach

B o x - J e n k in s m o d e ls
U n iv a r ia te
M u ltiv a r ia te (tr a n s fe r fu n c tio n )

Cooray
Transfer function modeling
Yt
= (B)Xt where
(B) = 0 + 1B + 2B2 + ..
B
is the backshift operator
BmXt = Xt - m
Cooray
Transfer function modeling (cont.)

The
study of process dynamics can achieve:
Better control
Improved design
Methods
for estimating transfer function models
Classical methods
Based on deterministic perturbations

Uncontrollable disturbances (noise)
noise are not accounted for,
and hence, these methods have not always been successful
Statistical methods
Make allowance for noise

noise
The Box-Jenkins
Timemethodology
Series Analysis Lecture Notes
Cooray
Process control
Feed-forward
control
Feedback
control
Deviation from target output

P
Deviation from
target output
Nt
Nt
( B)(B)B
1
1
L (B) L 2 (B)B
f 1
Compensating
variable Xt+
zt
Control equation

Cooray
L 11 (B) L 2 ( B)B f
Compensating
variable Xt+
Control equation
Process control (cont.)

D is tu r b a n c e s
m easu ed
u n m e a su r ed
fe e d -fo r w a r d c o n to l
fe e d b a c k c o n tr o l

Cooray
Process control (cont.)

The
Box-Jenkins approach to control is to typify the

disturbance by a suitable time series or stochastic model
and the inertial characteristics of the system by a
suitable transfer function model
The Control equation, allows the action which should
be taken at any given time to be calculated given the
present and previous states of the system
Various ways corresponding to various levels of
technological sophistication can be used to execute a
control action called for by the control equation

Cooray
The Box-Jenkins model building

process
Model identification
Model estimation
No
Modify
model
Is model
adequate ?
Yes
Forecasts

Cooray

process (cont.)
Model estimation
The objective is to minimize the sum of squares of

errors
Model validation
Autocorrelations
Partial-autocorrelations
Certain diagnostics are used to check the validity of the

model
Model forecasting
The estimated model is used to generate forecasts and

confidence limits of the forecasts
Cooray
Important Fundamentals
A Normal
process
Stationarity
Regular differencing
Autocorrelations (ACs)
The white noise process
The linear filter model
Invertibility
Cooray
A Normal process (A Gaussian

process)
The
Box-Jenkins methodology analyze a time series as a

realization of a stochastic process.
The observation zt at a given time t can be regarded as a
realization of a random variable zt with probability density

function p(zt)
The observations at any two times t 1 and t2 may be regarded as
realizations of two random variables zt1, zt2 and with joint
probability density function p(z t1, zt2)
If the probability distribution associated with any set of times is
multivariate Normal distribution, the process is called a normal

or Gaussian process
Cooray
Stationary stochastic processes

In
order to model a time series with the Box-Jenkins
approach, the series has to be stationary

In practical terms,
terms
the series is stationary if tends to
wonder more or less uniformly about some fixed level

In statistical terms,
terms
a stationary process is assumed to
be in a particular state of statistical equilibrium, i.e.,

p(xt) is the same for all t

Cooray
Stationary stochastic processes (cont.)

the
process is called strictly stationary

stationary
if the joint probability distribution of any m
observations made at times t1, t2, , tm is the same as

that associated with m observations made at times t1 + k,
t2 + k, , tm + k
When
m = 1, the stationarity assumption implies
that the probability distribution p(zt) is the same

for all times t
Cooray
Stationary stochastic processes (cont.)

In
particular, if zt is a stationary process, then the
first difference zt = zt - zt-1and higher differences

dzt are stationary
Most
time series are nonstationary

Cooray
Achieving stationarity
Regular
differencing (RD)
(1st order)
(2nd order)
xt = (1 B)xt = xt xt-1
2xt = (1 B)2xt = xt 2xt-1 + xt-2
B is the backward shift operator

It
is unlikely that more than two regular differencing
would ever be needed

Sometimes
regular differencing by itself is not
sufficient and prior transformation is also needed

Cooray
Some nonstationary series

Cooray
Some nonstationary series (cont.)

Cooray
Some nonstationary series (cont.)

How
Howcan
canwe
wedetermine
determinethe
the
number
numberof
ofregular
regular
differencing
differencing??

Cooray
Autocorrelations (ACs)
Autocorrelations are
statistical measures that indicate

how a time series is related to itself over time
The
autocorrelation at lag 1 is the correlation between

the original series zt and the same series moved
forward one period (represented as zt-1)

Cooray
Autocorrelations (cont.)
The
theoretical autocorrelation function

E (z t )(z t k )
k
2z
The
sample autocorrelation
N k
rk
(z
t 1
z )(z t k z )
2
(
z
z
)
t
t 1
k 0,1,2,...k

Cooray
Autocorrelations (cont.)
A graph
of the correlation values is called a

correlogram
correlogram
In practice, to obtain a useful estimate of the
autocorrelation function, at least 50 observations
are needed
The estimated autocorrelations rk would be
calculated up to lag no larger than N/4

Cooray
A correlogram of a nonstationary
time seies

Cooray
After one RD

Cooray
After two RD

Cooray
The white noise process

The
Box-Jenkins models are based on the idea that

a time series can be usefully regarded as generated
from (driven by) a series of uncorrelated
independent shocks
shocks et
E e t 0 var e t e2
1
k
0
k0
k0
Such
a sequence et, et-1, et-2, is called a white

noise process
process
Cooray
The linear filter model

( B)
White noise
Linear filter
xt
et
A linear filter
filter
is a model that transform the

white noise process et to the process that generated
the time series xt
x t e t 1e t 1 2 e t 2 ... (B)e t
Cooray
The linear filter model (cont.)

is the transfer function

function of the filter
(B) 1 1 B 2 B ... j B with 0 1

2
j 0

Cooray
The linear filter model (cont.)

The
linear filter can be put in another form
x t 1x t 1 2 x t 2 ... e t
x
j
j1
This
t j
et
form can be written
( B) x t e t
( B) 1
jB j
z t (B)at
(B)z t at
jTime
1 Series Analysis Lecture
( BNotes
)
Cooray
1 (B)
Stationarity and invertibility

conditions for a linear filter
For
a linear process
to be stationary,
stationary
(B) must converge

for B 1
If
the current observation xt

depends on past observations
with weights which decrease as
we go back in time, the series is
called invertible
For
a linear process to be
invertible,
invertible (B) must converge
for B 1

Cooray
Model building blocks

Autoregressive
(AR) models
Moving-average (MA) models
Mixed ARMA models
Non stationary models (ARIMA models)
The mean parameter
The trend parameter

Cooray
Autoregressive (AR) models

An
autoregressive model of order p
x t 1x t 1 2 x t 2 ... p x t p e t
(B) x t e t
(B) 1 1B 2 B 2 ... p B p
The
autoregressive process can be thought of as

the output from a linear filter with a transfer
function -1(B),
(B) when the input is white noise et
equation (B) = 0 is called the characteristic
equation
equation
The
Cooray
Moving-average (MA) models

A moving-average
model of order q
x t e t 1e t 1 2 e t 2 ... q e t q
x t (B)e t
2
(B) 1 1B 2 B ... q B
The
moving-average process can be thought of as

the output from a linear filter with a transfer
function (B),
(B) when the input is white noise et
equation (B) = 0 is called the characteristic
equation
equation
The
Cooray
Mixed AR and MA (ARMA) models

A moving-average
written as
process of 1st order can be
1
zt a t
(1 1B)
(1 1B 12 B 2 13 B3 ...) x t e t
x t 1x t 1 12 x t 2 13 x t 3 ... e t
Hence,
Hence
if the process were really MA(1), we

would obtain a non parsimonious representation in
terms of an autoregressive model
Cooray
Mixed AR and MA (ARMA) models

(cont.)
In
order to obtain a parsimonious model,

sometimes it will be necessary to include both AR
and MA terms in the model
(B)
An ARMA(p, q) model
xt
et
(B)
The ARMA process
can be thought of as the

output from a linear filter with a transfer function
(B)/(B),
(B) when the input is white noise at
Cooray

process
Autocorrelations
Partial-autocorrelations
Model
estimation
Model validation
Certain diagnostics are used to check the validity of the
model
Model
forecasting
Cooray
Partial-autocorrelations (PACs)
Partial-autocorrelations are
another set of statistical

measures are used to identify time series models
PAC
is Similar to AC, except that when calculating

it, the ACs with all the elements within the lag are
partialled out (Box & Jenkins, 1976)

Cooray
Partial-autocorrelations (cont.)
PACs
can be calculated from the values of the ACs

where each PAC is obtained from a different set of
linear equations that describe a pure
autoregressive model of an order that is equal to
the value of the lag of the partial-autocorrelation
computed
PAC
at lag k is denoted by kk
The double notation kk is to emphasize that kk is the
autoregressive parameter k of the autoregressive model

of order k
Cooray
The
sample ACs and PACs are computed for the

series and compared to theoretical autocorrelation
and partial-autocorrelation functions for candidate
models investigated
Stationarity and
invertibility
conditions
Theoretical ACs and

PACs

Cooray
Stationarity and invertibility

conditions
For
a linear process
to be stationary,
stationary
For
a linear process to be
invertible,
invertible
(B) must converge
(B) must converge
for B 1
for B 1

Cooray
Stationarity requirements for AR(1)

model
For an AR(1) to be stationary:
-1 < 1 < 1
i.e., the roots of the characteristic
equation 1 - 1B = 0 lie outside
the unit circle
For an AR(1) it can be shown that:

k = 1 k 1 which with 0 = 1 has
the solution
k = 1k
k>0
i.e., for a stationary AR(1) model,
the theoretical autocorrelation
function decays exponentially to
zero, however, the theoretical
partial-autocorrelation function has
a cut off after the 1st lag
Cooray
Invertibility requirements for a MA(1)

model
For
a MA(1) to be invertible:
-1 < 1 < 1
i.e., the roots of the characteristic equation 1 - 1B = 0

lie outside the unit circle
1
For
a MA(1) it can be shown that:
k 1
0
12
k 1
k 1
i.e., for an invertible MA(1) model, the theoretical

autocorrelation function has a cut off after the 1st lag,
however, the theoretical partial-autocorrelation function
decays exponentially
to zero
Cooray
Higher order models

For
an AR model of order p > 1:
The autocorrelation function consists of a mixture of
damped exponentials and damped sine waves

The partial-autocorrelation function has a cut off after
the p lag
For
a MA models of order q > 1:
The autocorrelation function has a cut off after the q lag

The partial-autocorrelation function consists of a
mixture of damped exponentials and damped sine

waves
Cooray
Permissible regions for the AR and

MA parameters

Cooray
Theoretical ACs and PACs (cont.)

Cooray
Theoretical ACs and PACs (cont.)

Cooray

Cooray
Model estimation

Cooray
Model verification

Cooray

Time Series

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Time Series

Uploaded by

Copyright:

Available Formats

Time Series Analysis and

Time Series Analysis Lecture Notes

series is a set of observations generated

vs. discrete time series

observations from a discrete time series, made

at some fixed interval h, at times 1, 2,, N may

time series may arise in two ways:

1- By sampling a continuous time series

Time periods are of equal length

Time Series Analysis Lecture Notes

Components of a time series

Time Series Analysis Lecture Notes

of a transfer function of a system

of simple feed-forward and feedback

Time Series Analysis Lecture Notes

Economic and business planning

time of the forecasts

is the period over which forecasts are needed

Complex statistical concepts

Time Series Analysis Lecture Notes

Time Series Analysis Lecture Notes

Approaches to forecasting (cont.)

Quickly and easily applied

Bring more information

Not useful for forecasting into the

Time Series Analysis Lecture Notes

Forecasts of the explanatory

Some traditional self-projecting models

The trend could be linear, exponential, parabolic, etc.

Short-term changes are difficult to track

Respond to the most recent behavior of the series

Some traditional self-projecting

term trend patterns

The series is decomposed into its separate patterns

Drawbacks of the use of traditional

is no systematic approach for the identification

and practical considerations rather than from a statistical

deal efficiently with all time series

Time Series Analysis Lecture Notes

ARIMA models (Cont.)

the 1960s Box and Jenkins recognized the

1st edition was in 1976

called The Box-Jenkins approach

M u ltiv a r ia te (tr a n s fe r fu n c tio n )

Transfer function modeling

is the backshift operator

Transfer function modeling (cont.)

study of process dynamics can achieve:

for estimating transfer function models

Based on deterministic perturbations

Make allowance for noise

Deviation from target output

Time Series Analysis Lecture Notes

Process control (cont.)

Time Series Analysis Lecture Notes

Process control (cont.)

Box-Jenkins approach to control is to typify the

Time Series Analysis Lecture Notes

The Box-Jenkins model building

Time Series Analysis Lecture Notes

The Box-Jenkins model building

The objective is to minimize the sum of squares of

Certain diagnostics are used to check the validity of the

The estimated model is used to generate forecasts and