Isabel Silva Magalhães Integer-valued time series

Integer-valued time series

Isabel Silva Magalhães

Departamento de Engenharia Civil, Faculdade de Engenharia da Universidade do Porto

Unidade de Investigação Matemática e Aplicações (UIMA), Universidade de Aveiro

PDMA-UP - October 2009

PDMA-UP - October 2009 1 / 30

 Isabel Silva Magalhães Integer-valued time series

Outline

Motivation
Thinning operation
INteger-valued AutoRegressive (INAR) processes
◮ INAR(1) processes
◮ INAR(p) processes

Parameter estimation for INAR(p) processes

Application to real data
Recent developments
Theme proposal

PDMA-UP - October 2009 2 / 30

 Isabel Silva Magalhães Integer-valued time series

Motivation
Discrete time non-negative integer-valued time series → counting series

Motivation PDMA-UP - October 2009 3 / 30

 Isabel Silva Magalhães Integer-valued time series

Motivation
Discrete time non-negative integer-valued time series → counting series

Examples:
the daily number of seizures of epileptic patients
the yearly number of plants in a region
the daily number of guest nights in a hotel
the monthly incidence of a disease.

Motivation PDMA-UP - October 2009 3 / 30

 Isabel Silva Magalhães Integer-valued time series

Motivation
Discrete time non-negative integer-valued time series → counting series

Examples:
the daily number of seizures of epileptic patients
the yearly number of plants in a region
the daily number of guest nights in a hotel
the monthly incidence of a disease.

Usual linear time series models (ARMA processes) are not suitable.

Motivation PDMA-UP - October 2009 3 / 30

 Isabel Silva Magalhães Integer-valued time series

Motivation
Discrete time non-negative integer-valued time series → counting series

Examples:
the daily number of seizures of epileptic patients
the yearly number of plants in a region
the daily number of guest nights in a hotel
the monthly incidence of a disease.

Usual linear time series models (ARMA processes) are not suitable.

Thinning operation

m
Multiplication counterpart on the integer-valued context.

Motivation PDMA-UP - October 2009 3 / 30

 Isabel Silva Magalhães Integer-valued time series

Thinning operation
Binomial thinning operation [Steutel and Van Harn (1979)]
X : non-negative integer-valued random variable (r.v.), α ≥ 0
X
α ◦ X = ∑ Yj
j=1
{Yj } ⊆ N0 (counting series): sequence of independent and identically distributed
(i.i.d.) r.v., independent of X, P(Yj = 1) = 1 − P(Yj = 0) = α
⇓
α ◦ X is the number of successes, with probability α , in X trials

Thinning operation PDMA-UP - October 2009 4 / 30

 Isabel Silva Magalhães Integer-valued time series

Thinning operation
Binomial thinning operation [Steutel and Van Harn (1979)]
X : non-negative integer-valued random variable (r.v.), α ≥ 0
X
α ◦ X = ∑ Yj
j=1
{Yj } ⊆ N0 (counting series): sequence of independent and identically distributed
(i.i.d.) r.v., independent of X, P(Yj = 1) = 1 − P(Yj = 0) = α
⇓
α ◦ X is the number of successes, with probability α , in X trials

Generalized thinning operation [Gauthier and Latour (1994); Silva and Oliveira (2004, 2005)]
X
α ∗ X = ∑ Yj
j=1
{Yj }: sequence of i.i.d.r.v., independent of X, with some discrete distribution such that
E[Yj ] = α , Var[Yj ] = σ 2 , E[Yj 3 ] = γ , E[Yj 4 ] = κ
Thinning operation PDMA-UP - October 2009 4 / 30
 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

INAR(1) processes [McKenzie (1985, 1988); Al-Osh and Alzaid (1987)]

Xt = α ∗ Xt−1 + et , t = 1, . . . , N

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 5 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

INAR(1) processes [McKenzie (1985, 1988); Al-Osh and Alzaid (1987)]

Xt = α ∗ Xt−1 + et , t = 1, . . . , N

0<α ≤1
{et } ⊆ N0 : sequence of i.i.d. discrete r.v. (arrival process)

E[et ] = µe , Var[et ] = σe 2 , E[et 3 ] = γe , E[et 4 ] = κe

counting series, {Yj }, are independent, and independent of {et }, and such that

E[Yj ] = α , Var[Yj ] = σ 2 , E[Yj 3 ] = γ , E[Yj 4 ] = κ

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 5 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Poisson INAR(1) process with binomial thinning operation

Xt = α ◦ Xt−1 + et

⇓
{et } Poisson distributed and {Yj } Bernoulli distributed

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 6 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Poisson INAR(1) process with binomial thinning operation

Xt = α ◦ Xt−1 + et

⇓
{et } Poisson distributed and {Yj } Bernoulli distributed

α ◦ Xt−1 : survivors of the elements of the process at time t − 1, each with

probability of survival α
et : elements which enter in the system in the interval ]t − 1, t]

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 6 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Poisson INAR(1) process with binomial thinning operation

Xt = α ◦ Xt−1 + et

⇓
{et } Poisson distributed and {Yj } Bernoulli distributed

α ◦ Xt−1 : survivors of the elements of the process at time t − 1, each with

probability of survival α
et : elements which enter in the system in the interval ]t − 1, t]

    
λ 
X1 ∼ Po 1−α λ
=⇒ {Xt } ∼ Po
{et } ∼ Po(λ )  1−α

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 6 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Likelihood function of the Poisson INAR(1) process

p(Xt |Xt−1 ) : convolution of binomial distribution and Poisson distribution
 
Mt
λ (Xt )−i Xt−1 i
p(Xt |Xt−1 ) = exp(−λ ) ∑ α (1 − α )(Xt−1 )−i , Mt = min{Xt−1 , Xt }
i=0 ((X t ) − i)! i

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 7 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Likelihood function of the Poisson INAR(1) process

p(Xt |Xt−1 ) : convolution of binomial distribution and Poisson distribution
 
Mt
λ (Xt )−i Xt−1 i
p(Xt |Xt−1 ) = exp(−λ ) ∑ α (1 − α )(Xt−1 )−i , Mt = min{Xt−1 , Xt }
i=0 ((X t ) − i)! i
w
w
X = {X0 , X1 , . . . , XN }
  N
[λ /(1 − α )]X0 λ
1−α ∏
L(X, α , λ ) = exp p(Xt |Xt−1 )
X0 ! t=1
| {z }
L(X,α ,λ |X0 )

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 7 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Likelihood function of the Poisson INAR(1) process

p(Xt |Xt−1 ) : convolution of binomial distribution and Poisson distribution
 
Mt
λ (Xt )−i Xt−1 i
p(Xt |Xt−1 ) = exp(−λ ) ∑ α (1 − α )(Xt−1 )−i , Mt = min{Xt−1 , Xt }
i=0 ((X t ) − i)! i
w
w
X = {X0 , X1 , . . . , XN }
  N
[λ /(1 − α )]X0 λ
1−α ∏
L(X, α , λ ) = exp p(Xt |Xt−1 )
X0 ! t=1
| {z }
L(X,α ,λ |X0 )

Poisson distribution → mean = variance → problem in practical applications

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 7 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Likelihood function of the Poisson INAR(1) process

p(Xt |Xt−1 ) : convolution of binomial distribution and Poisson distribution
 
Mt
λ (Xt )−i Xt−1 i
p(Xt |Xt−1 ) = exp(−λ ) ∑ α (1 − α )(Xt−1 )−i , Mt = min{Xt−1 , Xt }
i=0 ((X t ) − i)! i
w
w
X = {X0 , X1 , . . . , XN }
  N
[λ /(1 − α )]X0 λ
1−α ∏
L(X, α , λ ) = exp p(Xt |Xt−1 )
X0 ! t=1
| {z }
L(X,α ,λ |X0 )

Poisson distribution → mean = variance → problem in practical applications

Overdispersion: variance > mean → binomial, negative binomial, geometric or
generalized Poisson
Underdispersion: variance < mean → generalized Poisson
INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 7 / 30
 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

INAR(p) processes [Du and Li (1991); Latour (1998)]

Xt = α1 ∗ Xt−1 + · · · + αp ∗ Xt−p + et , t = 1, . . . , N,

αi ≥ 0, i = 1, . . . , p − 1, and αp > 0, such that ∑pi=1 αi < 1,

{et } ⊆ N0 : sequence of i.i.d. discrete r.v. (arrival process)

E[et ] = µe , Var[et ] = σe 2 , E[et 3 ] = γe , E[et 4 ] = κe ,

all counting series, {Yj,i }, of the thinning operations

αi ∗ Xt−i = ∑j=0
t−i X
Yj,i , i = 1, . . . , p,

are mutually independent, and independent of {et }, and such that

E[Yj,i ] = αi , Var[Yj,i ] = σi 2 , E[Yj,i 3 ] = γi , E[Yj,i 4 ] = κi .

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 8 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

INAR(p) processes [Du and Li (1991); Latour (1998)]

Xt = α1 ∗ Xt−1 + · · · + αp ∗ Xt−p + et , t = 1, . . . , N,

αi ≥ 0, i = 1, . . . , p − 1, and αp > 0, such that ∑pi=1 αi < 1,

{et } ⊆ N0 : sequence of i.i.d. discrete r.v. (arrival process)

E[et ] = µe , Var[et ] = σe 2 , E[et 3 ] = γe , E[et 4 ] = κe ,

all counting series, {Yj,i }, of the thinning operations

αi ∗ Xt−i = ∑j=0
t−i X
Yj,i , i = 1, . . . , p,

are mutually independent, and independent of {et }, and such that

E[Yj,i ] = αi , Var[Yj,i ] = σi 2 , E[Yj,i 3 ] = γi , E[Yj,i 4 ] = κi .

INAR(p) has the same second-order structure as an AR(p) process

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 8 / 30
 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Autocovariance function [Gauthier and Latour (1994)]
(
R(0) = Vp + ∑pi=1 αi R(i),
R(k) = ∑pi=1 αi R(i − k), k ∈ Z\{0},

Spectral density function [Silva and Oliveira (2004, 2005)]

1 Vp
f (ω ) =   , −π ≤ ω ≤ π ,
2π 1 − ∑ αk e−iω k 2
p
k=1

One-step-ahead prediction error [Silva (2005)]

Vp = σe 2 + µX ∑i=1 σi 2 .
p

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 9 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Autocovariance function [Gauthier and Latour (1994)]
(
R(0) = Vp + ∑pi=1 αi R(i),
R(k) = ∑pi=1 αi R(i − k), k ∈ Z\{0},

Spectral density function [Silva and Oliveira (2004, 2005)]

1 Vp
f (ω ) =   , −π ≤ ω ≤ π ,
2π 1 − ∑ αk e−iω k 2
p
k=1

One-step-ahead prediction error [Silva (2005)]

Vp = σe 2 + µX ∑i=1 σi 2 .
p

“One-sided” general linear representation [Silva (2005)]

∞
Xt = ∑u=0 γu εt−u , {εt } white noise process

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 9 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Third-order characterization (in terms of moments)

[Silva and Oliveira (2004, 2005) and Silva (2005)]

µX (0, 0) = ∑pi=1 ∑pj=1 ∑pk=1 αi αj αk µX (i − j, i − k) + 3 ∑pi=1 ∑pj=1 αj σi 2 µX (i − j)

+3µX (σe2 + µe 2 ) ∑pi=1 αi + 3µe ∑pi=1 ∑pj=1 αi αj µX (i − j)
+3µX µe ∑pi=1 σi 2 + µX ∑pi=1 (γi − 3αi σi 2 − αi3 ) + γe

µX (0, k) = ∑pi=1 αi µX (0, k − i) + µe µX (0), k > 0

µX (k, k) = ∑pi=1 ∑pj=1 αi αj µX (k − i, k − j) + ∑pi=1 σi 2 µX (k − i) + 2µe µX (k)

−µX (µe 2 − σe 2 ), k > 0

µX (k, m) = ∑pi=1 αi µX (k, m − i) + µe µX (k), m>k>0

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 10 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Third-order characterization (in terms of cumulants)
[Silva and Oliveira (2004, 2005) and Silva (2005)]
CX (0, 0) = ∑pi=1 ∑pj=1 ∑pk=1 αi αj αk µX (i − j, i − k) + 3 ∑pi=1 ∑pj=1 αj σi 2 µX (i − j) + γe
+3(µe − µX ) ∑pi=1 ∑pj=1 αi αj µX (i − j) + 3µX (µe − µX ) ∑pi=1 σi 2 + 2µX3
−6µe µX 2 ∑pi=1 αi − 3µe (µe 2 + σe2 ) + µX ∑pi=1 (γi − 3αi σi 2 − αi3 )

CX (0, k) = ∑pi=1 αi CX (0, k − i), k>0

CX (k, k) = ∑pi=1 ∑pj=1 αi αj CX (k − i, k − j) + ∑pi=1 σi 2 CX (k − i), k>0

CX (k, m) = ∑pi=1 αi CX (k, m − i), m>k>0

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 11 / 30

 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Third-order characterization (in terms of cumulants)
[Silva and Oliveira (2004, 2005) and Silva (2005)]
CX (0, 0) = ∑pi=1 ∑pj=1 ∑pk=1 αi αj αk µX (i − j, i − k) + 3 ∑pi=1 ∑pj=1 αj σi 2 µX (i − j) + γe
+3(µe − µX ) ∑pi=1 ∑pj=1 αi αj µX (i − j) + 3µX (µe − µX ) ∑pi=1 σi 2 + 2µX3
−6µe µX 2 ∑pi=1 αi − 3µe (µe 2 + σe2 ) + µX ∑pi=1 (γi − 3αi σi 2 − αi3 )

CX (0, k) = ∑pi=1 αi CX (0, k − i), k>0

CX (k, k) = ∑pi=1 ∑pj=1 αi αj CX (k − i, k − j) + ∑pi=1 σi 2 CX (k − i), k>0

CX (k, m) = ∑pi=1 αi CX (k, m − i), m>k>0

INAR processes have a non-linear structure

⇓
1st and 2nd order moments and cumulants are not sufficient to describe dependence structure
INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 11 / 30
 Isabel Silva Magalhães Integer-valued time series

INteger-valued AutoRegressive (INAR) processes

Some extensions of INAR processes and relations with other models

Switching INAR(1) process [Franke and Seligmann (1993)]

Inclusion of explanatory variables [Brännäs (1995)]

Panel data [Brännäs (1994, 1995)]

INteger-valued Moving Average (INMA) model [Al-Osh and Alzaid (1988); Mckenzie (1988);
Brännäs and Hall (2001)]

INteger-valued AutoRegressive-Moving Average (INARMA) model [Mckenzie (1985, 1986);

Al-Osh and Alzaid (1991)]

Multivariate INAR(p) process [Franke and Subba Rao (1995); Latour (1997)]

INAR(p) of Alzaid and Al-Osh (1990) ∼ ARMA(p, p − 1)

Poisson INAR(1) process is a M/M/∞ queueing system observed at regularly spaced interval of
times [Steutel et al. (1983); Mckenzie (1988)]

INAR(p) is a Multitype Branching processes with immigration, BGWI(p) [Dion et al. (1995)]

INteger-valued AutoRegressive (INAR) processes PDMA-UP - October 2009 12 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Time domain
Method of Moments
◮ Second-order: Yule-Walker estimation (YW)
◮ Third-order: Estimation using the Cumulant Third-Order Recursion equation
(TOR),
Least Squares (LS) estimation
◮ Second-order: Conditional Least Squares (CLS)
◮ Third-order: LS estimation based on high-order statistics (LS_HOS),

Frequency domain
Whittle estimation (WHT),

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 13 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Time domain
Method of Moments
◮ Second-order: Yule-Walker estimation (YW)
◮ Third-order: Estimation using the Cumulant Third-Order Recursion equation
(TOR),
Least Squares (LS) estimation
◮ Second-order: Conditional Least Squares (CLS)
◮ Third-order: LS estimation based on high-order statistics (LS_HOS),

Frequency domain
Whittle estimation (WHT),

YW, CLS, WHT, TOR, LS_HOS: do not assume the Poisson distribution for the
arrival process → more adaptive and flexible methods

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 13 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Theorem (Asymptotic distribution of the Yule-Walker estimators of an INAR(p) process [Silva and Silva (2006)])
α the Yule-Walker estimator of α
Let {Xt } be an INAR(p) process and α̂ [Al-Osh and Alzaid

(1987); Du and Li (1991)], that is

    
R̂(0) R̂(1) ··· R̂(p − 1) α̂1 R̂(1)
    

 R̂(1) R̂(0) ··· R̂(p − 2) 
 α̂2   R̂(2) 
  
α = r̂p ⇔ 
R̂p−1α̂ .. .. .. ..  ..  =  . .
 . . . .  .   .. 
    
R̂(p − 1) R̂(p − 2) ··· R̂(0) α̂p R̂(p)

Then
N 1/2 (α̂ − α ) is AN(0p , Vyw ),

where 0n is a vector of n zeros and Vyw = DhT Σ Rr D, for Σ Rr given by i 

Σ Rr (i, j) = Cov(V̂Rr (i), V̂Rr (j)) and DT = − R(1)Ip · · · R(p)Ip (Rp−1 −1 )T ⊗
h i h i
Rp−1 −1 Ip2 0p2 ×p + 0p×p2 Rp−1 −1 , with In the n × n identity matrix,
0n×m the n × m matrix of zeros and ⊗ the Kronecker.
Parameter estimation for INAR(p) processes PDMA-UP - October 2009 14 / 30
 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Theorem (Cumulant Third-Order Recursion (TOR) equation of INAR(p) processes [Silva (2005)])
Let {Xt } be a stationary INAR(p) process. Then the third-order cumulants of Xt can
be written in a single cumulant Third-Order Recursion (TOR) equation by

CX (−k, −m) − ∑i=1 αi CX (i − k, i − m) = δ (k) ∑i=1 σi 2 CX (i − m),

p p

where 0 ≤ k ≤ m, m 6= 0, and δ (a) is the Kronecker delta function.

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 15 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Theorem (Cumulant Third-Order Recursion (TOR) equation of INAR(p) processes [Silva (2005)])
Let {Xt } be a stationary INAR(p) process. Then the third-order cumulants of Xt can
be written in a single cumulant Third-Order Recursion (TOR) equation by

CX (−k, −m) − ∑i=1 αi CX (i − k, i − m) = δ (k) ∑i=1 σi 2 CX (i − m),

p p

where 0 ≤ k ≤ m, m 6= 0, and δ (a) is the Kronecker delta function.

~
w
 CX (−k, −k) = CX (0, k)

    
CX (0, 0) CX (1, 1) ··· CX (p − 1, p − 1) α1 CX (0, 1)
    
 CX (0, 1) CX (0, 0) ··· CX (p − 2, p − 2)   α2   CX (0, 2) 
    
C3,X α =  . . .. .  . = .  = c3,X
 . . . .  .   . 
 . . .  .   . 
CX (0, p − 1) CX (0, p − 2) ··· CX (0, 0) αp CX (0, p)

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 15 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Estimation using the Cumulant TOR equation

(1) (1) (2) (2) (B) (B)
{X1 , . . . , XN=B M } = {X1 , . . . , XM , X1 , . . . , XM , . . . , X1 , . . . , XM },

1 M−k (i)
M ∑j=1 j
(i) (i) (i) (i)
ĈX (k, k) = (X − X )(Xj+k − X )2 , k = 0, . . . , p − 1,
1 M−k (i)
ĈX (0, k) = ∑j=1 (Xj − X )2 (Xj+k − X ),
(i) (i) (i) (i)
k = 1, . . . , p,
M
1 B (i)
B ∑i=1 X
ĈX (k, k) = Ĉ (k, k), k = 0, . . . , p − 1,
1 B (i)
ĈX (0, k) = ∑i=1 ĈX (0, k), k = 1, . . . , p,
B

α = ĉ3,X .
Solve Ĉ3,X α̂

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 16 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Estimation using the Cumulant TOR equation

(1) (1) (2) (2) (B) (B)
{X1 , . . . , XN=B M } = {X1 , . . . , XM , X1 , . . . , XM , . . . , X1 , . . . , XM },

1 M−k (i)
M ∑j=1 j
(i) (i) (i) (i)
ĈX (k, k) = (X − X )(Xj+k − X )2 , k = 0, . . . , p − 1,
1 M−k (i)
ĈX (0, k) = ∑j=1 (Xj − X )2 (Xj+k − X ),
(i) (i) (i) (i)
k = 1, . . . , p,
M
1 B (i)
B ∑i=1 X
ĈX (k, k) = Ĉ (k, k), k = 0, . . . , p − 1,
1 B (i)
ĈX (0, k) = ∑i=1 ĈX (0, k), k = 1, . . . , p,
B

α = ĉ3,X .
Solve Ĉ3,X α̂

Asymptotic distribution: sixth-order moments and cumulants

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 16 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Theorem (Asymptotic distribution of the CLS estimators of an INAR(p) process [Latour (1998)])
Let {X1 , . . . , XN } be observations of an INAR(p) process such that E [e3t ] < ∞ and
(k)
E [(Yi )3 ] < ∞, for k = 1, . . . , p and k ∈ N. Let θ = [ α1 · · · αp µe ]T and let θ̂
denote the Conditional Least Squares estimator of θ , given by
( )
N
QN (θ̂ ) = min{QN (θ )} = min
θ θ
∑ (Xt − α1 Xt−1 − . . . − αp Xt−p − µe ) 2
.
t=p+1

Then
√
N(θ̂ − θ ) ∼ N (0p+1 , V−1 WV−1 ),

where 0p+1 is a (p + 1) × 1 vector of zeros and the entries of V and W are defined by

 µX (i − j) i, j = 1, . . . , p, p
i = p + 1 or j = p + 1, i 6= j, [W]i,j = σe [V]i,j + ∑ σk µX (k − i, k − j).
2 2
[V]i,j = µX
 k=1
1 i = j = p + 1,

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 17 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

LS estimation based on high-order statistics (HOS)
{x1 , x2 , . . . , xn } : realization of a non-negative integer-valued stationary stochastic
process with third-order moments µ (0, k), k > 0

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 18 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

LS estimation based on high-order statistics (HOS)
{x1 , x2 , . . . , xn } : realization of a non-negative integer-valued stationary stochastic
process with third-order moments µ (0, k), k > 0
Approximating model: INAR(p) with parameters α1 , · · · , αp , µe , σe2 and
third-order moments µX (0, k), k > 0, which can be represented in the following
matrix form:
µ 3,X = M3,X α + µe µX (0)1p

m
      
µX (0, 1) µX (0, 0) µX (1, 1) ... µX (p − 1, p − 1) α1 1
      
 µX (0, 2)   µX (0, 1) µX (0, 0) ... µX (p − 2, p − 2)   α2   1 
      
 . = . . .. .  .  + µe µX (0)  . 
 .   . . . .  .   . 
 .   . . .  .   . 
µX (0, p) µX (0, p − 1) µX (0, p − 2) ... µX (0, 0) αp 1
p p
µX (0) = ∑ αi µX (i) + µe µX + Vp , with Vp = σe 2 + µX ∑ σi 2
i=1 i=1

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 18 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Defining H = [M3,X µX (0)1p ] and θ = [ α1 ··· αp µe ]T
⇓

µ 3,X = Hθ

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 19 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Defining H = [M3,X µX (0)1p ] and θ = [ α1 ··· αp µe ]T
⇓

µ 3,X = Hθ

θ may be estimated by least squares, ie, minimizing the squared error between µ 3,X
and the third-order moments of the data:

µ 3 = [ µ (0, 1) · · · µ (0, p) ]T

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 19 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Defining H = [M3,X µX (0)1p ] and θ = [ α1 ··· αp µe ]T
⇓

µ 3,X = Hθ

θ may be estimated by least squares, ie, minimizing the squared error between µ 3,X
and the third-order moments of the data:

µ 3 = [ µ (0, 1) · · · µ (0, p) ]T

Least Squares estimator of θ using HOS (LS_HOS)

θ̂ = min {L∗ (θ )} = min {(µ 3 − Hθ )T (µ 3 − Hθ )}

θ θ

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 19 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Defining H = [M3,X µX (0)1p ] and θ = [ α1 ··· αp µe ]T
⇓

µ 3,X = Hθ

θ may be estimated by least squares, ie, minimizing the squared error between µ 3,X
and the third-order moments of the data:

µ 3 = [ µ (0, 1) · · · µ (0, p) ]T

Least Squares estimator of θ using HOS (LS_HOS)

θ̂ = min {L∗ (θ )} = min {(µ 3 − Hθ )T (µ 3 − Hθ )}

θ θ

In practice: θ̂ = min {L̂∗ (θ )} = min {(µ̂ 3 − Ĥθ )T (µ̂ 3 − Ĥθ )}

θ θ

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 19 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Spectral density function easy to obtain
[Silva (2005)]
{Xt } is an INAR process −−−−−−−→ {Xt } is a Non-Gaussian Mixing process:
◮ {Xt } is strictly stationary,
◮ E[|Xt |k ] < ∞, t ∈ Z, k ∈ N,
∞ ∞
◮
∑ ... ∑ |CX (s1 , . . . , sk−1 )| < ∞, k ≥ 2, (mixing condition)
s1 =−∞ sk−1 =−∞

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 20 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Spectral density function easy to obtain
[Silva (2005)]
{Xt } is an INAR process −−−−−−−→ {Xt } is a Non-Gaussian Mixing process:
◮ {Xt } is strictly stationary,
◮ E[|Xt |k ] < ∞, t ∈ Z, k ∈ N,
∞ ∞
◮
∑ ... ∑ |CX (s1 , . . . , sk−1 )| < ∞, k ≥ 2, (mixing condition)
s1 =−∞ sk−1 =−∞

 2
1  N 
−iωj t  f (ωj ) 2
IN (ωj ) =  ∑ Xt e  ∼ χ2 ,
2π N t=1  2

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 20 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Spectral density function easy to obtain
[Silva (2005)]
{Xt } is an INAR process −−−−−−−→ {Xt } is a Non-Gaussian Mixing process:
◮ {Xt } is strictly stationary,
◮ E[|Xt |k ] < ∞, t ∈ Z, k ∈ N,
∞ ∞
◮
∑ ... ∑ |CX (s1 , . . . , sk−1 )| < ∞, k ≥ 2, (mixing condition)
s1 =−∞ sk−1 =−∞

 2
1  N 
−iωj t  f (ωj ) 2
IN (ωj ) =  ∑ Xt e  ∼ χ2 ,
2π N t=1  2

Whittle estimator [Silva and Oliveira (2004, 2005)]

  
IN (ωj )
∑
[N/2]
θ̃θ = min{L(X)}
e = min log(f (ωj )) +
θ θ j=1 f (ωj )

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 20 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

Spectral density function easy to obtain
[Silva (2005)]
{Xt } is an INAR process −−−−−−−→ {Xt } is a Non-Gaussian Mixing process:
◮ {Xt } is strictly stationary,
◮ E[|Xt |k ] < ∞, t ∈ Z, k ∈ N,
∞ ∞
◮
∑ ... ∑ |CX (s1 , . . . , sk−1 )| < ∞, k ≥ 2, (mixing condition)
s1 =−∞ sk−1 =−∞

 2
1  N 
−iωj t  f (ωj ) 2
IN (ωj ) =  ∑ Xt e  ∼ χ2 ,
2π N t=1  2

Whittle estimator [Silva and Oliveira (2004, 2005)]

  
IN (ωj )
∑
[N/2]
θ̃θ = min{L(X)}
e = min log(f (ωj )) +
θ θ j=1 f (ωj )

Asymptotic variance: Fourth-order cumulant spectral density function

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 20 / 30
 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

µ̂e = X 1 − ∑i=1 α̂i , σ̂e2 = V̂p − X ∑i=1 σ̂i2 ,
p p

where V̂p = R̂(0) − ∑pi=1 α̂i R̂(i) and σ̂i2 is an estimator of Var[Yj,i ], i = 1, . . . , p. For
instance, σ̂i2 = α̂i (1 − α̂i ), for the binomial thinning operation.

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 21 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

µ̂e = X 1 − ∑i=1 α̂i , σ̂e2 = V̂p − X ∑i=1 σ̂i2 ,
p p

where V̂p = R̂(0) − ∑pi=1 α̂i R̂(i) and σ̂i2 is an estimator of Var[Yj,i ], i = 1, . . . , p. For
instance, σ̂i2 = α̂i (1 − α̂i ), for the binomial thinning operation.

Simulation results
Non-admissible estimates → constrained estimation (CLS, WHT, LS_HOS)

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 21 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

µ̂e = X 1 − ∑i=1 α̂i , σ̂e2 = V̂p − X ∑i=1 σ̂i2 ,
p p

where V̂p = R̂(0) − ∑pi=1 α̂i R̂(i) and σ̂i2 is an estimator of Var[Yj,i ], i = 1, . . . , p. For
instance, σ̂i2 = α̂i (1 − α̂i ), for the binomial thinning operation.

Simulation results
Non-admissible estimates → constrained estimation (CLS, WHT, LS_HOS)
Sample bias, variance, mean square error and univariate skewness decrease as
the sample size increases ⇒ consistency and symmetry

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 21 / 30

 Isabel Silva Magalhães Integer-valued time series

Parameter estimation for INAR(p) processes

µ̂e = X 1 − ∑i=1 α̂i , σ̂e2 = V̂p − X ∑i=1 σ̂i2 ,
p p

where V̂p = R̂(0) − ∑pi=1 α̂i R̂(i) and σ̂i2 is an estimator of Var[Yj,i ], i = 1, . . . , p. For
instance, σ̂i2 = α̂i (1 − α̂i ), for the binomial thinning operation.

Simulation results
Non-admissible estimates → constrained estimation (CLS, WHT, LS_HOS)
Sample bias, variance, mean square error and univariate skewness decrease as
the sample size increases ⇒ consistency and symmetry
LS_HOS, WHT and CLS provides good results
◮ in terms of smaller sample bias, variance and mean square error

Parameter estimation for INAR(p) processes PDMA-UP - October 2009 21 / 30

 Isabel Silva Magalhães Integer-valued time series

Application to real data

Poliomyelitis incidence in the United States
14

Monthly number of U.S. cases of 12

poliomyelitis, from 1970 to 1983 10

monthly counts
8
[Zeger (1988): Parameter-driven
6
model], and sample
4
autocorrelation and partial 2

autocorrelation functions 0
1970 1972 1974 1976 1978 1980 1982 1984
year

1.2 0.3

1
0.2
0.8

0.1
0.6
ρ(k)

φ(k)

0.4
0

0.2
−0.1
0

−0.2 −0.2
0 5 10 15 20 25 30 0 5 10 15 20 25 30
k k

Application to real data PDMA-UP - October 2009 22 / 30

 Isabel Silva Magalhães Integer-valued time series

Application to real data

Poliomyelitis incidence in the United States
X = 1.33 and S2 = 3.48
↓
INAR(1) with binomial thinning operation and discrete arrival process
Xt = α ◦ Xt−1 + et , t = 2, . . . , 168, E[et ] = µe , Var[et ] = σe 2 ,

Application to real data PDMA-UP - October 2009 23 / 30

 Isabel Silva Magalhães Integer-valued time series

Application to real data

Poliomyelitis incidence in the United States
X = 1.33 and S2 = 3.48
↓
INAR(1) with binomial thinning operation and discrete arrival process
Xt = α ◦ Xt−1 + et , t = 2, . . . , 168, E[et ] = µe , Var[et ] = σe 2 ,

Method α̂ µ̂e σ̂e2

YW 0.2948 0.9403 2.9041
CLS 0.3063 0.9414 2.8862
WHT 0.2799 0.9601 2.9279
LS_HOS 0.2083 0.9277 3.0504
LS_HOS_C 0.2344 0.7477 3.0040
TOR_1B 0.1475 1.1367 3.1650
TOR_2B 0.1431 1.1425 3.1737

Table: Parameter estimates of the INAR(1) model fitted to the polio data.
Application to real data PDMA-UP - October 2009 23 / 30
 Isabel Silva Magalhães Integer-valued time series

Application to real data

Number of plants within the industrial sector

50

45
The number of Swedish
40
mechanical paper and
35

Number of plants
pulp mills, from 1921 to 30

1981 [Brännäs (1995) 25

and Brännäs and 20

Hellström (2001): 15

Explanatory variables] 10

5
1920 1930 1940 1950 1960 1970 1981

Application to real data PDMA-UP - October 2009 24 / 30

 Isabel Silva Magalhães Integer-valued time series

Application to real data

Simple INAR(1)
It is not assumed the Poisson distribution for the arrival process:

X = 20.40 and S2 = 155.16

Application to real data PDMA-UP - October 2009 25 / 30

 Isabel Silva Magalhães Integer-valued time series

Application to real data

Simple INAR(1)
It is not assumed the Poisson distribution for the arrival process:

X = 20.40 and S2 = 155.16

Method α̂ µ̂e σ̂e2 µ̂x σ̂x2 MSE

CLS 0.9591 0.2017 15.2268 4.9279 192.2764 9.3254
LS_HOS 0.9269 1.3635 19.2253 18.6516 145.4513 9.2997
TOR_1B 0.9631 0.7518 14.7219 20.374 213.5073 8.9224

Application to real data PDMA-UP - October 2009 25 / 30

 Isabel Silva Magalhães Integer-valued time series

Application to real data

Simple INAR(1)
It is not assumed the Poisson distribution for the arrival process:

X = 20.40 and S2 = 155.16

Method α̂ µ̂e σ̂e2 µ̂x σ̂x2 MSE

CLS 0.9591 0.2017 15.2268 4.9279 192.2764 9.3254
LS_HOS 0.9269 1.3635 19.2253 18.6516 145.4513 9.2997
TOR_1B 0.9631 0.7518 14.7219 20.374 213.5073 8.9224

µ̂e (1 − α̂ )(µ̂e α̂ + σ̂e2 )

Mean and variance of the estimated models: µ̂x = and σ̂x2 =
1 − α̂ (1 − α̂ )2 (1 + α̂ )

Application to real data PDMA-UP - October 2009 25 / 30

 Isabel Silva Magalhães Integer-valued time series

Application to real data

Simple INAR(1)
It is not assumed the Poisson distribution for the arrival process:

X = 20.40 and S2 = 155.16

Method α̂ µ̂e σ̂e2 µ̂x σ̂x2 MSE

CLS 0.9591 0.2017 15.2268 4.9279 192.2764 9.3254
LS_HOS 0.9269 1.3635 19.2253 18.6516 145.4513 9.2997
TOR_1B 0.9631 0.7518 14.7219 20.374 213.5073 8.9224

µ̂e (1 − α̂ )(µ̂e α̂ + σ̂e2 )

Mean and variance of the estimated models: µ̂x = and σ̂x2 =
1 − α̂ (1 − α̂ )2 (1 + α̂ )

MSE between the observations and the fitted models based on TOR_1B, LS_HOS and
CLS estimates
Application to real data PDMA-UP - October 2009 25 / 30
 Isabel Silva Magalhães Integer-valued time series

Application to real data

50
Real data
45 CLS
LS_HOS
40

The number of plants and 35

Number of plants
30
the fitted values
25
considering the LS_HOS
20
and CLS estimates
15

10

5
1920 1930 1940 1950 1960 1970 1981

Application to real data PDMA-UP - October 2009 26 / 30

 Isabel Silva Magalhães Integer-valued time series

Recent developments

Overdispersion → new thinning operations and/or different distributions for the

arrival processes
Extreme value theory
INAR with periodic structure
Outliers
Forecasting
Heteroskedasticity
Random-coefficient INAR

Recent developments PDMA-UP - October 2009 27 / 30

 Isabel Silva Magalhães Integer-valued time series

Theme proposal

Random-coefficient integer-valued autoregressive processes

Theme proposal PDMA-UP - October 2009 28 / 30

 Isabel Silva Magalhães Integer-valued time series

Theme proposal

Random-coefficient integer-valued autoregressive processes

Describe the first-order random-coefficient integer-valued autoregressive,

RCINAR(1), process proposed by Gomes and Canto e Castro (2009) and Zheng et al.
(2007) and explain the principal differences/similarities between these two processes.

Gomes, D., Canto e Castro, L., 2009. Generalized integer-valued random coefficient for a first order
structure autoregressive (RCINAR) process. Journal of Statistical Planning and Inference, vol. 139
(12), pp. 4088–4097.

Zheng, H., Basawa, I. V., Datta, S., 2007. First-order random coefficient integer-valued
autoregressive processes. Journal of Statistical Planning and Inference, vol. 137 (1), pp. 212–229.

Theme proposal PDMA-UP - October 2009 28 / 30

 Isabel Silva Magalhães
References I
Al-Osh, M.A. and Alzaid, A.A., 1987.
First-order integer-valued autoregressive (INAR(1))
process.
Journal of Time Series Analysis, vol. 8, pp. 261–275.
Al-Osh, M.A. and Alzaid, A.A., 1988.
Integer-valued moving average (INMA) process.
Statistical Papers, vol. 29, pp. 281–300.
Al-Osh, M.A. and Alzaid, A.A., 1991.
Binomial autoregressive moving average models.
Communications in Statistics: Stochastic Models, vol. 7,
pp. 261–282.
Alzaid, A.A. and Al-Osh, M.A., 1990.
An integer-valued pth-order autoregressive structure
(INAR(p)) process.
Journal of Applied Probability, vol. 27, pp. 314–324.
Brännäs, K., 1994.
Estimation and testing in integer-valued AR(1) models.
Technical Report, Umeå University, Sweden, 335.
Brännäs, K., 1995.
Explanatory variables in the AR(1) count data model.
Technical Report, Umeå University, Sweden, 381.
References
Integer-valued time series
Brännäs, K. and Hall, A., 2001.
Estimation in integer-valued moving average models.
Applied Stochastic Models in Business and Industry, vol.
17, pp. 277–291.
Dion, J-P. and Gauthier, G. and Latour, A., 1995.
Branching processes with immigration and integer-valued
time series.
Serdica Mathematical Journal, vol. 21, pp. 123–136.
Du, Jin-Guan and Li, Yuan, 1991.
The integer-valued autoregressive (INAR(p)) model.
Journal of Time Series Analysis, vol. 12, pp. 129–142.
Franke, J. and Seligmann, T., 1993.
Conditional maximum likelihood estimates for INAR(1)
processes and their application to modelling epileptic
seizure counts.
In Developments in Time Series Analysis: in honour of
Maurice B. Priestley, Chapman & Hall, pp. 310–330.
Franke, J. and Subba Rao, T., 1995.
Multivariate first order integer valued autoregressions.
Technical report, Math. Dep., UMIST.
PDMA-UP - October 2009 29 / 30
 Isabel Silva Magalhães
References II
Gauthier, G. and Latour, A., 1994.
Convergence forte des estimateurs des paramètres dt’un
processus GENAR(p).
Annales des Sciences Mathématiques du Québec, vol. 18,
pp. 49–71
Latour, A., 1997.
The multivariate GINAR(p) process.
Advances in Applied Probability, vol. 29, pp. 228–248.
Latour, A., 1998.
Existence and stochastic structure of a non-negative
integer-valued autoregressive process.
Journal of Time Series Analysis, vol. 19, pp. 439–455.
McKenzie, E., 1985.
Some simple models for discrete variate time series.
Water Resources Bulletin, vol. 21, pp. 645–650.
McKenzie, E., 1986.
Autoregressive moving-average processes with
negative-binomial and geometric marginal distributions.
Advances in Applied Probability, vol. 18, pp. 679–705.
McKenzie, E., 1988.
Some ARMA models for dependent sequences of Poisson
counts.
Advances in Applied Probability, vol. 20, pp. 822–835.
References
Integer-valued time series
Silva, M.E. and Oliveira, V.L., 2004.
Difference equations for the higher-order moments and
cumulants of the INAR(1) model.
Journal of Time Series Analysis, vol. 25, pp. 317–333.
Silva, M.E. and Oliveira, V.L., 2005.
Difference equations for the higher-order moments and
cumulants of the INAR(p) model.
Journal of Time Series Analysis, vol. 26, pp. 17–36.
Silva, I. e Silva, M.E., 2006.
Asymptotic distribution of the Yule-Walker estimator for
INAR(p) processes.
Statistics & Probability Letters, vol. 76, pp. 1655-1663.
Steutel, F.W. and Van Harn, K., 1979.
Discrete analogues of self-decomposability and stability.
The Annals of Probability, vol. 7, pp. 893–899.
Steutel, F.W. and Vervaat, W. and Wolfe, S.J., 1983.
Integer valued branching processes with immigration.
Advances in Applied Probability, vol. 15, pp. 713–725.
Zeger, S.L., 1988.
A regression model for time series of counts.
Biometrika, vol. 75, pp. 621–629.
PDMA-UP - October 2009 30 / 30