Lec15order Stat

Section 4.
6 Order Statistics
Order Statistics
Let X1 , X2 , X3 , X4 , X5 be iid random variables with a distribution F with a

Lecture 15: Order Statistics range of (a, b). We can relabel these X’s such that their labels correspond
to arranging them in increasing order so that
Statistics 104 X(1) ≤ X(2) ≤ X(3) ≤ X(4) ≤ X(5)
Colin Rundel
X(1) X(2) X(3) X(4) X(5)
March 14, 2012 a X5 X1 X4 X2 X3 b
In the case where the distribution F is continuous we can make the

stronger statement that
X(1) < X(2) < X(3) < X(4) < X(5)

Since P(Xi = Xj ) = 0 for all i 6= j for continuous random variables.
Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 1 / 24
Section 4.6 Order Statistics Section 4.6 Order Statistics
Order Statistics, cont. Notation Detour
For X1 , X2 , . . . , Xn iid random variables Xk is the kth smallest X , usually For a continuous random variable we can see that
called the kth order statistic.
f (x) ≈ P(x ≤ X ≤ x + ) = P(X ∈ [x, x + ])
X(1) is therefore the smallest X and lim f (x) = lim P(X ∈ [x, x + ])
→0 →0
f (x) = lim P(X ∈ [x, x + ])/

X(1) = min(X1 , . . . , Xn ) →0
Similarly, X(n) is the largest X and

P (x  X  x + ✏) = P (X 2 [x, x + ✏])
X(n) = max(X1 , . . . , Xn )
f (x) f (x + ✏)
x x+✏
Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 2 / 24 Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 3 / 24
Density of the maximum Density of the minimum
For X1 , X2 , . . . , Xn iid continuous random variables with pdf f and cdf F For X1 , X2 , . . . , Xn iid continuous random variables with pdf f and cdf F
the density of the maximum is the density of the minimum is
P(X(n) ∈ [x, x + ]) = P(one of the X ’s ∈ [x, x + ] and all others < x) P(X(1) ∈ [x, x + ]) = P(one of the X ’s ∈ [x, x + ] and all others > x)
n
X n
X
= P(Xi ∈ [x, x + ] and all others < x) = P(Xi ∈ [x, x + ] and all others > x)
i=1 i=1
= nP(X1 ∈ [x, x + ] and all others < x) = nP(X1 ∈ [x, x + ] and all others > x)
= nP(X1 ∈ [x, x + ])P(all others < x) = nP(X1 ∈ [x, x + ])P(all others > x)
= nP(X1 ∈ [x, x + ])P(X2 < x) · · · P(Xn < x) = nP(X1 ∈ [x, x + ])P(X2 > x) · · · P(Xn > x)
= nf (x)F (x)n−1 = nf (x)(1 − F (x))n−1
f(n) (x) = nf (x)F (x)n−1 f(1) (x) = nf (x)(1 − F (x))n−1
Density of the kth Order Statistic Cumulative Distribution of the min and max
For X1 , X2 , . . . , Xn iid continuous random variables with pdf f and cdf F For X1 , X2 , . . . , Xn iid continuous random variables with pdf f and cdf F
the density of the kth order statistic is the density of the kth order statistic is
P(X(k) ∈ [x, x + ]) = P(one of the X ’s ∈ [x, x + ] and exactly k − 1 of the others < x) F(1) (x) = P(X(1) < x) = 1 − P(X(1) > x)
n
X = 1 − P(X1 > x, . . . , Xn > x) = 1 − P(X1 > x) · · · P(Xn > x)
= P(Xi ∈ [x, x + ] and exactly k − 1 of the others < x) = 1 − (1 − F (x))n
i=1
= nP(X1 ∈ [x, x + ] and exactly k − 1 of the others < x)

F(n) (x) = P(X(n) < x) = 1 − P(X(n) > x)
= nP(X1 ∈ [x, x + ])P(exactly k − 1 of the others < x)
! ! ! = P(X1 < x, . . . , Xn < x) = P(X1 < x) · · · P(Xn < x)
n−1 n−1 n
= nP(X1 ∈ [x, x + ]) k−1
P(X < x) P(X > x) n−k
= nf (x) F (x) = (1
k−1 F (x)
− F (x))n−k
k −1 k −1
!
n−1 d dF (x)
f(k) (x) = nf (x) F (x)k−1 (1 − F (x))n−k f(1) (x) = (1 − F (x))n = n(1 − F (x))n−1 = nf (x)(1 − F (x))n−1
k −1 dx dx
d dF (x)
f(n) (x) = F (x)n = nF (x)n−1 = nf (x)F (x)n−1
dx dx
Order Statistic of Standard Uniforms Beta Distribution

iid The Beta distribution is a continuous distribution defined on the range
Let X1 , X2 , . . . , Xn ∼ Unif(0, 1) then the density of X(n) is given by
(0, 1) where the density is given by
1
!
f(k) (x) = nf (x)
n−1
F (x)k−1 (1 − F (x))n−k f (x) = x r −1 (1 − x)s−1
k −1 B(r , s)
(
n−1
n k−1
k−1
x (1 − x)n−k if 0 < x < 1 where B(r , s) is called the Beta function and it is a normalizing constant
= which ensures the density integrates to 1.
0 otherwise
Z 1
This is an example of the Beta distribution where r = k and s = n − k + 1. 1= f (x)dx
0
Z 1
1
1= x r −1 (1 − x)s−1 dx
X(k) ∼ Beta(k, n − k + 1) 0 B(r , s)
Z 1
1
1= x r −1 (1 − x)s−1 dx
B(r , s) 0
Z 1
B(r , s) = x r −1 (1 − x)s−1 dx
0
Beta Function Beta Function - Expectation
The connection between the Beta distribution and the kth order statistic Let X ∼ Beta(r , s) then
of n standard Uniform random variables allows us to simplify the Beta
function. 1
Z
1
E (X ) = x x r −1 (1 − x)s−1 dx
0 B(r , s)
Z
Z 1 1
B(r , s) = x r −1 (1 − x)s−1 dx = , 1x (r +1)−1 (1 − x)s−1 dx
B(r , s) 0
0
1 B(r + 1, s)
B(k, n − k + 1) = =
n−1 B(r , s)

n k−1
(k − 1)!(n − 1 − k + 1)! r !(s − 1)! (r + s − 1)!
= =
n(n − 1)! (r + s)! (r − 1)!(s − 1)!
(r − 1)!(n − k)! r! (r + s − 1)!
= =
n! (r − 1)! (r + s)!
r
(r − 1)!(s − 1)! Γ(r )Γ(s) =
= = r +s
(r + s − 1)! Γ(r + s)
Beta Function - Variance Beta Distribution - Summary
Let X ∼ Beta(r , s) then If X ∼ Beta(r , s) then

Z 1
1 1
E (X 2 ) = x r −1 (1 − x)s−1 dx
x2 f (x) = x r −1 (1 − x)s−1
B(r , s)
0 B(r , s)
Z x
B(r + 2, s) (r + 1)!(s − 1)! (r + s − 1)! 1 Bx (r , s)
= = F (x) = x r −1 (1 − x)s−1 dx =
B(r , s) (r + s + 1)! (r − 1)!(s − 1)! 0 B(r , s) B(r , s)
(r + 1)r
=
(r + s + 1)(r + s) Z 1
(r − 1)!(s − 1)! Γ(r )Γ(s)
B(r , s) = x r −1 (1 − x)s−1 dx = =
(r + s − 1)! Γ(r + s)
Var (X ) = E (X 2 ) − E (X )2 Z0 x
(r + 1)r r2 Bx (r , s) = x r −1 (1 − x)s−1 dx
= − 0
(r + s + 1)(r + s) (r + s)2
(r + 1)r (r + s) − r 2 (r + s + 1)
= r
(r + s + 1)(r + s)2 E (X ) =
r +s
rs rs
= Var (X ) =
(r + s + 1)(r + s)2
(r + s)2 (r + s + 1)
Minimum of Exponentials Maximum of Exponentials

iid iid
Let X1 , X2 , . . . , Xn ∼ Exp(λ), we previously derived a more general result Let X1 , X2 , . . . , Xn ∼ Exp(λ) then the density of X(n) is given by
where the X ’s were not identically distributed and showed that
min(X1 , . . . , Xn ) ∼ Exp(λ1 + · · · + λn ) = Exp(nλ) in this more restricted f(n) (x) = nf (x)F (x)n−1
case. n−1
= n λe −λx 1 − e −λx
Lets confirm that result using our new more general methods
f(1) (x) = nf (x)(1 − F (x))n−1

n−1 Which we can’t do much with, instead we can try the cdf of the maximum.
= n λe −λx 1 − [1 − e −λx ]
n−1
= nλe −λx e −λx ]
n
= nλ e −λx ]
= nλe −nλx
Which is the density for Exp(nλ).

Maximum of Exponentials, cont. Limit Distributions of Maxima and Minima

iid
Let X1 , X2 , . . . , Xn ∼ Exp(λ) then the cdf of X(n) is given by Previous we have shown that
F(n) (x) = F (x)n

n F(1) (x) = P(X(1) < x) = 1 − (1 − F (x))n
= 1 − e −λx
n F(n) (x) = P(X(n) < x) = F (x)n
ne −λx

= 1−
n
F(n) (x) ≈ exp(−ne −λx ) When n tends to infinity we get
lim F(n) (x) = lim exp(−ne −λx ) = 0 (

n→∞ n→∞
0 if F (x) = 0
lim F(1) (x) = lim 1 − (1 − F (x))n =
n→∞ n→∞ 1 if F (x) > 0
This result is not unique to the exponential distribution... (
1 if F (x) = 1
lim F(n) (x) = lim F (x)n =
n→∞ n→∞ 0 if F (x) < 1
Limit Distributions of Maxima and Minima, cont. Maximum of Exponentials, cont.

iid
These results show that the limit distributions are degenerate as they only Let X1 , X2 , . . . , Xn ∼ Exp(λ) and an = log(n)/λ, bn = 1/λ then the cdf of
take values of 0 or 1. To avoid the degeneracy we would like to use a X(n) is given by
simple transform that such that the limit distributions are not degenerate.
F(n) (an + bn x) = F ((log(n) + x)/λ)n
Let’s consider simple linear transformations n
= 1 − e −λ(log(n)+x)/λ
n
lim F(n) (an + bn x) = lim F (an + bn x)n = F 0 (x) = 1 − e − log(n) e −x
n→∞ n→∞ n
lim F(1) (cn + dn x) = lim 1 − (1 − F (cn + dn x))n = F 00 (x) = 1 − e −x /n
n→∞ n→∞
lim F(n) (an + bn x) = exp(−e −x )

X(n) − an

n→∞
F(n) (an + bn x) = P(X(n) < an + bn x) = P <x
bn
X(1) − cn

F(1) (cn + dn x) = P(X(1) < cn + dn x) = P <x This is known as the standard Gumbel distribution.
dn
Gumbel Distribution Maximum of Exponentials, cont.

iid
Let X ∼ Gumbel(0, 1) then Let X1 , X2 , . . . , Xn ∼ Exp(λ) and an = log(n)/λ, bn = 1/λ then if n is
large we can use the Standard Gumbel to calculate properties of X(n) .
Standard Gumbel Distribution - pdf Standard Gumbel Distribution - cdf
1.0
Median(X(n) ) = m(n)
0.3
P(X(n) < m(n) ) = 1/2
0.8
0.6
0.2
X(n) − an

F(x)
f(x)
P < mG = 1/2
0.4
bn
0.1
P(X(n) < an + bn mG ) = 1/2
0.2
0.0
0.0
-5 0 5 10 -5 0 5 10
x x m(n) = an + bn mG
−x √ 1 1
F (x) = exp(−e ) E (X ) = π/ 6 = log n − log log 2
−x λ λ
f (x) = e exp(−e −x ) Median(X ) = − log(log(2))
Maximum of Exponentials, Example Maximum of Uniforms

iid
In 2009 Usain Bolt broke the world record in the 100 meters with a time of Let X1 , X2 , . . . , Xn ∼ Unif(0, 1) and an = 1, bn = 1/n then the cdf of X(n)
9.58 seconds in Berlin, Germany. If we imagine that the running speed in is given by
m/s of competitive sprinters is given by an Exponential distribution with
λ = 1. How many sprinters would need to run to have a 50/50 chance of F(n) (x) = F (x)n
beating Usian Bolt’s record (having a faster running speed)? = xn
Let X ∼ Exp(1) then we need to find n such that

F(n) (an + bn x) = F (an + bn x)n
Median(X(n) ) ≥ 100/9.58
= (1 + x/n)n
lim F(n) (an + bn x) = e x
n→∞
This is example of the Reverse Weibul distribution.
Section 4.6 Order Statistics
Maximum of Paretos
This is a distribution we have not seen yet, but is useful for describing many physical
processes. It’s key feature is that it has long tails meaning it goes to 0 slower than a
distribution like the normal.
iid
Let X1 , X2 , . . . , Xn ∼ Pareto(α, k) and an = 0, bn = kn1/α then the cdf of X is
( α
1 − kx if x ≥ k,
FX (x) =
0 otherwise
The cdf of X(n) is then given by
α n
n k
F(n) (x) = F (x) = 1 −
x
α n n
x −α

k
F(n) (an + bn x) = F (an + bn x)n = 1 − = 1 −
kxn1/α n
α
lim F(n) (an + bn x) = e −x
n→∞
This is example of the Fréchet distribution.

Statistics 104 (Colin Rundel) Lecture 15 March 14, 2012 24 / 24

Lec15order Stat

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lec15order Stat

Uploaded by

Copyright:

Available Formats

Section 4.

Let X1 , X2 , X3 , X4 , X5 be iid random variables with a distribution F with a

Statistics 104 X(1) ≤ X(2) ≤ X(3) ≤ X(4) ≤ X(5)

March 14, 2012 a X5 X1 X4 X2 X3 b

In the case where the distribution F is continuous we can make the

X(1) < X(2) < X(3) < X(4) < X(5)

Section 4.6 Order Statistics Section 4.6 Order Statistics

Order Statistics, cont. Notation Detour

f (x) = lim P(X ∈ [x, x + ])/

Similarly, X(n) is the largest X and

Density of the maximum Density of the minimum

f(n) (x) = nf (x)F (x)n−1 f(1) (x) = nf (x)(1 − F (x))n−1

Section 4.6 Order Statistics Section 4.6 Order Statistics

= nP(X1 ∈ [x, x + ] and exactly k − 1 of the others < x)

Order Statistic of Standard Uniforms Beta Distribution

Section 4.6 Order Statistics Section 4.6 Order Statistics

Beta Function Beta Function - Expectation

Beta Function - Variance Beta Distribution - Summary

Let X ∼ Beta(r , s) then If X ∼ Beta(r , s) then

Section 4.6 Order Statistics Section 4.6 Order Statistics

Minimum of Exponentials Maximum of Exponentials

f(1) (x) = nf (x)(1 − F (x))n−1

Which is the density for Exp(nλ).

Maximum of Exponentials, cont. Limit Distributions of Maxima and Minima

F(n) (x) = F (x)n

lim F(n) (x) = lim exp(−ne −λx ) = 0 (

Section 4.6 Order Statistics Section 4.6 Order Statistics

Limit Distributions of Maxima and Minima, cont. Maximum of Exponentials, cont.

lim F(n) (an + bn x) = exp(−e −x )

Gumbel Distribution Maximum of Exponentials, cont.

P(X(n) < m(n) ) = 1/2

P(X(n) < an + bn mG ) = 1/2

Section 4.6 Order Statistics Section 4.6 Order Statistics

Maximum of Exponentials, Example Maximum of Uniforms

Let X ∼ Exp(1) then we need to find n such that

This is example of the Reverse Weibul distribution.

This is example of the Fréchet distribution.

You might also like

f (x) = lim P(X ∈ [x, x + ])/

= nP(X1 ∈ [x, x + ] and exactly k − 1 of the others < x)