5.

2 Modified Likelihood
There are 3 commonly used modified likelihood in solving nuisance parameter problem. They are z Marginal likelihood z Conditional likelihood z Profile likelihood

(a)

Marginal likelihood

This likelihood function is based on a suitably chosen subset of the complete data vector. The nuisance parameter can be eliminated by working with the marginal likelihood function.
Example:

Y1 Y Y = 2 ~ N ( X , ( )), i.e., Y = X + , ~ N (0, ( )) M , Yn

where

Y 1 , Y 2 , K , Y n are observations at spatial locations,

s 1 , s 2 , K , s n and (
parameterized by

is a known covariance function

. For example,

si s j = ij ( ) , ij ( ) = ij ( 1 , 2 ) = 12 exp 2

In this area,

is the parameter of interest and

is regarded as

a nuisance parameter. First, we can use the transformed response

 R1 R R = 2 = I X X tX M Rn

X t Y = (I P )Y residuals

Then,

E (R ) = 0, V (R ) = (I P )( )(I P ) .
Note the density function of

R 1 , R 2 , K , R n is independent of

and only depends on

. However, the covariance matrix is singular. Therefore, we choose the linearly independent column vectors

V (R ) = (I P )( )(I P )
matrix K including the of

n p

I P . Then, the new response

R 1 R2 t = K ( n p ) n Y = M R n p
since

with

t t K t X = 0 . Then, K Y ~ N (0, K ( )K )

E K tY = K t E (Y ) = 0, V K tY = K tV (Y )K = K t ( )K .
Then, the marginal log-likelihood function is

l ( )

1 1 1 log det K t ( )K y t K K t ( )K K t y 2 2 1 1 1 = log [det [ ( )]] log det X t 1 ( )X y t Sy 2 2 2

[ [

]]

[ [

]]

where

1 1 1 log det K t ( )K = log [det [ ( )]] log det X t 1 ( )X 2 2 2

and

[ [

]]

[ [

]]

K K t ( )K

] [

K t = S = 1 ( ) 1 ( )X X t 1 ( )X

X t 1 ( )

(i.e., by the following theorem: if

KtX = 0 ,

where

Kt

has

maximum row rank and V is positive definite, then

K K tVK
) Since

K t = V 1 V 1 X X tV 1 X

X tV 1

(I P )K = K

1 t y K K t ( )K 2

Kty = =

1 t y (I P )K K t ( )K 2

K t (I P ) y

1 t R K K t ( )K 2

KtR

then

l ( )

1 log [det [ ( 2

)]] 1 log [det [X t 1 ( )X ]] 1 R t SR

2 2

Note:
The marginal log-likelihood is sometimes called the restricted log-likelihood. The maximum likelihood estimate obtained based on the restricted log-likelihood is called restricted (or residual) maximum likelihood estimate (REMLE).

Note:
As =

2 I , the marginal log-likelihood is

1 (n p )log 2 1 RSS 2 2 2

( )

where RSS is the residual sum of square. The above marginal log-likelihood is also equal to the marginal likelihood derived from the transformed response
2 RSS ~ 2 n p .