MultilevelPRI Workshop

PRI workshop
Multilevel modeling: A comparison of software packages
Juan Leon Jara Almonte

Pennsylvania State University
814 Oswald
jjl292@psu.edu
March 2009
I. Multilevel Modeling
Multilevel models are statistical models used to analyze data that have a hierarchical or
nested structure, for example:
-
Students nested in schools

Children nested in households
Patients nested in hospitals
Criminals nested in jails
Repeated measures nested in individuals (e.g.: students, patients)
Thus, to address this issues, using a multilevel approach will be a more accurate model
than a ordinary least squares regression (OLS). The names received for this model in the
different fields or literature around this topic is: random coefficient model, variance
component model, and hierarchical linear model.
The basic two level model:
Level 1
Yij
0j + 1jXij + rij
- equation 1
00 + 01Zj + u0j
10
- equation 2
- equation 3
Level 2
0j
1j
=
=
Substitution of ( 2 ) and ( 3 ) in ( 1 )
Yij
00 + 10Xij + 01Zj + rij + u0j
- equation 4
Where
i
j
0j
00
Xij
10
rij and u0j
: subjects
: groups
: the mean of Yij for group j
: the grand mean of Yij
: the predictor variable for the subject i in the group j
: the fixed effect of the predictor variable Xij across groups
: random components normally distributed and independent of each other
We could add more predictors at level 1 and we could add random components at level 2.
However, the basic idea of multilevel modeling is to decompose the component variance
in two non-observables: one at a subject level and one at a group level.
To estimate this model, Maximum Likelihood estimators (MLE) are generally used.
MLEs require an iterative procedure until the model converges, allowing us to obtain
unbiased and efficient estimators.
Assumptions of OLS and Multilevel modeling
OLS estimation
Multilevel estimation
Linearity: Function
Linearity: at each level
Normality: Residuals
Normality: at each level
Homoscedasticity: Constant variance
Homoscedasticity: At level 1
Independence: between observations
Independence:
- No correlation between residuals at
level 1 and level 2.
- Observations in the highest level are
independent of each other.
Mean indicators for multilevel modeling
Proportion of variance:
At level 1: rij / (rij + u0j )
At level 2: u0j / (rij + u0j)
intraclass correlation coefficient
Variance explained (Proxy for an R-square):

Take as a base the null model
At level 1: ( rij(null) - rij(estimated))/ rij(null)
At level 2: ( u0j(null) - u0j(estimated))/ u0j(null)
Datasets that will be employed:

Student_level.dta/sav :
School_level.dta/sav :
Student level variables for Sixth grade students

Classroom and school level variables for each student
Variables in the dataset for the exercises:

idschool
idclass
idstudent
math
age
gender
nsibbl
nuclear
ses_i
indig
gestion
tipo
area
prog_ebi
:
:
:
:
:
:
:
:
:
:
:
:
:
:
bserv
dtotal
:
:
Schools ID
Classroom code
Students code
Math achievement in 2004
Age (years)
1 Female 0 Male
Number of siblings
The student lives with both parents
Socioeconomic index
The student has as a mother tongue an indigenous language
The school is public
Full grade school
1 Urban 0 Rural
Schools that participate in the government-headed bilingual
program
Number of basic services at school
Days of class during the year
II. Mixed models STATA

*There is two commands to run a multilevel model in STATA
*xtmixed
*gllamm
*XTMIXED
*********
*1st - Identified the dependent variable: math
*2nd - Number of levels: 2
*3rd - Identified level two id: idschool
*4th - With or without random coefficients: Without
*5th - Identified predictors at level 1: age gender nsibbl nuclear ses_i
indig
*6th - Identified predictors at level 2: gestion tipo area bserv dtotal
prog_ebi
*Syntax*
********
*Note: the id for level two has to be numeric
destring
idschool, replace
*****************************************a two level model
*xtmixed [dependent variable] [independent variables] || [id level 2]:,
[type of estimation: reml mle] [random component: var sd]
*****************************************a three level model
*xtmixed [dependent variable] [independent variables] || [id level 3]:
|| [id level 2]:, [type of estimation: reml mle] [random component: var
sd]
*****************************************a two level model with random
coefficients
*xtmixed [dependent variable] [independent variables] || [id level 2]:
[variable with random coefficient], [type of estimation: reml mle]
[random component: var sd]
******************************************************Random components
*var(_cons)
: Level 2 variance
*var(Residual)
: Level 1 variance
*Examples*
**********
*Null Model
xtmixed
math || idschool:, var
Performing EM optimization:
Performing gradient-based optimization:
Iteration 0:
Iteration 1:
log restricted-likelihood = -55799.507

Computing standard errors:

Mixed-effects REML regression
Group variable: idschool
Log restricted-likelihood = -55799.507
Number of obs
Number of groups
=
=
10327
564
Obs per group: min =

avg =
max =
1
18.3
30
Wald chi2(0)
Prob > chi2
=
=
.
.
-----------------------------------------------------------------------------math |
Coef.
Std. Err.
z
P>|z|
[95% Conf. Interval]
-------------+---------------------------------------------------------------_cons |
291.5814
2.171078
134.30
0.000
287.3262
295.8366
----------------------------------------------------------------------------------------------------------------------------------------------------------Random-effects Parameters |
Estimate
Std. Err.
-----------------------------+-----------------------------------------------idschool: Identity
|
var(_cons) |
2422.165
160.494
2127.172
2758.067
-----------------------------+-----------------------------------------------var(Residual) |
2489.893
35.66315
2420.967
2560.782
-----------------------------------------------------------------------------LR test vs. linear regression: chibar2(01) = 4803.15 Prob >= chibar2 = 0.0000
*Model 1 - Student variables

xtmixed
math age gender nsibbl nuclear ses_i indig ||
idschool:, var
Iteration 0:
Iteration 1:


Number of obs
Number of groups
=
=
10084
564

avg =
max =
1
17.9
30
Wald chi2(6)
Prob > chi2
=
=
581.48
0.0000
-----------------------------------------------------------------------------math |
Coef.
Std. Err.
z
P>|z|
-------------+---------------------------------------------------------------age | -8.197397
.5646148
-14.52
0.000
-9.304021
-7.090772
gender | -10.04743
1.053241
-9.54
0.000
-12.11175
-7.983119
nsibbl | -1.418298
.2599372
-5.46
0.000
-1.927766
-.9088307
nuclear | -4.110831
1.19185
-3.45
0.001
-6.446815
-1.774847
ses_i |
.4424885
.0532977
8.30
0.000
.338027
.54695
indig | -16.40169
2.547377
-6.44
0.000
-21.39446
-11.40892
_cons |
360.9887
9.142899
39.48
0.000
343.069
378.9085
Estimate
Std. Err.
|
var(_cons) |
1418.413
111.9487
1215.126
1655.709
-----------------------------+-----------------------------------------------var(Residual) |
2415.155
35.33061
2346.892
2485.404
*Model 2 - School variables

xtmixed
math age gender nsibbl nuclear ses_i indig gestion
tipo area bserv dtotal prog_ebi || idschool:, var
Iteration 0:
Iteration 1:


Number of obs
Number of groups
=
=
10084
564

avg =
max =
1
17.9
30
Wald chi2(12)
Prob > chi2
=
=
1170.77
0.0000
-----------------------------------------------------------------------------math |
Coef.
Std. Err.
z
P>|z|
-------------+---------------------------------------------------------------age | -7.704743
.5607968
-13.74
0.000
-8.803884
-6.605601
gender | -10.23747
1.045682
-9.79
0.000
-12.28697
-8.187969
nsibbl | -1.104413
.2590295
-4.26
0.000
-1.612102
-.5967245
nuclear | -3.823928
1.185488
-3.23
0.001
-6.147442
-1.500415
ses_i |
.1828045
.0559183
3.27
0.001
.0732067
.2924024
indig | -13.38815
2.510379
-5.33
0.000
-18.3084
-8.467896
gestion | -39.19536
4.042729
-9.70
0.000
-47.11896
-31.27175
tipo |
11.72788
3.866162
3.03
0.002
4.150338
19.30541
area |
4.210408
4.328804
0.97
0.331
-4.273892
12.69471
bserv |
10.77618
1.946826
5.54
0.000
6.960472
14.59189
dtotal |
.404681
.1593815
2.54
0.011
.092299
.7170629
prog_ebi | -16.20831
4.441248
-3.65
0.000
-24.91299
-7.503619
_cons |
270.5271
44.33491
6.10
0.000
183.6323
357.4219
Estimate
Std. Err.
|
var(_cons) |
817.257
63.49015
701.8292
951.6688
-----------------------------+-----------------------------------------------var(Residual) |
2400.291
34.86962
2332.912
2469.617
*GLLAMM*
********
*1st - Identified the dependent variable: math
*2nd - Number of levels: 2
*3rd - Identified level two id: idschool
*4th - With or without random coefficients: Without
*5th - Identified predictors at level 1: age gender nsibbl nuclear ses_i
indig
*6th - Identified predictors at level 2: gestion tipo area bserv dtotal
prog_ebi
*Syntax
******************************a two level model
*Define equation
*gen
cons=1
*eq int
: cons
/* Intercept equation */
*gllamm
[dependent variable] [independent variables], i([id
level variable]) eqs([equations]) nrf([number of random components])
[other options]
******************************a three level model
*Define equation
*gen
cons=1
*eq int
: cons
*gllamm
[dependent variable] [independent variables], i([level
2 level 3]) eqs([equations]) nrf([number of random components]) [other
options]
******************************a two level model with random coefficients
*Define equation
*gen
cons=1
*eq int
: cons
*eq slope
: [variable with random coefficient]
*gllamm
[dependent variable] [independent variables], i([id
level variable]) eqs([equations]) nrf([number of random components])
[other options]
*Examples*
**********
*Null model
gen
eq int
gllamm
cons=1
: cons
math, i(idschool) eqs(int) nrf(1) adapt
Running adaptive quadrature

Iteration 0:
log likelihood
Iteration 1:
log likelihood
Iteration 2:
log likelihood
Iteration 3:
log likelihood
Iteration 4:
log likelihood
= -58198.98
= -56186.657
= -55826.109
= -55801.265
= -55801.26
Adaptive quadrature has converged, running Newton-Raphson

Iteration 0:
log likelihood = -55801.26
Iteration 1:
Iteration 2:
log likelihood =
-55801.2
Iteration 3:
log likelihood =
-55801.2
number of level 1 units = 10327
Condition Number = 303.60003
gllamm model
-----------------------------------------------------------------------------math |
Coef.
Std. Err.
z
P>|z|
-------------+---------------------------------------------------------------_cons |
291.5841
2.171009
134.31
0.000
287.329
295.8392
-----------------------------------------------------------------------------Variance at level 1
-----------------------------------------------------------------------------2489.9024 (35.663352)
Variances and covariances of random effects
-----------------------------------------------------------------------------***level 2 (idschool)
var(1): 2417.2909 (160.0545)
------------------------------------------------------------------------------
10

gllamm
math age gender nsibbl nuclear ses_i indig,
i(idschool) eqs(int) nrf(1) adapt
Iteration 0:
log likelihood
Iteration 1:
log likelihood
Iteration 2:
log likelihood
Iteration 3:
log likelihood
= -55094.61
= -54598.877
= -54489.213
= -54489.213

Iteration 0:
Iteration 1:
Iteration 2:
Iteration 3:
Iteration 4:
gllamm model
-----------------------------------------------------------------------------math |
Coef.
Std. Err.
z
P>|z|
-------------+---------------------------------------------------------------age | -8.198358
.5652368
-14.50
0.000
-9.306201
-7.090514
gender | -10.04702
1.053157
-9.54
0.000
-12.11117
-7.982871
nsibbl | -1.419342
.2613114
-5.43
0.000
-1.931503
-.9071808
nuclear |
-4.11179
1.191921
-3.45
0.001
-6.447912
-1.775668
ses_i |
.4432147
.0580544
7.63
0.000
.3294303
.5569991
indig | -16.41488
2.601117
-6.31
0.000
-21.51298
-11.31679
_cons |
360.9406
9.283431
38.88
0.000
342.7454
379.1358
-----------------------------------------------------------------------------2413.8148 (35.301464)
-----------------------------------------------------------------------------***level 2 (idschool)
var(1): 1413.3884 (111.42859)
------------------------------------------------------------------------------
11

gllamm
math age gender nsibbl nuclear ses_i indig gestion
tipo area bserv dtotal prog_ebi, i(idschool) eqs(int) nrf(1) adapt
Iteration 0:
log likelihood
Iteration 1:
log likelihood
Iteration 2:
log likelihood
Iteration 3:
log likelihood
Iteration 4:
log likelihood
=
=
=
=
=
-54787.036
-54495.168
-54276.704
-54069.796
-54069.796

Iteration 0:
log likelihood = -54069.796 (not concave)
Iteration 1:
Iteration 2:
Iteration 3:
gllamm model
-----------------------------------------------------------------------------math |
Coef.
Std. Err.
z
P>|z|
-------------+---------------------------------------------------------------age | -7.705684
.5605493
-13.75
0.000
-8.80434
-6.607027
gender | -10.23462
1.045321
-9.79
0.000
-12.28341
-8.185827
nsibbl | -1.106948
.2591939
-4.27
0.000
-1.614959
-.5989372
nuclear | -3.827367
1.185166
-3.23
0.001
-6.15025
-1.504485
ses_i |
.1848875
.0567318
3.26
0.001
.0736953
.2960798
indig | -13.46722
2.534508
-5.31
0.000
-18.43476
-8.499676
gestion | -39.15573
4.01795
-9.75
0.000
-47.03076
-31.28069
tipo |
11.7064
3.839827
3.05
0.002
4.180478
19.23232
area |
4.186199
4.298805
0.97
0.330
-4.239304
12.6117
bserv |
10.77714
1.933419
5.57
0.000
6.987705
14.56657
dtotal |
.4046668
.1582621
2.56
0.011
.0944787
.7148548
prog_ebi | -16.14732
4.418169
-3.65
0.000
-24.80677
-7.487869
_cons |
270.3501
44.04093
6.14
0.000
184.0315
356.6687
-----------------------------------------------------------------------------2399.0027 (34.841723)
-----------------------------------------------------------------------------***level 2 (idschool)
var(1): 803.02789 (62.165406)
------------------------------------------------------------------------------
*Main problem: Time to estimate the models*

*******************************************
12
III. Mixed models SPSS

*To run the multilevel model we have to use the command MIXED
***********************************************a two level model
*MIXED [dependent variable] WITH [independent variables]
*/ PRINT = SOLUTION TESTCOV
: To print the model and the test for the matrix of variance covariance
*/ FIXED = INTERCEPT [independent variables]
: Indicate the fixed part in the model
*/ RANDOM = INTERCEPT | SUBJECT([id level variable])
: Indicate the random part and the grouping variable for the data
***********************************************a three level model
*/ RANDOM = INTERCEPT | SUBJECT([id level variable])
: Indicate the random part and the grouping variable for level 2
*/ RANDOM = INTERCEPT | SUBJECT([id level variable 2]*[id level variable 3])
: Indicate the random part and the grouping variable for level 2 and level
3
*********************************************** a two level model with random
coefficient
*/ RANDOM = INTERCEPT [other random variables] | SUBJECT([id level variable])
COVTYPE(UN)
: Indicate the random part and the grouping variable for the data
and the type of matrix
*
covariance for the level 2 (unestructured)
*Examples*
**********
*Null Model
mixed math
/ print = solution testcov
/ fixed = intercept
/ random = intercept | subject(idschool).
13
Estimates of Fixed Effects(a)

95% Confidence Interval
Parameter
Intercept
Estimate
Std. Error
291.58139
2.171077
1
a Dependent Variable: MAth.
df
539.227
Sig.
134.303
Lower Bound
.000
Upper Bound
287.316585
295.846196
Estimates of Covariance Parameters(a)

Parameter
Residual
Intercept [subject = Variance
idschool]
Estimate
2489.8932
07
2422.1624
26
Std. Error
Wald Z
Sig.
Lower Bound
Upper Bound
35.663150
69.817
.000
2420.966727
2560.782067
160.49373
8
15.092
.000
2127.170040
2758.063864

mixed math with age gender nsibbl nuclear ses_i indig
/ fixed = intercept age gender nsibbl nuclear ses_i indig
Parameter
Intercept
Nsibbl
Estimate
360.98869
8
-8.197397
10.047433
-1.418299
nuclear
-4.110831
Age
gender
Std. Error
df
Sig.
Lower Bound
Upper Bound
9.142899
9964.925
39.483
.000
343.066768
378.910628
.564615
9830.408
-14.519
.000
-9.304158
-7.090636
1.053241
9675.797
-9.540
.000
-12.112006
-7.982860
.259937
9791.842
-5.456
.000
-1.927829
-.908768
1.191850
9579.555
-3.449
.001
-6.447110
-1.774552
.442489
.053298
2.547377
16.401693
8310.954
8.302
.000
.338012
.546965
8871.112
-6.439
.000
-21.395141
-11.408245
ses_i
Indigo

Parameter
Residual
idschool]
Estimate
2415.1550
55
1418.4124
24
Std. Error
Wald Z
Sig.
Lower Bound
Upper Bound
35.330614
68.359
.000
2346.891618
2485.404054
111.94861
1
12.670
.000
1215.125679
1655.708409

mixed math with age gender nsibbl nuclear ses_i indig gestion tipo area bserv dtotal prog_ebi
14

/ fixed = intercept age gender nsibbl nuclear ses_i indig gestion tipo area bserv dtotal prog_ebi
Parameter
Intercept
Nsibbl
Estimate
270.52710
2
-7.704743
10.237467
-1.104413
nuclear
-3.823929
1.185488
9732.357
-3.226
.001
-6.147731
-1.500126
ses_i
.055918
10022.105
3.269
.001
.073193
.292416
2.510379
8566.070
-5.333
.000
-18.309099
-8.467202
4.042729
502.568
-9.695
.000
-47.138085
-31.252623
Tipo
.182805
13.388150
39.195354
11.727876
3.866161
506.687
3.033
.003
4.132195
19.323557
Area
4.210408
4.328804
489.480
.973
.331
-4.294922
12.715738
Bserv
10.776182
1.946826
547.469
5.535
.000
6.952018
14.600345
.404681
.159381
prog_ebi
4.441248
16.208304
514.081
2.539
.011
.091562
.717800
549.555
-3.649
.000
-24.932203
-7.484406
Age
gender
Indigo
gestion
Std. Error
df
Sig.
Lower Bound
Upper Bound
44.334909
547.503
6.102
.000
183.439762
357.614443
.560797
9941.506
-13.739
.000
-8.804018
-6.605467
1.045682
9863.263
-9.790
.000
-12.287217
-8.187717
.259030
9865.989
-4.264
.000
-1.612164
-.596662
Dtotal

Parameter
Residual
idschool]
Estimate
2400.2913
25
817.25668
9
Std. Error
Wald Z
Sig.
Lower Bound
Upper Bound
34.869622
68.836
.000
2332.911916
2469.616793
63.490120
12.872
.000
701.828974
951.668455
15
IV. Hierarchical Linear Model Program - HLM

1. Select the datasets for level 1 and level 2
2. Select the variables that will be used for the analysis at each level
16
3. Save the template file (.mdmt) and the mdm file (.mdm) with names that you prefer.
4. Press Make MDM to create the data file with the information of both levels. In the
case of a error, a message will appear on the screen.
5. Press Check Stats and save the descriptive statistics of your dataset at each level
(mean, standard deviation, minimum and maximum).
6. Press Done and you will have the following window
7. Once you have obtained the window with the data at both levels, we could start
running the different multilevel models.
17
8. Running a null model: Do a click on Math and select the option Outcome variable
Thus, we could see that we are running a simple anova with random effects by school
18
Select Run Analysis from the menu and then under option, choose Run the model
shown
Then go to File from your menu options and select View Output and a txt file will
appear with the regression analysis developed by the program
Program:
Authors:
Publisher:
HLM 6 Hierarchical Linear and Nonlinear Modeling

Stephen Raudenbush, Tony Bryk, & Richard Congdon
Scientific Software International, Inc. (c) 2000
techsupport@ssicentral.com
www.ssicentral.com
------------------------------------------------------------------------------Module:
HLM2.EXE (6.04.2754.2)
Date:
30 March 2009, Monday
Time:
3:35:36
------------------------------------------------------------------------------SPECIFICATIONS FOR THIS HLM2 RUN
Problem Title: no title
The data source for this run = C:\seminar\datasets\hlm\sixthgrade.mdm
The command file for this run = whlmtemp.hlm
Output file name
= C:\seminar\datasets\hlm\hlm2.txt
The maximum number of level-1 units = 10327
The maximum number of iterations = 100
Method of estimation: restricted maximum likelihood
Weighting Specification
-----------------------
Level 1
Level 2
Precision
Weighting?
no
no
no
Weight
Variable
Name
The outcome variable is
Normalized?
MATH
The model specified for the fixed effects was:

---------------------------------------------------Level-1
Coefficients
----------------------
Level-2
Predictors
---------------
19
INTRCPT1, B0
INTRCPT2, G00
The model specified for the covariance components was:

--------------------------------------------------------Sigma squared (constant across level-2 units)
Tau dimensions
INTRCPT1
Summary of the model specified (in equation format)
--------------------------------------------------Level-1 Model
Y = B0 + R
Level-2 Model
B0 = G00 + U0
Iterations stopped due to small change in likelihood function
******* ITERATION 5 *******
Sigma_squared =
Tau
INTRCPT1,B0
2489.89329
2422.15990
Tau (as correlations)

INTRCPT1,B0 1.000
---------------------------------------------------Random level-1 coefficient
Reliability estimate
---------------------------------------------------INTRCPT1, B0
0.911
---------------------------------------------------The value of the likelihood function at iteration 5 = -5.579951E+004
MATH
Final estimation of fixed effects:

---------------------------------------------------------------------------Standard
Approx.
Fixed Effect
Coefficient
Error
T-ratio
d.f.
P-value
---------------------------------------------------------------------------For
INTRCPT1, B0
INTRCPT2, G00
291.581398
2.171072
134.303
563
0.000
---------------------------------------------------------------------------The outcome variable is
MATH
Final estimation of fixed effects

(with robust standard errors)
---------------------------------------------------------------------------Standard
Approx.
Fixed Effect
Coefficient
Error
T-ratio
d.f.
P-value
---------------------------------------------------------------------------For
INTRCPT1, B0
INTRCPT2, G00
291.581398
2.169134
134.423
563
0.000
---------------------------------------------------------------------------Final estimation of variance components:
----------------------------------------------------------------------------Random Effect
Standard
Variance
df
Chi-square P-value
Deviation
Component
-----------------------------------------------------------------------------
20
INTRCPT1,
U0
49.21544
2422.15990
563
9762.13458
0.000
level-1,
R
49.89883
2489.89329
----------------------------------------------------------------------------Statistics for current covariance components model
-------------------------------------------------Deviance
= 111599.013039
Number of estimated parameters = 2
21
9. Running the model 1 Student variables only
We add the level one variable by clicking on each variable and selecting the option add
variable uncentered. There are other options like: group mean centered and grand mean
centered. However, for this exercise we will run all the variables uncentered.
Program:
Authors:
Publisher:

www.ssicentral.com
------------------------------------------------------------------------------Module:
HLM2.EXE (6.04.2754.2)
Date:
Time:
3:38: 7
Output file name
-----------------------
Level 1
Weighting?
no
Weight
Variable
Name
Normalized?
22
Level 2
Precision
no
no
MATH

---------------------------------------------------Level-1
Coefficients
---------------------INTRCPT1, B0
#
AGE slope, B1
#
GENDER slope, B2
#
NSIBBL slope, B3
#
NUCLEAR slope, B4
#
SES_I slope, B5
#
INDIG slope, B6
Level-2
Predictors
--------------INTRCPT2, G00
INTRCPT2, G10
INTRCPT2, G20
INTRCPT2, G30
INTRCPT2, G40
INTRCPT2, G50
INTRCPT2, G60
'#' - The residual parameter variance for this level-1 coefficient has been set
to zero.
Tau dimensions
INTRCPT1
--------------------------------------------------Level-1 Model
Y = B0 + B1*(AGE) + B2*(GENDER) + B3*(NSIBBL) + B4*(NUCLEAR) + B5*(SES_I) +
B6*(INDIG) + R
Level-2 Model
B0 = G00 + U0
B1 = G10
B2 = G20
B3 = G30
B4 = G40
B5 = G50
B6 = G60
Run-time deletion has reduced the number of level-1 records to 10084
******* ITERATION 6 *******
Sigma_squared =
Tau
INTRCPT1,B0
2415.14524
1418.53123

INTRCPT1,B0 1.000
---------------------------------------------------INTRCPT1, B0
0.865
23
MATH

---------------------------------------------------------------------------Standard
Approx.
Fixed Effect
Coefficient
Error
T-ratio
d.f.
P-value
---------------------------------------------------------------------------For
INTRCPT1, B0
INTRCPT2, G00
360.992707
9.142942
39.483
563
0.000
For
AGE slope, B1
INTRCPT2, G10
-8.197324
0.564615
-14.518
10077
0.000
For
GENDER slope, B2
INTRCPT2, G20
-10.047485
1.053240
-9.540
10077
0.000
For
NSIBBL slope, B3
INTRCPT2, G30
-1.418230
0.259937
-5.456
10077
0.000
For NUCLEAR slope, B4
INTRCPT2, G40
-4.110753
1.191849
-3.449
10077
0.001
For
SES_I slope, B5
INTRCPT2, G50
0.442432
0.053298
8.301
10077
0.000
For
INDIG slope, B6
INTRCPT2, G60
-16.400386
2.547400
-6.438
10077
0.000
MATH

---------------------------------------------------------------------------Standard
Approx.
Fixed Effect
Coefficient
Error
T-ratio
d.f.
P-value
---------------------------------------------------------------------------For
INTRCPT1, B0
INTRCPT2, G00
360.992707
9.682294
37.284
563
0.000
For
AGE slope, B1
INTRCPT2, G10
-8.197324
0.586866
-13.968
10077
0.000
For
GENDER slope, B2
INTRCPT2, G20
-10.047485
1.131407
-8.881
10077
0.000
For
NSIBBL slope, B3
INTRCPT2, G30
-1.418230
0.249420
-5.686
10077
0.000
INTRCPT2, G40
-4.110753
1.231744
-3.337
10077
0.001
For
SES_I slope, B5
INTRCPT2, G50
0.442432
0.057503
7.694
10077
0.000
For
INDIG slope, B6
INTRCPT2, G60
-16.400386
2.673658
-6.134
10077
0.000
----------------------------------------------------------------------------Random Effect
Standard
Variance
df
Chi-square P-value
Deviation
Component
----------------------------------------------------------------------------INTRCPT1,
U0
37.66339
1418.53123
563
6158.23620
0.000
level-1,
R
49.14413
2415.14524
-------------------------------------------------Deviance
= 108426.073337
24
10. To run the model 2 School variables, we need to click on the level 2 button and
select where we want to add the level two variables: intercept or coefficients; in other
words if we want to add predictors for the intercept or cross-level interactions. In this
case for simplicity we only add predictors to the intercept.
Program:
Authors:
Publisher:

www.ssicentral.com
------------------------------------------------------------------------------Module:
HLM2.EXE (6.04.2754.2)
Date:
Time:
3:40:15
Output file name
-----------------------
Level 1
Weighting?
no
Weight
Variable
Name
Normalized?
25
Level 2
Precision
no
no
MATH

---------------------------------------------------Level-1
Coefficients
---------------------INTRCPT1, B0
#
#
#
#
#
#
AGE
GENDER
NSIBBL
NUCLEAR
SES_I
INDIG
slope,
slope,
slope,
slope,
slope,
slope,
B1
B2
B3
B4
B5
B6
Level-2
Predictors
--------------INTRCPT2, G00
GESTION, G01
TIPO, G02
AREA, G03
PROG_EBI, G04
BSERV, G05
DTOTAL, G06
INTRCPT2, G10
INTRCPT2, G20
INTRCPT2, G30
INTRCPT2, G40
INTRCPT2, G50
INTRCPT2, G60
'#' - The residual parameter variance for this level-1 coefficient has been set
to zero.
Tau dimensions
INTRCPT1
--------------------------------------------------Level-1 Model
Y = B0 + B1*(AGE) + B2*(GENDER) + B3*(NSIBBL) + B4*(NUCLEAR) + B5*(SES_I) +
B6*(INDIG) + R
Level-2 Model
B0 = G00 + G01*(GESTION) + G02*(TIPO) + G03*(AREA) + G04*(PROG_EBI)
+ G05*(BSERV) + G06*(DTOTAL) + U0
B1 = G10
B2 = G20
B3 = G30
B4 = G40
B5 = G50
B6 = G60
Run-time deletion has reduced the number of level-1 records to 10084
******* ITERATION 6 *******
Sigma_squared =
Tau
INTRCPT1,B0
2400.29539
817.21163

INTRCPT1,B0 1.000
----------------------------------------------------
26
INTRCPT1, B0
0.799
MATH

---------------------------------------------------------------------------Standard
Approx.
Fixed Effect
Coefficient
Error
T-ratio
d.f.
P-value
---------------------------------------------------------------------------For
INTRCPT1, B0
INTRCPT2, G00
270.525672 44.332588
6.102
557
0.000
GESTION, G01
-39.195039
4.042508
-9.696
557
0.000
TIPO, G02
11.727700
3.865951
3.034
557
0.003
AREA, G03
4.210208
4.328564
0.973
557
0.332
PROG_EBI, G04
-16.207826
4.441016
-3.650
557
0.001
BSERV, G05
10.776193
1.946724
5.536
557
0.000
DTOTAL, G06
0.404681
0.159373
2.539
557
0.012
For
AGE slope, B1
INTRCPT2, G10
-7.704750
0.560796
-13.739
10071
0.000
For
GENDER slope, B2
INTRCPT2, G20
-10.237444
1.045681
-9.790
10071
0.000
For
NSIBBL slope, B3
INTRCPT2, G30
-1.104433
0.259029
-4.264
10071
0.000
INTRCPT2, G40
-3.823956
1.185488
-3.226
10071
0.002
For
SES_I slope, B5
INTRCPT2, G50
0.182821
0.055918
3.269
10071
0.001
For
INDIG slope, B6
INTRCPT2, G60
-13.388784
2.510361
-5.333
10071
0.000
MATH

---------------------------------------------------------------------------Standard
Approx.
Fixed Effect
Coefficient
Error
T-ratio
d.f.
P-value
---------------------------------------------------------------------------For
INTRCPT1, B0
INTRCPT2, G00
270.525672 48.248823
5.607
557
0.000
GESTION, G01
-39.195039
4.046229
-9.687
557
0.000
TIPO, G02
11.727700
3.976412
2.949
557
0.004
AREA, G03
4.210208
4.054383
1.038
557
0.300
PROG_EBI, G04
-16.207826
4.640755
-3.492
557
0.001
BSERV, G05
10.776193
2.026056
5.319
557
0.000
DTOTAL, G06
0.404681
0.174626
2.317
557
0.021
For
AGE slope, B1
INTRCPT2, G10
-7.704750
0.585244
-13.165
10071
0.000
For
GENDER slope, B2
INTRCPT2, G20
-10.237444
1.119275
-9.146
10071
0.000
For
NSIBBL slope, B3
INTRCPT2, G30
-1.104433
0.245914
-4.491
10071
0.000
INTRCPT2, G40
-3.823956
1.219071
-3.137
10071
0.002
For
SES_I slope, B5
INTRCPT2, G50
0.182821
0.056285
3.248
10071
0.002
For
INDIG slope, B6
INTRCPT2, G60
-13.388784
2.500433
-5.355
10071
0.000
----------------------------------------------------------------------------Random Effect
Standard
Variance
df
Chi-square P-value
Deviation
Component
----------------------------------------------------------------------------INTRCPT1,
U0
28.58691
817.21163
557
3813.89027
0.000
27
level-1,
R
48.99281
2400.29539
-------------------------------------------------Deviance
= 108083.173301
11. If we want to have random coefficients, we need to click next to the coefficient for
each variable at Level 2 Model, and then add a random component for this variable
across schools.
In this case, we are adding a random component for the coefficient of age; this means that
this coefficient varies randomly across schools.
28
V. Multilevel in SAS
The command to run a multilevel model in SAS is PROC MIXED and the syntax is:
PROC MIXED DATA = [name of the data] METHOD = REML COVTEST;
CLASS [level 2 id];
MODEL [dependent variable] = [independent variables] / SOLUTION;
RANDOM [random variables] / SUBJECT=[level 2 id]
RUN;
COVTEST
METHOD
CLASS
MODEL
SOLUTION
RANDOM
SUBJECT
RUN
: option calls the significance test of random components (Wald test).

: method that will be used to estimate the model.
: indicates which variable is categorical.
: to define the model that will be estimated.
: requires to SAS to print the results of the estimation in the output
: indicates the component or variables that will be random in the model
: indicates the grouping variable in the model
: to run the program.
*********************************************a two level model

CLASS [level 2 id];
RANDOM [random variables] / SUBJECT=[level 2 id];
RUN;
*********************************************a three level model
CLASS [level 2 id and level 3 id];
RANDOM [random variables] / TYPE=UN SUBJECT=[level 3 id];
RANDOM [random variables] / TYPE=UN SUBJECT=[level 2 id([level 3 id])];
RUN;
Note: TYPE = UN indicates that will be estimated the unstructured variance for the
random effects.
*********************************************a two level model with random
coefficients
CLASS [level 2 id];
RANDOM [random variables] / TYPE=UN SUBJECT=[level 2 id];
29
RUN;
*Example*
*********
*Model Null
PROC MIXED DATA = achievement METHOD = REML COVTEST;
CLASS idschool;
MODEL math = / SOLUTION;
RANDOM intercept / SUBJECT=idschool;
RUN;
*Model 1 Student variables
CLASS idschool;
MODEL math = age gender nsibbl nuclear ses_i indig / SOLUTION;
RUN;
*Model 2 School variables
CLASS idschool;
MODEL math = age gender nsibbl nuclear ses_i indig gestion tipo area bserv dtotal
prog_ebi / SOLUTION;
RUN;
30
VI. Summary
In sum, the three programs give us the same results in terms of significance, coefficients and standard errors estimated, as we could
see in the following table:
Table 1. Math Achievement in Math for Sixth Grade in Peru (2004)
Age (years)
Female
Number of siblings
Nuclear family
Socioeconomic status
Indigenous
Intercept
Level 1 variance
Level 2 variance
GLLAMM
se ( )
-8.20
0.57
-10.05
1.05
-1.42
0.26
-4.11
1.19
0.44
0.06
-16.41
2.60
360.94
9.28
2414
1413
63
37
***
***
***
**
***
***
***
***
***
XTMIXED
se ( )
-8.20
0.56
-10.05
1.05
-1.42
0.26
-4.11
1.19
0.44
0.05
-16.40
2.55
360.99
9.14
2415
1418
63
37
***
***
***
**
***
***
***
-8.20
-10.05
-1.42
-4.11
0.44
-16.40
360.99
***
***
2415
1418
MIXED
se ( )
0.56
1.05
0.26
1.19
0.05
2.55
9.14
63
37
***
***
***
**
***
***
***
-8.20
-10.05
-1.42
-4.11
0.44
-16.40
360.99
***
***
2415
1419
HLM
se ( )
0.59
1.13
0.25
1.23
0.06
2.67
9.68
63
37
***
***
***
**
***
***
***
***
***
***p<.001, **p<.01, *p<.05, +p<.10
The main difference between the different programs relies on the type of dependent variable that we want to model. Thus, in the case
of HLM and GLLAMM, it is possible to model two level or three level models with dependent variables with discrete or binary
outcomes; while the XTMIXED and MIXED in STATA and SPSS respectively could be employed with continuous variables. Also,
one of the disadvantages of STATA is that XTMIXED and GLLAMM could not be estimated in combination with sampling weights,
while SPSS and HLM allow the user to use sampling weights. Also, in the case of HLM, one advantage is that it is possible to use
sampling weights at each level.
31
Useful links for multilevel modeling

UCLA: Statistical Computing web page
http://www.ats.ucla.edu/
Urbana Champaign Illinois: Multilevel Analysis Seminar
http://www.ed.uiuc.edu/courses/EdPsy490CK/
University of Texas at Austin: Division of Statistics + Scientific computation
http://ssc.utexas.edu/
Indiana University: Stat/Math Center
http://www.indiana.edu/~statmath/contact/consult.html
32

MultilevelPRI Workshop

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

MultilevelPRI Workshop

Uploaded by

Copyright:

Available Formats

PRI workshop

Multilevel modeling: A comparison of software packages

Juan Leon Jara Almonte

Students nested in schools

00 + 10Xij + 01Zj + rij + u0j

intraclass correlation coefficient

Variance explained (Proxy for an R-square):

Datasets that will be employed:

Student level variables for Sixth grade students

Variables in the dataset for the exercises:

II. Mixed models STATA

math || idschool:, var

log restricted-likelihood = -55799.507

Computing standard errors:

Log restricted-likelihood = -55799.507

Obs per group: min =

*Model 1 - Student variables

log restricted-likelihood = -54213.956

Computing standard errors:

Log restricted-likelihood = -54213.956

Obs per group: min =

*Model 2 - School variables

log restricted-likelihood = -54042.506

Computing standard errors:

Log restricted-likelihood = -54042.506

Obs per group: min =

Running adaptive quadrature

Adaptive quadrature has converged, running Newton-Raphson

*Model 1 - Student variables

Adaptive quadrature has converged, running Newton-Raphson

*Model 2 - School variables

Adaptive quadrature has converged, running Newton-Raphson

*Main problem: Time to estimate the models*

III. Mixed models SPSS

Estimates of Fixed Effects(a)

Estimates of Covariance Parameters(a)

*Model 1 - Student variables

Estimates of Covariance Parameters(a)

*Model 2 - School variables

/ print = solution testcov

Estimates of Covariance Parameters(a)

IV. Hierarchical Linear Model Program - HLM

HLM 6 Hierarchical Linear and Nonlinear Modeling

The outcome variable is

The model specified for the fixed effects was:

The model specified for the covariance components was:

Tau (as correlations)

Final estimation of fixed effects:

Final estimation of fixed effects

9. Running the model 1 Student variables only

HLM 6 Hierarchical Linear and Nonlinear Modeling

The outcome variable is

The model specified for the fixed effects was:

Tau (as correlations)

The outcome variable is

Final estimation of fixed effects:

Final estimation of fixed effects

HLM 6 Hierarchical Linear and Nonlinear Modeling

The outcome variable is

The model specified for the fixed effects was:

Tau (as correlations)

Final estimation of fixed effects:

Final estimation of fixed effects

: option calls the significance test of random components (Wald test).

Main problem: Time to estimate the models

p<.001, p<.01, p<.05, +p<.10