ORIGINAL ARTICLE

Diagnostic, Pharmacy-Based, and Self-Reported Health

Measures in Risk Equalization Models
Pieter J. A. Stam, PhD,* Rene C. J. A. van Vliet, PhD,* and Wynand P. M. M. van de Ven, PhD*
Background: Current research on the added value of self-reported
health measures for risk equalization modeling does not include all
types of self-reported health measures; and/or is compared with a
limited set of medically diagnosed or pharmacy-based diseases;
and/or is limited to specic populations of high-risk individuals.
Objective: The objective of our study is to determine the predictive
power of all types of self-reported health measures for prospective
modeling of health care expenditures in a general population of adult
Dutch sickness fund enrollees, given that pharmacy and diagnostic
data from administrative records are already included in the risk
equalization formula.
Research Design: We used 4 models of 2002 total, inpatient and
outpatient expenditures to evaluate the separate and combined pre-
dictive ability of 2 kinds of data: (1) Pharmacy-based (PCGs) and
Diagnosis-based (DCGs) Cost Groups and (2) summarized self-
reported health information. Model performance is measured at the
total population level using R
2
and mean absolute prediction error;
also, by examining mean discrepancies between model-predicted
and actual expenditures (ie, expected over- or undercompensation)
for members of potentially mispriced subgroups. These subgroups
are identied by self-reports from prior-year health surveys and
utilization and expenditure data from 5 preceding years.
Subjects: Subjects were 18,617 respondents to a health survey, held
among a stratied sample of adult members of the largest Dutch
sickness fund in 2002, with an overrepresentation of people in poor
health.
Data: The data were extracted from a claims database and a health
survey. The claims-based data are the outcomes of total, inpatient,
and outpatient annualized expenditures in 2002; age, gender, PCGs,
DCGs in 2001; and health care expenditures and hospitalizations
during the years 1997 to 2001. The SF-36, Organization for Eco-
nomic Cooperation and Development items, and long-term diseases
and conditions were collected by a special purpose health survey
conducted in the last quarter of 2001.
Results: Out-of-sample R
2
equals 17.2%, 2.6%, and 32.4% for the
models of total, inpatient and outpatient expenditures including PCGs,
DCGs, and self-reported health measures. Self-reported health mea-
sures contribute less to predictive power than PCGs and DCGs. PCGs
and DCGs also predict better than self-reported health measures for
people with top 25% total expenditures or hospitalizations in each year
during a 5-year period. On the other hand, self-reported health measures
are better predictors than PCGs and DCGs for people without any top
25% expenditures during the 5-year period, for switchers, and for most
subgroups of relatively unhealthy people dened by self-reported health
measures. Among the set of self-reported health measures, the SF-36
adds most to predictive power in terms of R
2
, mean absolute prediction
error, and for almost all studied subgroups.
Conclusion: It is concluded that the self-reported health measures
make an independent contribution to forecasting health care expen-
ditures, even if the prediction model already includes diagnostic and
pharmacy-based information currently used in Dutch risk equaliza-
tion models.
Key Words: health insurance, risk equalization, health status,
SF-36
(Med Care 2010;48: 448457)
I
n several countries, competition among health insurers is
used to stimulate efciency and responsiveness to consum-
ers preferences in the health care sector.
1
The ultimate goal
is to stimulate health insurance companies to act as prudent
purchasers or providers of care for their members. At the
same time, nancial transfers are needed in such markets for
individual health insurance to avoid problems of access to
coverage for those at high risk. The rst and best solution in
this case is to organize a system of risk-adjusted equalization
payments.
2
A prediction model of health care expenditures is
then used to calculate these payments.
A common challenge in all countries is to nd adequate
predictors of health care expenditures, in particular direct
measures of health status differences among insured individ-
uals. Currently, the Dutch prospective model is one of the
most sophisticated risk equalization models in the world. It
includes more than 30 diagnosis-based and pharmacy-based
cost groups (DCGs and PCGs) derived from administrative
data. PCGs and DCGs are examples of variables based on the
providers diagnosis of the patients medical condition and
corresponding treatment decisions. DCGs are derived from
ICD-9-CM diagnoses with respect to prior year hospitaliza-
tions.
35
Prior year pharmacy recipes are used to construct
PCGs based on selected Anatomical Therapeutic Chemical
(ATC) Classication System-codes.
5,6
From the *Institute of Health Policy and Management, Erasmus University
Rotterdam, Rotterdam, The Netherlands; and SiRM Strategies in
Regulated Markets, Rotterdam, The Netherlands.
Reprints: Pieter J. A. Stam, PhD., Institute of Health Policy and Manage-
ment, Erasmus University Rotterdam, PO Box 1738, 3000 DR, Rotter-
dam, The Netherlands. E-mail: piet.stam@sirm.nl.
Copyright 2010 by Lippincott Williams & Wilkins
ISSN: 0025-7079/10/4805-0448
Medical Care Volume 48, Number 5, May 2010 448 | www.lww-medicalcare.com
Even in the Dutch model, many diseases are still not
explicitly identied and therefore risk-adjusted equaliza-
tion payments may lead to undercompensation of patients
with rare diseases, for example.
7
The use of most rare
diseasesdened as a (very often: innate, hereditary)
disease with a prevalence of less than 1 in 4000as a risk
adjuster is basically ruled out, because of stability prob-
lems arising from small sample sizes and the often ob-
served inherent lack of medical protocols and guidelines.
In general, given the set of PCGs and DCGs from admin-
istrative data sources, it appears difcult to nd diagnoses
not yet included in the model to construct new risk adjust-
ers. Multiyear diagnostic information may improve predic-
tive accuracy of the model, but the necessary datasets are
scarcely available in practice.
3
Other risk adjusters used in practice are indirect mea-
sures of health status based on sociodemographic, socioeco-
nomic, and behavioral characteristics, such as self-employ-
ment or disability allowance.
8
However, such risk adjusters
may be inappropriate, because they predict expenditure vari-
ation for which the equalization payments are not intended to
be adjusted.
9
Furthermore, mortality rate and prior expendi-
tures or utilization may induce inappropriate incentives, be-
cause compensation of associated expenditures comes at the
risk of unintended behavioral changes by providers or insur-
ers.
1
Direct measures of health status besides PCGs and
DCGs are physiologic and self-reported health measures. In
this study, we focus on self-reported health measures. Van de
Ven and Ellis distinguish into following 3 types.
1
1. General health status: Examples are single item self-
assessed health questions (excellent/very good/good/fair/
poor), change in health status, and multi-item health ques-
tions such as the SF-36 or the closely related RAND-36
survey.
1012
2. Functional status: This can be measured by the Long-
term Disability Questionnaire from the Organization for
Economic Cooperation and Development (OECD) or
the question on activities of daily living (ADL) for
which people need assistance, for example.
13,14
The
ADL question is appropriate when focusing on people
with physical disabilities and/or the elderly population.
Only 3 out of 7 ADL items are appropriate for the 12
population as well: Getting in and out of bed, Going
up and down the stairs, and Getting in or out of
chairs. The OECD questionnaire is preferred in case of
a more general population. Van den Berg and Van der
Wulp note that the SF-36 does not include all OECD
and ADL items.
1517
3. Self-reported chronic conditions: Our focus is on long-
term conditions and diseases in the context of prospective
risk equalization models; some of these may overlap with
DCG- and PCG-identied conditions.
Lifestyle, marital status, employment, and education are ex-
amples of another type of self-reported information listed by
Van de Ven and Ellis in this context.
1
However, such rather
indirect health measures may be inappropriate, because they
predict expenditure variation for which the equalization pay-
ments are not intended to be adjusted. In this study, we
therefore concentrate on the 3 types of direct health measures
collected by self-reports.
Comparing the ability of diagnostic and pharmacy-
based information to predict expenditures with that of self-
reported health measures has been the subject of considerable
research.
1829
However, its added value remains unclear
because not all 3 types of self-reported health measures are
applied in those studies; performance is compared with a
limited set of medically diagnosed or pharmacy-based dis-
eases (or: cost groups); and/or specic populations of high-
risk individuals are targeted.
Our study adds to the literature by determining the
performance of all 3 types of self-reported health measures,
for a general population of both high-risk and low-risk adults,
relative to a diagnostic- and pharmacy-based risk equalization
model. Model performance is measured at the total popula-
tion level using R
2
and mean absolute prediction error; also,
by examining mean discrepancies between model-predicted
and actual expenditures (expected over- or undercompensa-
tion) for members of potentially mispriced subgroups iden-
tied by self-reports from prior-year health surveys and
utilization and expenditure data from 5 preceding years.
METHODS
Study Population and Data source
The population studied is the 2001/2002 Dutch popu-
lation of sickness fund enrollees who constitute about two-
thirds of total Dutch population. They were legally obliged to
enroll with a sickness fund (together with their family-
members) because they earned an annual wage below an
income threshold of 30,700 (2002). The study sample is
based on the individual claims data of the largest Dutch
sickness fund at the time, called Agis Health Insurance
(market share: 16%). These data are combined with the
answers of respondents (N 18,617) to a health survey in the
last quarter of 2001. This special purpose survey was con-
ducted to collect self-reported health status information, long-
term diseases and conditions, health care utilization, and
individual characteristics not present in the claims adminis-
tration.
Health Survey Data
Following the Dillman mailing procedure guidelines,
the postal health survey was sent to a stratied sample of
50,022 noninstitutionalized sickness fund members of be-
tween 16 and 90 years of age.
30
The stratication was
designed to get an overrepresentation of people with a poor
health status. Gross response was 46.3% (23,163 respon-
dents). Nonresponse analysis showed that standardization
for selective nonresponse is not needed for the purpose of
our study.
31
A net response of 22,029 records remains after
validity and completeness checks, following the CAHPS
3.0 Adult Commercial Questionnaire (CAHPS) guide-
lines.
32
In the analysis, the strata are weighted back to
Medical Care Volume 48, Number 5, May 2010 Health Measures in Risk Equalization
2010 Lippincott Williams & Wilkins www.lww-medicalcare.com | 449
population proportions to take account of the stratied
sampling procedure.
The 8 SF-36 Likert scales are general health measures,
called Physical Functioning, RolePhysical, Bodily Pain,
General Health, Vitality, Social Functioning, Role-Emo-
tional, and Mental Health. They are derived for 18,617
respondents and appear to be both reliable and valid.
31,33
Sample size is relatively high compared with other studies.
31
The 8 scales are used instead of the PCS and MCS summary
scales to capture the maximum extent of systematic variation
present in this study sample.
20
Functional status is measured by responses to 7 items
from the OECD Questionnaire, which concern communica-
tion, visual, and mobility problems.
13,34
Each OECD item
concerns a question about with how much effort a specic
task could be performed. The functional status measure is a
count of the number of questions answered by with a lot of
effort or I cannot do this. The interpretation is that func-
tional status worsens as this count approaches 7.
Finally, a weighted so-called Chronic Conditions Index
(CCI) was developed that takes into account both the number
and the relative importance of 20 self-reported long-term
diseases and conditions.
35
The relative importance weight of
each condition is based on average expenditures in 2001 for
people reporting that condition. Some examples of the
chronic conditions are diabetes mellitus, stroke, heart condi-
tions, cancer, hypertension, urinary incontinence, hernia, and
osteoarthritis. A chronic condition is only taken into account
if the respondent indicates that he/she still has complaints or
is under treatment for it at the time of lling out the ques-
tionnaire.
Administrative Data
The administrative data include 2001 demographics,
PCGs, DCGs and 1997 to 2002 individual annual expendi-
tures for general practitioner (GP) care, inpatient room and
board, inpatient and outpatient specialist care, prescribed
drugs delivered outside the hospital, dental care, obstetrics,
physical therapy, medical devices, sick-transport, and mater-
nity care. Every member had the same insurance coverage
without deductibles or copayments and was insured during
the 2001 to 2002 period, 3.9% of them for less than 2 years.
All data on expenditures refer to actual charges.
The mean total, inpatient and outpatient expenditures in
2002 are 1753, 613 and 1140, respectively, with coef-
cients of variation of about 3.4, 7.3, and 2.5. Only 3.7% of the
members had zero expenditures in 2002. The correlation
between total expenditures in 2001 and 2002 is about 0.4,
which is equal to that found for the 1992 to 1993 sample of
Dutch sickness fund members.
36
Lower intertemporal corre-
lations are found elsewhere, however.
36,37
Statistical Analysis
Equation (1) describes the prospective risk equalization
model:
Y
t

0,t

j1
J

j,t
X
j, t1

t
(1)
where Y
t
are the expenditures observed in year t, X
j,t-1
is the j
th
risk adjuster observed in year t1, j 1, . . . , J, and

t
is an independent and identically distributed error term in
year t (t 2002 in our empirical application). The selection
of the X
j,t1
is guided by the criteria of appropriate incentives,
fair payments, and data feasibility.
1
The variables Y
t
, X
j,t1
,
and
t
are N 1 vectors, the elements of which contain the
observations for N individuals. The index t with respect to the
unknown
j,t
coefcients indicates that these coefcients may
not be constant over the years. Predicted expenditures Y

t
are
given by
Y

t

0, t

j1
J

j,t
X
j,t1
(2)
after estimation of the parameters of equation (1) by
ordinary least-squares. We predict total, inpatient and outpa-
tient expenditures for 4 different sets of risk adjusters, all
including age and gender (df 15):
1. PCGs and DCGs (treated additively, df 12 13).
2. All self-report data (individual SF-36 scales, number of
OECD limitations and CCI, df 8 3 1).
3. PCGs, DCGs, and SF-36 scales (df 25 8).
4. PCGs, DCGs, and all self-report data (df 37).
The rst model is the reference model in this study, in
which PCGs and DCGs are included because of their auton-
omous effects on observed expenditures. These also adjust
the association between the SF-36 scores and observed ex-
penditures for treatment effects in model 3. Such adjustment
applies as the SF-36 scores for people under treatment may be
similar to those for people without any disease, although the
level of expenditures differs between these groups. We did
not adjust for within-group variation in expenditures caused
by severity differences, because this would give rise to
sample size issues.
12
The predictive performance of the model variants is
evaluated at the total population level in terms of R
2
and
the mean absolute prediction error
1
N
t 1
N
y
it
y
it
MAPE, where more powerful models have higher
R
2
and lower MAPE. We also apply a repeated random
split-sample approach to mitigate the inuence of outlier
observations and avoid overtting when calculating each
models R
2
and MAPE. Equation (1) is estimated on a
random draw of half the observations in our study sample
(the training le). Predicted expenditures are then calcu-
lated with equation (2) for the remaining half of the sample
(the validation le), from which the so-called out-of-
sample versions of R
2
and MAPE can be calculated. This
procedure is repeated 100 times for each model specica-
tion. Finally, the average R
2
and MAPE over these 100
runs are calculated. From a theoretical point of view, this
split-sample approach shows the generalizability of our
results to other populations.
Predictive performance is also determined by the mean
result 1/N
SiS
y
it
y
it
for selected subgroups, where N
S
is
the number of individuals in subgroup S. Mean subgroup
results can be interpreted as expected over- or undercompen-
sations and thus as an indication of the incentives for selec-
Stam et al Medical Care Volume 48, Number 5, May 2010
2010 Lippincott Williams & Wilkins 450 | www.lww-medicalcare.com
tion against these subgroups. The smaller expected over- and
undercompensations, the better model performance.
The selection of subgroups in our study is based on the
survey answers of the respondents who:
Reported being in fair or poor health (UNHEALTHY).
Answered with a lot of effort or I cannot do this to at
least 1 of the 3 OECD questions on mobility impairments
(IMMOBIL).
Reported still having complaints or being under treatment
for each of 5 long-term diseases and conditions: diabetes
mellitus (DIAB), stroke, brain hemorrhage/infarction
(BRAIN), myocardial infarction (AMI), other serious heart
disease (NON-AMI HEART), and cancer or other malig-
nant condition (CANCER);
and, based on the number of years during the 1997 to 2001
period that they had:
Total expenditures in the top 25%.
One or more hospitalizations.
These subgroups are chosen to explore model perfor-
mance for those at high risk.
35,38
Undercompensation may be
expected for the rst 3 subgroups listed above and for people
with top 25% expenditures or one or more hospitalizations
each year during the period 1997 to 2001.
RESULTS
Table 1 shows the sample characteristics and the
full-sample regression results for 3 different prediction
models of total expenditures. There are 37.8% male re-
spondents in the weighted study sample (about 45% of the
Dutch population consists of male enrollees, including
children), and 19.5% is 65 years or above. Mean age is
about 47 years (not tabulated). About 2.8% is assigned to
at least 1 DCG, about 8.8% to at least 1 PCG, and about
20.4% has 1 or more OECD limitations. Table 1 also
shows the mean SF-36 and CCI scores.
The full-sample R
2
equals 3.2% for the model of total
2002 expenditures if predicted by age and gender alone,
comparable to results found elsewhere in the literature. R
2
equals 17.9% for model 1 and almost all of the PCG and
DCG regression coefcients are signicantly different from
zero (P 0.05). However, unbiasedness of these estimated
coefcients is strongly rejected by a Hausman specication
test against model 4 (
2
40 262.79, P 0.000).
39
R
2
increases to 19.6% for model 4, with a majority of the
coefcients for the self-reported health indicators being sig-
nicantly different from zero (P 0.05) and their joint
equality to zero also strongly rejected (F 14, 18,562
28.85, P 0.000). Most of the age and gender coefcients
are no longer signicant (P 0.05). Using dummy variables
for each OECD limitation instead of the count measure did
not improve their explanatory power (not tabulated). The
variance ination factors did not signal any problem of
multicollinearity (not tabulated).
Table 2 shows R
2
and MAPE for prediction models of
inpatient and outpatient expenditures. R
2
of model 2 is about
half the size of that of model 1 for inpatient expenditures, and
about a third when modeling outpatient expenditures. R
2
increases from 31.4% to 34.3% and the MAPE decreases
when the self-reported health measures are added to model 1
for outpatient expenditures. R
2
increases from 6.0% to 6.5%
and MAPE even slightly increases in case of inpatient ex-
penditures.
Table 2 also shows out-of-sample R
2
- and MAPE-
outcomes. The out-of-sample R
2
is only slightly smaller for
models of total and outpatient expenditures, but it is about
half the size of the full-sample R
2
when predicting inpatient
expenditures for the models 1, 3, and 4. This illustrates the
contribution of PCGs and DCGs to model performance by
capturing outlier observations. The out-of-sample R
2
for
model 4 of total expenditures equals 17.2%.
Table 3 shows model performance for subgroups of
relatively unhealthy people dened by self-reported general
health, functional status, and chronic conditions. With a few
exceptions, for example, for people with cancer or other
malignant condition, model 2 predicts expenditures better
than model 1. Another exception is that model 1 sufces to
remove expected undercompensation for those who reported
chronic conditions when modeling inpatient expenditures.
The expected compensations appear smallest if model 4 is
applied. They are not signicantly different from zero (P
0.05) anymore, except for those who reported myocardial
infarction, or cancer or other malignant condition when
modeling total and outpatient expenditures. Among the self-
reported health measures, predictive power of the SF-36
dominates for subgroups dened by general health and func-
tional status, whereas it is about equal for people who
reported chronic conditions.
Table 4 shows model performance for subgroups de-
ned by the number of years with top 25% expenditures
during the period 1997 to 2001. The expected undercompen-
sation for people with top 25% expenditures each year during
this 5-year period is smaller for model 1 than for model 2. It
is smallest for model 4, though still signicantly different
from zero (P 0.05). Analogous results hold for the models
of inpatient and outpatient expenditures. Predictive power
among the set of self-reported health measures is largest for
the SF-36 for any tabulated subgroup.
Table 4 also shows that model 2 performs better than
model 1 for switcherswho are insured for less than 5 years
during the 1997 to 2001 periodand people without any top
25% expenditures. Model 4 further improves performance for
the latter subgroup, but not for switchers.
Table 5 shows model performance for subgroups de-
ned by the number of years with hospitalizations during the
period 1997 to 2001. Not surprisingly, given the focus on
hospitalizations, for all subgroups model 1 performs better
than model 2, except for switchers and people with a single
year of hospitalizations. Model 4 performs best, although
model 2 already sufces for the switchers. The additional
predictive power of the self-reported health measures de-
clines with the number of years with hospitalizations. Again,
predictive power among the set of self-reported health mea-
sures is dominated by the SF-36.
Medical Care Volume 48, Number 5, May 2010 Health Measures in Risk Equalization
2010 Lippincott Williams & Wilkins www.lww-medicalcare.com | 451
TABLE 1. Weighted Full-Sample Estimated Regression Coefficients and Full-Sample R
2
for 3
Variants of Prospective Risk Equalization Models of 2002 Total Expenditures (N 18,617,
1 1)
Explanatory Variables
Prevalence or Mean*
Age Gender Model Model 1

Model 4

Unweighted Weighted
M: 1524 (reference category) 1.9% 4.2%
M: 2534 3.7% 7.1% 127 207 186
M: 3544 4.4% 6.9% 57 255 448
M: 4554 5.2% 6.3% 1083

734

368
M: 5564 6.4% 5.7% 1559

831

242
M: 6574 8.2% 5.0% 2966

1734

1103

M: 7584 4.7% 2.3% 4226

2988

2072

M: 85 0.6% 0.3% 2923

1833

704
F: 1524 3.4% 6.2% 6 103 189
F: 2534 8.3% 11.3% 438 400 283
F: 3544 10.5% 12.8% 192 91 115
F: 4554 10.0% 11.3% 632

343 134
F: 5564 10.4% 8.9% 1049

581

49
F: 6574 11.9% 7.2% 1996

1256

387
F: 7584 8.9% 4.0% 3444

2516

1180

F: 85 1.6% 0.7% 2810

1940

87
No PCG (reference category) 80.3% 91.2%
Asthma/COPD 7.9% 4.0% 2218

1540

Epilepsy 1.1% 0.5% 1872

1392

Crohn/colitis ulcerosa 0.4% 0.2% 458 184

Cardiac disease 8.5% 3.1% 1875

1098

Rheumatism 0.5% 0.3% 3969

2837

Parkinson 0.4% 0.1% 2942

2054
Diabetes (type I) 3.5% 1.2% 3143

2346

Transplantation 0.4% 0.1% 7197

6600

Cystic brosis 0.1% 0.0% 3464 3098

Neuromuscular disorder 0.2% 0.1% 8687

6609

HIV/AIDS 0.2% 0.1% 12,158

11,796

Renal disease/ESRD 0.1% 0.0% 19,632

19,502

No DCG (reference category) 94.9% 97.2%

DCG01 0.9% 0.5% 1391

653
DCG02 1.0% 0.6% 4994

4567

DCG03 0.8% 0.5% 4029

3440

DCG04 0.8% 0.5% 5197

4644

DCG05 0.5% 0.3% 2721

2325

DCG06 0.3% 0.1% 7259

6209

DCG07 0.5% 0.3% 4461

3737

DCG08 0.4% 0.2% 7348

6857

DCG09 0.1% 0.0% 6459

5489

DCG10 0.2% 0.1% 15,997

15,022

DCG11 0.1% 0.1% 7486

7245

DCG12 0.2% 0.1% 9556

8709

DCG13 0.2% 0.0% 75,672

74,891

PF (physical functioning)

0.70 0.80 1656

RP (role physical)

0.59 0.72 153

BP (bodily pain)

0.66 0.73 104

GH (general health)

0.60 0.67 777

VT (vitality)

0.59 0.64 438

SF (social functioning)

0.75 0.81 599

RE (role emotional)

0.72 0.79 35
MH (mental health)

0.72 0.74 936

(Continued)
Stam et al Medical Care Volume 48, Number 5, May 2010
2010 Lippincott Williams & Wilkins 452 | www.lww-medicalcare.com
CONCLUSIONS
From our study it follows that the self-reported health
measures have an independent contribution to the prediction
of individual expenditures in terms of R
2
and MAPE. By their
own, however, these add less predictive power than PCGs
and DCGs. This result is also found for high-risk people with
top 25% total expenditures in each year during a 5-year
period and even more pronounced if based on multiyear
hospitalizations. Expected undercompensation for these sub-
groups are smallest, but still quite substantial, if PCGs,
TABLE 1. (Continued)
Explanatory Variables
Prevalence or Mean*
Age Gender Model Model 1

Model 4

Unweighted Weighted
No OECD limitation 64.6% 79.6%
One OECD limitation 16.1% 10.3% 379

Two OECD limitations 10.3% 4.9% 625

Three or more OECD limitations 13.0% 5.2% 883

Chronic conditions index (CCI)

0.07 0.04 4457

Intercept 1 1 818

765

2538

R
2
3.2% 17.9% 19.6%
* The sizes of the (multiple) PCGs and (multiple) DCGs are presented here. Mean values are presented instead of subgroup sizes in case
of the continuous metric SF-36 subscales and the self-reported chronic conditions.

Model 1 contains age, gender, PCGs and DCGs as risk adjusters, model 4 contains all self-report data (individual SF-36 scales, number
of OECD limitations and chronic conditions index) in addition.

The estimated coefcient is signicantly different from zero (2-sided t test, P 0.05).

The original 0 to 100 scale of these SF-36 scales is transformed to a 0 to 1 scale in order for the size of the estimated coefcients to
be comparable to those of the estimated coefcients corresponding to the regression dummy variables. People with higher scale scores are
healthier. The standard deviations corresponding to the PF, RP, BP, GH, VT, SF, RE, and MH scales are 0.24, 0.40, 0.25, 0.21, 0.20, 0.24,
0.36, and 0.19, respectively.

The respondents were asked whether they suffered from one or more of 17 named chronic conditions; in addition there were 3 open
categories of chronic conditions. The self-reported chronic conditions variable is equal to a weighted sum of the prevalences per insured,
where the weights are average full-sample 2001 total expenditures for each of the conditions. The scale is 0 to 1 (minimum 0, maximum
0.79, median 0.00, standard deviation 0.07).

The presented full-sample R

2
is adjusted for the actual number K of explanatory variables in regression equation (1) following the
standard procedure, ie, R
2
1 (1 R
2
UNADJUSTED
) (N 1)/(N K). Note that this is not an out-of-sample R
2
.
PCG indicates pharmacy-based cost group; COPD, Chronic obstructive pulmonary disease; ESRD, end-stage renal disease; DCG,
diagnosis-based cost group; OECD, Organization for Economic Cooperation and Development.
TABLE 2. Full-Sample and Out-of-Sample R
2
and MAPE for Prospective Risk Adjustment Models of 2002 Expenditures
(N 18,617)
Model Number Risk Adjusters
Model Degrees
of Freedom
Full-Sample
Out-of-
Sample

R
2
* MAPE R
2
* MAPE
Y
t
total expenditures 2002 (

Y
t
1753)
1 Age/gender, PCGs, DCGs 40 17.9% 1764 15.7% 1790
2 Age/gender, SF-36, OECD, CCI 27 7.3% 1848 6.7% 1863
3 Age/gender, PCGs, DCGs, SF-36 48 19.3% 1717 17.1% 1750
4 Age/gender, PCGs, DCGs, SF-36, OECD, CCI 52 19.6% 1703 17.2% 1736
Y
t
inpatient expenditures 2002 (

Y
t
613)
1 Age/gender, PCGs, DCGs 40 6.0% 1037 2.4% 1053
2 Age/gender, SF-36, OECD, CCI 27 2.9% 1090 2.3% 1102
3 Age/gender, PCGs, DCGs, SF-36 48 6.4% 1046 2.7% 1072
4 Age/gender, PCGs, DCGs, SF-36, OECD, CCI 52 6.5% 1042 2.6% 1069
Y
t
outpatient expenditures 2002 (

Y
t
1140)
1 Age/gender, PCGs, DCGs 40 31.4% 900 29.8% 912
2 Age/gender, SF-36, OECD, CCI 27 10.1% 929 9.6% 935
3 Age/gender, PCGs, DCGs, SF-36 48 33.8% 850 32.1% 863
4 Age/gender, PCGs, DCGs, SF-36, OECD, CCI 52 34.3% 837 32.4% 850
* R
2
is adjusted for the model degrees of freedom K 1 as follows: R
2
ADJ
1 (1 R
2
)*(N 1)/(N K), where number K equals the number of risk adjusters (including
the intercept) that are included in equation (1).

The out-of-sample statistics result from a repeated random split-half approach (estimation based on one half of the sample, predictions on the other half), with 100 replications
for each model variant.
MAPE indicates mean absolute prediction error; PCG, pharmacy-based cost group; DCG, diagnosis-based cost group; OECD, Organization for Economic Cooperation and
Development; CCI, chronic conditions index.
Medical Care Volume 48, Number 5, May 2010 Health Measures in Risk Equalization
2010 Lippincott Williams & Wilkins www.lww-medicalcare.com | 453
DCGs, and self-reported health measures are applied alto-
gether. On the other hand, self-reported health measures are
better predictors than PCGs and DCGs for people without any
top 25% expenditures during the 5-year period, for switchers,
and for most subgroups of relatively unhealthy people dened
by self-reported health measures. Model performance is best
if PCGs, DCGs, and self-reported health measures are ap-
plied. Noticeable exceptions are that PCGs and DCGs sufce
to remove expected undercompensation for people who re-
ported chronic conditions when predicting inpatient expendi-
tures, and self-reported health measures sufce for switchers.
Among the set of self-reported health measures, the
SF-36 adds most to predictive power in terms of R
2
,
MAPE, and for the tabulated subgroups. This does not hold
for people who reported chronic conditions, in which case
all 3 types of self-reported health measures should be
applied.
DISCUSSION
Our conclusion is in accordance with the common
conclusion in other studies that diagnostics, pharmacy-based
TABLE 3. Expected Over- and Undercompensations in 2002 for Subgroups of Relatively Unhealthy Insured People Defined
by Self-Reported General Health Status, Functional Status, and Chronic Conditions in 2001 (N 18,617, 1 1)*
Model
Number Risk Adjusters UNHEALTHY
Lowest PF
scores
Lowest MH
scores IMMOBIL DIAB

BRAIN AMI
NON-AMI
HEART CANCER
Unweighted
prevalences:
33.3% 19.6% 13.1% 26.3% 7.5% 2.4% 4.6% 3.7% 4.9%
Weighted prevalences: 20.5% 10.0% 10.0% 14.5% 3.6% 1.2% 2.0% 1.7% 2.9%
Y
t
total expenditures 2002 (

Y
t
1753)
Average expenditures
per insured
3996 5251 2751 4409 4647 5395 6027 5741 5226
1 Age/gender, PCGs,
DCGs
1059

1696

624

1244

681

1247 1346

880 1222

2 Age/gender, SF-36,
OECD, CCI
245 398

72 138 628

33 1139

437 1599

3 Age/gender, PCGs,
DCGs, SF-36
218 357 102 355

391 530 998

491 1038

4 Age/gender, PCGs,
DCGs, SF-36,
OECD, CCI
172 296 81 90 119 78 579 68 773

Y
t
inpatient expenditures 2002 (

Y
t
613)
Average expenditures
per insured
1586 2214 1047 1826 1650 2313 2659 2450 2287
1 Age/gender, PCGs,
DCGs
438

753

278

564

196 545 495 211 500

2 Age/gender, SF-36,
OECD, CCI
137 195 82 67 58 24 568 186 833

3 Age/gender, PCGs,
DCGs, SF-36
119 188 99 203 87 254 357 64 432
4 Age/gender, PCGs,
DCGs, SF-36,
OECD, CCI
101 151 91 52 14 73 247 91 370
Y
t
outpatientexpenditures 2002 (

Y
t
1140)
Average expenditures
per insured
2410 3037 1704 2583 2998 3082 3368 3291 2939
1 Age/gender, PCGs,
DCGs
621

944

346

680

485

702

851

669

722

2 Age/gender, SF-36,
OECD, CCI
111 205

11 72 584

161 688

426

835

3 Age/gender, PCGs,
DCGs, SF-36
98 169 3 151

304

275 641

427

606

4 Age/gender, PCGs,
DCGs, SF-36,
OECD, CCI
72 146 11 38 107 26 414

151 455

* The abbreviations used in this table are for subgroups of insured people who reported being in fair or poor health (UNHEALTHY), the size 10% subgroup of people with lowest
scores on the Physical Functioning (PF) or Mental Health (MH) scales, those confronted with OECD mobility impairments (IMMOBIL) and those who reported still suffering from
or being under treatment for one or more out of 5 of the following diseases and conditions: diabetes mellitus (DIAB), stroke, brain haemorrhage/infarction (BRAIN), myocardial
infarction (AMI), other serious heart conditions (NON-AMI HEART) and cancer or other malignant condition (CANCER).

The PCGs applied in this study only adjust for the differences in expenditures of diabetes patients who take insulin.

The mean result is statistically signicantly different from zero (2-sided t test, P 0.05).
PCG indicates pharmacy-based cost group; DCG, diagnosis-based cost group; OECD, Organization for Economic Cooperation and Development; CCI, chronic conditions index.
Stam et al Medical Care Volume 48, Number 5, May 2010
2010 Lippincott Williams & Wilkins 454 | www.lww-medicalcare.com
information, self-reported conditions, SF-36, (instrumental)
ADL and/or OECD questionnaire all make, to a signicant
degree, independent contributions to forecasting health care
expenditures.
1829
Apparently, this holds irrespective of the
wide array of study populations (the USA-Medicare popula-
tion being most prevalent), the various subsets of self-re-
ported health measures, and the set of PCGs and DCGs. Our
study is based on a stratied sample from a general popula-
tion of Dutch insured people of 16 years and older.
Advantages of self-reported health information are that
it is not contingent on having come into contact with a
medical expert, no prior claims history is needed to generate
predictions, consumer perceptions of need and anticipated
use are captured, there is a uniform mode of administration
across health insurers and in principle there is no limitation to
information that can be collected, such as with claims infor-
mation.
1
Some practical problems in adding health measures are
conceptually similar to those associated with incorporating
PCGs and DCGs in prospective payment systems.
16
In this
study, neither the PCGs and DCGs nor the self-reported
health measures were adjusted for intensity of treatment
differences among patients (within-group variance), for ex-
ample, by including physiologic health measures. Further-
more, just as DRG creep may occur in practice, it appears
possible to manipulate some self-reported health mea-
sures.
40,41
Unlike PCGs and DCGs, a relative disadvantage of
using self-reported health measures is that the collection for
every individual may be unfeasible in practice, although
some argue that the reason of collection costs has been
overstated.
42
However, the need to collect self-reports for
every individual can be overcome by applying a recently
developed limited-sample benchmark method.
9
Following
this new approach, the self-reported information of a limited
subsample of individuals can be used to improve the equal-
ization payments for every individual in the total population
in practice.
TABLE 4. Expected Over- and Undercompensations in 2002 for Subgroups of Insured People Defined by the Number of
Years That They Have Expenditures in the top 25% During the 5-Year Period 19972001 (N 18,617, 1 1)
Model nr. Risk Adjusters <5 yr Insured* Never 1 yr 2 yr 3 yr 4 yr All 5 yr
Unweighted prevalences: 8.0% 25.7% 16.4% 11.6% 8.4% 7.8% 22.1%
Weighted prevalences: 13.9% 40.6% 17.8% 9.6% 5.8% 4.3% 8.0%
Y
t
total expenditures 2002 (

Y
t
1753)
Average expenditures per
insured
997 742 1583 1920 2613 3748 6690
1 Age/gender, PCGs, DCGs 242

456

31 54 317 955

2128

2 Age/gender, SF-36, OECD,

CCI
130

392

48 263

168 452 2517

3 Age/gender, PCGs, DCGs,

SF-36
147

281

5 199

13 540

1651

4 Age/gender, PCGs, DCGs,

SF-36, OECD, CCI
137

255

14 200

49 473 1524

Y
t
inpatientexpenditures 2002 (

Y
t
613)
Average expenditures per
insured
238 243 655 649 888 1275 2456
1 Age/gender, PCGs, DCGs 134

120

78 93 12 183 671

2 Age/gender, SF-36, OECD,

CCI
89

100

101 146 182 22 755

3 Age/gender, PCGs, DCGs,

SF-36
97

52

90 147 120 24 483

4 Age/gender, PCGs, DCGs,

SF-36, OECD, CCI
92

45 98 145 130 2 440

Y
t
outpatient expenditures 2002 (
Y
t
1140)
Average expenditures per
insured
760 500 928 1270 1725 2473 4234
1 Age/gender, PCGs, DCGs 107

337

109

39 305

773

1457

2 Age/gender, SF-36, OECD,

CCI
43 295

151

117

18 478

1780

3 Age/gender, PCGs, DCGs,

SF-36
50 229

95

52 107 516

1168

4 Age/gender, PCGs, DCGs,

SF-36, OECD, CCI
45 211

84

55 83 473

1091

* About 13.9% of the insured people were insured at Agis for less than 5 year during the 5-year period 19972001. The mean results for this subgroup are presented separately
in this table.

The mean result is statistically signicantly different from zero (2-sided t test, P 0.05).
PCG indicates pharmacy-based cost group; DCG, diagnosis-based cost group; OECD, Organization for Economic Cooperation and Development; CCI, chronic conditions index.
Medical Care Volume 48, Number 5, May 2010 Health Measures in Risk Equalization
2010 Lippincott Williams & Wilkins www.lww-medicalcare.com | 455
REFERENCES
1. van de Ven WP, Ellis RP. Risk adjustment in competitive health plan
markets. In: Culyer AJ, Newhouse JP, eds. Handbook of Health Eco-
nomics. Vol 1. Amsterdam, The Netherland: Elsevier Science BV;
2000:755845.
2. van de Ven WP, van Vliet RC, Schut FT, et al. Access to coverage for
high-risks in a competitive individual health insurance market: via
premium rate restrictions or risk-adjusted premium subsidies? J Health
Econ. 2000;19:311339.
3. Lamers LM, van Vliet RC. Multiyear diagnostic information from prior
hospitalizations as a risk adjuster for premium subsidies. Med Care.
1996;34:549561.
4. Lamers LM. Risk-adjusted capitation payments: developing a diagnostic cost
groups classication for the Dutch situation. Health Policy. 1998;45:1532.
5. Prinsze FJ, van Vliet RC. Health-based risk adjustment: Improving the
pharmacy-based cost group model by adding diagnostic cost groups.
Inquiry. 2007;44:469480.
6. Lamers LM; Pharmacy Costs Groups. A risk-adjuster for premium subsi-
dies based on the use of prescribed drugs. Med Care. 1999;37:824830.
7. van Vliet RC. Vervolgonderzoek naar Compensatie voor Zeldzame
Chronische Aandoeningen in het Zvw-Risicovereveningsmodel; Bundel
Deelrapportages (WOR 323). Rotterdam, The Netherlands: Erasmus
University Rotterdam; 2007.
8. van de Ven WP, Beck K, Buchner F, et al. Risk adjustment and risk
selection on the sickness fund insurance market in 5 European countries.
Health Policy. 2003;65:7598.
9. Stam PJA, Van Vliet RCJA, Van de Ven WPMM. A limited-sample
benchmark approach to assess and improve the performance of risk
TABLE 5. Expected Over- and Undercompensations in 2002 for Subgroups of Insured People Defined by the Number of
Years That They Have Been Hospitalized During the 5-Year Period 1997 to 2001 (N 18,617, 1 1)
Model Number Risk Adjusters <5 yr Insured* Never 1 yr 2 yr 3 yr 4 yr All 5 yr

Unweighted prevalences: 8.0% 53.9% 23.8% 9.7% 3.2% 1.0% 0.3%

Weighted prevalences: 13.9% 63.1% 16.8% 4.7% 1.1% 0.3% 0.1%
Y
t
total expenditures 2002 (

Y
t
1753)
Average expenditures
per insured
997 1286 2676 4410 6945 16,964 22,332
1 Age/gender, PCGs,
DCGs
242

198

389

897

1726

7765

10,401

2 Age/gender, SF-36,
OECD, CCI
130

273

297

1227

2853

12,257

17,233

3 Age/gender, PCGs,
DCGs, SF-36
147

140

230 564

1247

7181

9825
4 Age/gender, PCGs,
DCGs, SF-36,
OECD, CCI
137

129

201 528

1156

6972

9679
Y
t
inpatient expenditures 2002 (

Y
t
613)
Average expenditures per insured 238 393 1077 1872 2812 8507 7455
1 Age/gender, PCGs,
DCGs
134

87

187 398

583 4913

2638
2 Age/gender, SF-36,
OECD, CCI
89

132

183 598

1125

6598

5334

3 Age/gender, PCGs,
DCGs, SF-36
97

64

125 264 388 4676

2405
4 Age/gender, PCGs,
DCGs, SF-36,
OECD, CCI
92

60

114 248 351 4595

2324
Y
t
outpatient expenditures 2002 (

Y
t
1140)
Average expenditures
per insured
760 893 1599 2538 4133 8457 14,878
1 Age/gender, PCGs,
DCGs
107

111

203

498

1143

2852

7763

2 Age/gender, SF-36,
OECD, CCI
43 144

121

646

1758

5697

11,936

3 Age/gender, PCGs,
DCGs, SF-36
50 76

106

300

859

2505

7420

4 Age/gender, PCGs,
DCGs, SF-36,
OECD, CCI
45 71

90

288

819

2389

7365

* About 13.9% of the insured people were insured at Agis for less than 5 year during the 5-year period 1997 to 2001. The mean results for this subgroup are presented separately
in this table.

Notice that undercompensations in case of the models of inpatient expenditures are unexpectedly smaller than in the previous column due to large variance of inpatient
expenditures within this subgroup. Undercompensations are signicant and equal to 4448, 6339, 4212, and 4130 for models 1, 2, 3, and 4, respectively, for the combined
subgroup of those who are hospitalized either 4 or 5 year during 1997 to 2001.

The mean result is statistically signicantly different from zero (2-sided t test, P 0.05).
PCG indicates pharmacy-based cost group; DCG, diagnosis-based cost group; OECD, Organization for Economic Cooperation and Development; CCI, chronic conditions index.
Stam et al Medical Care Volume 48, Number 5, May 2010
2010 Lippincott Williams & Wilkins 456 | www.lww-medicalcare.com
equalization models. Journal of Health Economics. Corrected proof
available at http://dx.doi.org/10.1016/j.jhealeco.2010.02.001.
10. Thomas JW, Lichtenstein R. Including health status in Medicares
adjusted average per capita cost capitation formula. Med Care. 1986;
24:259275.
11. Ware JE, Sherbourne CD. The MOS 36 item short form health survey
(SF-36). Med Care. 1992;30:473A483A.
12. Hornbrook MC, Goodman MJ. Chronic disease, functional health status,
and demographics: a multi-dimensional approach to risk adjustment.
Health Serv Res. 1996;31:283307.
13. McWhinnie JR. Disability assessment in population surveys: results of
the O. E.C.D.Common Development Effort. Rev Epidemiol Sante Pub-
lique. 1981;29:413419.
14. Porell F, Caro FG, Silva A, et al. A longitudinal analysis of nursing
home outcomes. Health Serv Res. 1998;33:835865.
15. van den Berg J, van der Wulp C. Rapport van de Werkgroep Revisie
POLS- Gezondheidsenquete 1999. Heerlen, The Netherland: CBS Neth-
erlands Statistics; 2003.
16. Newhouse JP. Rate adjusters for Medicare capitation. Health Care
Financ Rev. 1986;4555.
17. Iezzoni LI, McCarthy EP, Davis RB, et al. Mobility difculties are not
only a problem of old age. J Gen Intern Med. 2001;16:235243.
18. Wang MC, Rosen A, Kazis L, et al. Correlation of risk adjustment
measures based on diagnoses and patient self-reported health status.
Health Serv Outcomes Res Methodol. 2001;1:351365.
19. Nelson EC, McHorney CA, Manning WG, et al. A longitudinal study of
hospitalization rates for patients with chronic disease: results from the
Medical Outcomes Study. Health Serv Res. 1998;32:759774.
20. Pietz K, Ashton CM, McDonell M, et al. Predicting healthcare costs in
a population of Veterans Affairs beneciaries using diagnosis-based risk
adjustment and self-reported health status. Med Care. 2004;42:1027
1035.
21. Hornbrook MC, Goodman MJ. Assessing relative health plan risk with
the Rand-36 health survey. Inquiry. 1995;32:5674.
22. Fleishman JA, Cohen JW, Manning WG, et al. Using the SF-12 health
status measure to improve predictions of medical expenditures. Med
Care. 2006;44(suppl 5):I54I63.
23. Lamers LM. Risk-adjusted capitation based on the Diagnostic Cost
Group model: an empirical evaluation with health survey information.
Health Serv Res. 1999;33:17271744.
24. Kautter J, Pope GC. CMS frailty adjustment model. Health Care Financ
Rev. 2005;26:119.
25. Gruenberg L, Tompkins C, Porell F. The health status and utilization
patterns of the elderly: implications for setting Medicare payments to
HMOs. In: Schefer RM, Rossiter LF, eds. Advances in Health Eco-
nomics and Health Services Research. Vol 10. Greenwich: JAI Press;
1989:4173.
26. Pope GC, Adamache KW, Walsh EG, et al. Evaluating alternative risk
adjusters for Medicare. Health Care Financ Rev. 1998;20:109129.
27. Maciejewski ML, Liu CF, Derleth A, et al. The performance of admin-
istrative and self-reported measures for risk adjustment of veterans
affairs expenditures. Health Serv Res. 2006;40:887904.
28. Maciejewski ML, Liu CF, Fihn SD. Performance of comorbidity, risk
adjustment, and functional status measures in expenditure prediction for
patients with diabetes. Diabetes Care. 2009;32:7580.
29. DeSalvo KB, Jones TM, Peabody J, et al. Health care expenditure
prediction with a single item, self-rated health measure. Med Care.
2009;47:440447.
30. Dillman DA. Mail and Telephone Surveys: The Total Design Method.
New York, NY: John Wiley and Sons; 1978.
31. Stam PJA. Testing the Effectiveness of Risk Equalization Models in Health
Insurance: PhD Dissertation. Rotterdam, The Netherland: Erasmus Uni-
versity Rotterdam; 2007. Available at: http://www.stamonline.nl/phdthesis.
32. CAHPS. Article 9: Determining a Complete Questionnaire: CAHPS

Survey and Reporting Kit 2002. Rockville, MD: AHRQ; 2002.

33. Likert R. A technique for the measurement of attitudes. Arch Psychol.
1932;140:555.
34. van Sonsbeek JL. Gezondheidsenquetes: methodische en inhoudelijke
aspecten van de OESO-indicator betreffende langdurige beperkingen in
het lichamelijk functioneren. CBS Maandbericht Gezondheidsstatistiek.
1988;7:417.
35. van Vliet RC, van de Ven WP. Towards a capitation formula for
competing health insurers: an empirical analysis. Soc Sci Med. 1992;34:
10351048.
36. van Barneveld EM, Lamers LM, van Vliet RC, et al. Risk sharing as a
supplement to imperfect capitation: a tradeoff between selection and
efciency. J Health Econ. 2001;20:147168.
37. Newhouse JP, Manning WG, Keeler EB, et al. Adjusting capitation rates
using objective health measurers and prior utilization. Health Care
Financ Rev. 1989;10:4154.
38. van de Ven WP, van Vliet RC, Lamers LM. Health-adjusted premium
subsidies in the Netherlands. Health Aff. 2004;23:4555.
39. Hausman JA. Specication tests in econometrics. Econometrica. 1978;
46:12511271.
40. Simborg DW. DRG creep: a new hospital-acquired disease. N Engl
J Med. 1981;304:10621064.
41. Ginsburg PB, Carter GM. Medicare case-mix index increase. Health
Care Financ Rev. 1986;7:5164.
42. McClure W. On the research status of risk-adjusted capitation rates.
Inquiry. 1984;21:205213.
Medical Care Volume 48, Number 5, May 2010 Health Measures in Risk Equalization
2010 Lippincott Williams & Wilkins www.lww-medicalcare.com | 457