Scale Construction

Scale Construction
Methods, Statistics, and Models
Fridtjof Nussbeck, University of Zurich
Outline
psychological assessment
criteria for valid assessment
scale development
phases of scale development
statistics MTMM models outlook

ISSAS 2009 - scale construction 2
Psychological Measurement
systematic measurement of a persons behavior different strategies to assess target persons inferences (and clinical judgments) drawn from the results
ISSAS 2009 - scale construction
I. Assessment Methods
direct observation psychophysiological measurement questionnaires
self- or other reports on personality / disorders / emotional states / mood / well being single-shot studies repeated measures diary methods ambulatory assessment
Measurement and Validity

All decisions (assessments) in psychology should be based on the best information available. Information is best when it is objective, reliable, valid, and specific to a given problem. Reliable Measures: measurement error is small Cronbachs alpha Valid Measures: no impact on measurement scores than those one wants to measure Fit into nomological net
Burns & Haynes (2006); Messick (1995); Courvoisier, Nussbeck, Eid, Geiser, & Cole (2008)
II. Scale Construction

not just: looking for some words that tap the construct Standards for Educational and Psychological Testing (American Psychological Association)
When to Construct a New Scale?

no scale exists measuring a specific construct existing scales do not represent the construct adequately:
lack of reliability lack of validity outdated (old words; meaning of words changed; attitudes changed) trait measure vs. state measure insensitive for changes
Phases of test / scale construction
Construct Definition
scope of the scale (level of abstraction) what is to be measured?
broad or narrow construct?
well-being vs. specific emotion
definition of the construct in literature?

adopt a definition vs. work out an own definition clearly describe what is meant by your construct (and what is not meant)
aspects of construct continuum vs. categories
frequency / intensity of experiencing emotions (joy, anger)
scores on items can be combined using (weighted) means
emotional reactions (baroque, rational, positive reappraising)

response patterns can be analyzed yielding classes of reaction styles
unidimensional vs. multidimensional
frequency of experiencing specific positive emotions
love, affection, intimacy, security joy, happiness, cheerfulness, contentment
frequency of experiencing positive affect

Love, affection, joy, happiness
facets of a construct
positive affect may comprise
love Happiness pride
Recommendation
write out a brief, formal description of the construct relate it to other constructs search literature for information about dimensionality / facets
may also help to avoid known problems with respect to unclear instructions, problematic response formats, etc.
12
Design Scale
item writing operationalization of construct sample systematically all content that is potentially relevant to the construct
one may always discard items...
13
Design Scale
the initial item pool
is broader than ones own theoretical view about the construct includes multiple items for each potential facet / dimension includes also items that will finally proof to be distinct or tangential to the construct search for aspects in literature (scientific, but also fiction, dictionnaries, etc.) ask friends and family
item generation is a creative act

Design Scale
basic principles of item writing simple, straightforward, and appropriate language
adequate to reading level no trendy expressions / colloquialisms
Ex.: Do you feel happy? never sometimes always
15
Design Scale
basic principles of item writing one aspect at a time
I feel happy and beloved I do not insult people because it is morally wrong
individuals must differ on items

constants are useless I am the happiest individual alive
16
Design Scale
basic principles of item writing avoid frequencies in item wording
Sometimes, I am happy
avoid item wordings including negative mood terms

I worry about neuroticism
avoid negatives to reverse meaning of an item, but include negative aspects

I am not happy I am sad
Multidimensional Mood Questionnaire
items of the PleasantUnpleasant dimension
Steyer, Schwenkmezger, Notz, & Eid (1994)

Semantic Opposites?
vs.
19
Semantic Opposites?
implies
20
Design Scale
response format depends on introduction
How often do you feel .
frequency format (never, always)
Do you agree to.

Agreement (not at all, very much so)
How much does correspond to you

similarity (not at all like me, very much like me)
How do you judge

evaluation (terrible, excellent)
Design Scale
response format avoid middle category to increase variability
not always desirable
avoid too many categories, respondents cannot differentiate adequately (analog scale?) avoid check-lists response bias avoid forced choice formats
indicates relative strength of alternatives and cannot be compared across individuals (no normative interindividual information)
Design Scale
standard metrical analysis techniques work well with five or more categories However, labels from 1 to 5 do not guarantee a metric scale!
23
Pilot Test
small sample critique scale
ambiguous or confusing items mismatch of items and scale
24
Administration and Item Analysis

large set of items (all potential items) considerable sample size (about 100) correlation with criteria (nontest-criteria) internal consistency (interitem structure) item response theory (focused on latent trait)
25
Validate and Norm

Validity:
does the scale measure, what it intends to measure?
content validity convergent validity criterion validity discriminant validity
Norms:
what are the properties of the distribution of scores for a given population?
Statistics
Before you start:
items recoded?
save recoded items using a new name (mmq01 mmq01r)
look at distributions!
Wut
4
A
300
200
Frequency
Groll
100
Mean =1.86 Std. Dev. =0.71 N =481 0 0 1 2 3 4 5
Wut
27
Statistics
first simple analysis: item correlations with criterion
dichotomous with interval / dichotomous Pearson interval with interval Pearson dichotomous / ordinal with ordinal Wilcoxon
28
Internal Structure
interitem correlations factor analysis for metrical outcomes
standard factor analysis:
exploratory factor analysis confirmatory factor analysis
factor analysis for ordinal data

special estimators (WLS / WLSMV) required
29
Interitem Correlations
30
Exploratory Factor Analysis

how many underlying latent variables exist in the current set of items?
Yi = i + i + i
for all items i
or
Yi = i + i 1 + i
for some items i for some (other) items j

Yj = j + j 2 + j

data driven approach to reduce complexity model finding depends strongly on input (items) you get out, what you put in
32
Criteria for Factor Analysis

How many factors are needed? scree plot eigenvalues parallel analysis explained variance (goodness of fit coefficients) number of factors does not depend on extraction method for ease of interpretation choose (varimax) rotation
Scree Plot
34
Eigenvalues & Explained Variance
35
Factor Loadings
36

empirical structure should match the theoretically expected structure 1st case: items load on one specific factor (subdimensions factors can be correlated)
37
(Sub-) Dimensions
38

empirical structure should match the theoretically expected structure 2nd case: Items load on one common factor and on one specific factor (facets)
39
Facets
40
Confirmatory Factor Analysis

theory driven Model testing structure is known beforehand test if assumed factor structure explains observed variance-covariance matrix structural equation modeling software needed (AMOS, Mplus, LISREL)
Bollen (1989)
simple example
affection love intimacy security joy fortune happin. Conent. JOY LOVE
42
Criteria for Confirmatory Factor Analysis

goodness of fit: comparison of the expected and the observed covariance matrix ,Y )? Cov ( Y , Y ) = Cov ( Y 1 5 1 5 2 test ,Y Cov(Y 1 5 ) = 15Cov ( LOVE , JOY ) RMSEA comparative fit indices (TLI: Tucker-Lewis Index; CFI: Comparative Fit Index)
Schermelleh-Engel, Mller, & Moosbrugger (2003)
43
Internal Consistency
estimation: Cronbachs alpha (should be > .70)
2 2 k T T = 2 k 1 T
association of one item with scale (corrected) item-total correlation (discrimination)

broad constructs narrow construct medium rit higher rit
44
Reliability of items
CTT (Classical Test Theory):
Yi = + i +
reliability:
Rel(Yi ) =
i2 Var( )
Var(Yi )
45
Validity
item pool build in such a way that it is content valid reliability proofed criterion validity
concurrent: correlation with another measure that shall be predicted predictive: prediction (e.g., via regression) of a later event (test-score)
construct validity
46
Validity
construct validity Campbell & Fiske (1959): Multitrait-Multimethod (MTMM) Matrix convergent validity
high associations between different measures of the same construct
discriminant validity trait-method-unit (TMU) more than one method is needed

MTMM analysis foundations

convergent validity
high associations between different measures of the same construct
discriminant validity
Lower associations between measures of different constructs
trait-method-unit (TMU)
scores of a measure depend on underlying trait (construct) but also on method
more than one method is needed

disentangle influences due to trait and method
Methods?
different raters
self-report, friend, acquaintance teacher, pupils
different tests
BDI, Hamilton Scale
different kinds of data

cortisol, heart rate, stress rating productivity rate, ratings of preformance, number of production errors
(measurement occasions) sub-scales / items ISSAS 2009 - scale construction
49
Multitrait-Multimethod Matrix
heuristic inspection (Campbell & Fiske, 1959)
Monomethod blocks: reliabilities on diagonal Heterotrait-Monomethod triangle discriminant validity
50
Multitrait-Multimethod Matrix
heuristic inspection (Campbell & Fiske, 1959)
heteromethod-blocks convergent validity coefficients (bold) heterotrait-heteromethod-triangles
51
convergent and discriminant validity

convergent validity
correlations on validity significantly larger than 0?
discriminant validity
correlations besides the validity diagonal smaller than on validity diagonal? monotrait-heteromethod-correlations (MHC) should be higher than than heterotrait correlations of the same variables correlations of different traits should be similar across mono- and heteromethod blocks (pattern of associations between traits should be similar across methods)
method effects
degree to which monomethod correlations are higher than heteromethod correlations
Critique on C&F approach

correlations depend on reliabilities What is a high correlations? What are different patterns of correlations (what differences can be considered high?) no statistical model to explain genesis of data
no testable consequences trait- and method effects are not disentangled
53
Modern MTMM approaches

confirmatory factor analysis (CFA) CFA-MTMM / SEM-MTMM Models
Separation of true and error components of scores specification of measurement models allow for model testing some allow for a separation of trait- and method components Quantification of trait- and method effects possible
54
factor analysis
1 2
Y1 = 1 + 1 + 1 Y2 = 2 + 2 + 2
Y1 1
Y2 2
observed variables (Y1, Y2) : latent variable (common factor) 1, 2: residualvariablen [measurement error, uniqueness, unique factors] loadings i; intercepts i
ISSAS 2009 - scale construction 55 55
factor analysis
1 2
Y1 = 1 + 1 + 1 Y2 = 2 + 2 + 2
Y1 1
Y2 2
i2 Var () Rel (Yi ) = Var (Yi )
Important indices: - communality - reliability
2 factor model
Cov(F1, F2) Var(F1) 32
R1 3
F1: Extraversion
F2: Repair
Var(F2) 42
R2 4
11
E1 1
21
E2 2
Var(1)
Var(2)
Var(3)
Var(4)
57 57
rules of calculations for variances and covariances

E1 = 11F1 + 1 E2 = 21F1 + 2
Var(aX + bY) = a2 Var(X) + b2 Var(Y) + 2ab Cov(X,Y) Cov(a1X1 + b1Y1, a2X2 + b2Y2) = a1a2 Cov(X1, X2) + a1b2 Cov(X1, Y2) + b1a2 Cov(Y1, X2) + b1b2 Cov(Y1, Y2)
X, Y: variables a, b : constants
2 factor model
Var(E1) Cov(E2, E1) Cov(R1, E1) Cov(R2, E1)
EXTRA1 EXTRA1 EXTRA2 REPAIR1 REPAIR2 0.707 0.434 0.073 0.113 0.584 0.073 0.114 0.359 0.256 0.320
Var(E2) Cov(R1, E2) Cov(R2, E2)

EXTRA2
Var(R1) Cov(R2, R1)

REPAIR1
Var(R2) samplecovariances
REPAIR2
E12 Var(F1) + Var(1) E2 E1 Var(F1) E22 Var(F1) + Var(2) R2 R1 Var(F2) R22 Var(F2) + Var(4)
59 59
R1 E1 Cov(F1, F2) R1 E2 Cov(F1, F2) R12 Var(F2) + Var(3) R2 E1 Cov(F1, F2) R2 E2 Cov(F1, F2)
Single Indicator MTMM Models

Correlated Traits Model Correlated Traits-Correlated Uniqueness Model Correlated Traits-Uncorrelated Methods Model Correlated Traits-Correlated Methods Minus 1 Model
60
Correlated Traits Model
61
Y jk = jk + E jk Var (Y jk ) = Var ( jk Tk ) + Var ( E jk )

2 Var (Tk ) + E jk = jk
Implications
systematic variance in observed variables only caused by traits correlations between latent variables =discriminant validity reliability = consistency (convergent validity) residuals consist of measurement error and method effects (specific to TMU)
63
Correlated Traits-Correlated Uniqueness Modell
64
Y jk = jk + E jk Var (Y jk ) = Var ( jk Tk ) + Var ( E jk )

2 2009 - scale construction = jk Var (TkISSAS )+ E jk
65
Implications
systematic variance in observed variables only caused by traits correlations between latent variables =discriminant validity reliability = consistency (convergent validity) residuals consist of measurement error and method effects (specific to TMU) residuals belonging to one method may correlate (method effects generalize) no separation of effects due to method and measurement error
Correlated Traits-Uncorrelated Methods Modell
67
Y jk = jk + E jk Var (Y jk ) = Var (Tjk Tk ) + Var (Mjk M j ) + Var ( E jk )

2 2 = Tjk Var (Tk ) + Mjk Var ( M j ) + E jk
consistency :
2 Tjk Var (Tk ) CON (Y jk ) = 2 2 TjkVar (Tk ) + Mjk Var ( M j )
Method-specificity:
2 Mjk Var ( M j ) MS (Y jk ) = 2 2 Var ( M j ) TjkVar (Tk ) + Mjk
69
Implications
systematic variance in observed variables caused by traits and method each and every method deviates from the trait (no best method) correlations between latent variables =discriminant validity reliability = consistency (convergent validity) + method-specificity residuals consist of measurement error method effects generalize across traits
Correlated Traits-Correlated Methods Minus 1 Modell
71
Y jk = jk + E jk Var (Y jk ) = Var (Tjk Tk ) + Var ( E jk )

2 = Tjk Var (Tk ) + E jk
Y jk = jk + Mjk M j + E jk Var (Y jk ) = Var (Tjk Tk ) + Var (Mjk M j ) + Var ( E jk )

2 2 = Tjk Var (Tk ) + Mjk Var ( M j ) + E jk
consistency :
2 Tjk Var (Tk ) CON (Y jk ) = 2 2 TjkVar (Tk ) + Mjk Var ( M j )
Method-specificity:
2 Mjk Var ( M j ) MS (Y jk ) = 2 2 Var ( M j ) TjkVar (Tk ) + Mjk
73
Implications
one method is gold standard (reference method) contrast of other methods against reference trait factor is true-score of reference method (does not change if methods are added to the model; comprises trait and method effects of reference method) method effects are residuals in a latent regression variance components of trait and method effects estimable (consistency and method specificity)
Multiple Indicator MTMM-Models

basis: TM model choice of model depends on method structure and research question
75
TM-Model
Latent variables represent TMU separation of systematic and random influences latent variables consist of trait and method influences basis for further analysis
76
Choose among MTMM Models

method structure interchangeable raters / methods?
students in courses friends employees (at same level in hierarchy)
structurally different raters / methods?

self-, friend, and acquaintances student and teacher ratings und Lehrerrating group leader and subordinates
Interchangeable Raters
ANOVA:
Mean is expected value in factor level deviations from mean are independent
Met. A
Rating A1 Rating A2 mean Rating B1 Rating B2

Met. B
CTUM
Y111 1 Y211 Y112 Y212 Y121 Y221
E111 1 M11 M211 1 M12 M211 1 M21 M221 1 M22 M221 1 M31 Y231 M231 1 M32 M231 79
Rater 1
Trait 1
T111
T211 1 T211
Rater 2
Rater 1
Trait 2
T121 Y122 Y222 Y131
Rater 2
Rater 1
Trait 3
T131 Y132 Y232 ISSAS 2009 - scale construction
Rater 2
Structurally Different Raters

rater differ in the quality of their ratings standard? referenz?
standard is best approximation of true score (Platonic) true-score is trait deviations from this true-score are method-effects
Rating A1 Rating A2 Rating B1 Rating B2
Trait (Trait+Method of Standard)
Met. B
Rating A1 Rating A2 Rating B1 Rating B2
Trait (Trait+Method of Standard)
Met. B
81
CTC(M-1)
Y111 T111 Y211 Y112 T111 Y212 Y113 Y213 Y121 T121 Y221 Y122 T121 Y222 Y123 Y223 Y131 T131 Y231 Y132 T131 Y232 Y133 Y233 M133 M132 M123 M122 M113 M112
82
Multiple Indicator MTMM Models

separation of measurement error from systematic influences model choice based on method structure convergent and discriminant validity can be determined method factors are trait specific (one method may overestimate DIF but underestimate DDF) method factors can be related to external variables
CTUM and CTC(M-1) model
84
CTUM model
85
CTC(M-1) model
86
Research (common) practice

content validity is generally assured many authors report Cronbachs , (corrected) item-total correlations exploratory factor analysis are frequently used confirmatory factor analysis is less frequently used MTMM models are found in larger research programs
Norms
Scales of measurement for most constructs is arbitrary Meaning of scores can only be determined in relation to some frame of reference Normative approaches uses distribution of scores (across and within individuals) as frame
large sample Representative sample (multiple samples)
descriptive statistics of scale scores
Norms for subpopulations (male and female; students and non-students samples; ethnical backgrounds; different cultures sharing the same language)
Outlook
Computer programs Dichotomous / ordinal data Cross-cultural research Longitudinal data Multilevel MTMM models
89
Computer Programs
Simple statistics, Cronbachs , item total correlations, exploratory factor analysis:
SPSS R S-Plus
Confirmatory Factor Analysis (SEM)

Mplus LISREL AMOS EQS
Dichotomous / Ordinal Data

Graded Response Model (Samejima, 1969) The observed variable Yijk is linked to the item-specific probit variable by the following probability function:
P (Yijk s | ijk ) = ( ijk isjk ) =
( ijk isjk )
1 e 2
x2 2
dx
The parameter isjk is a difficulty parameter for each response category bound s (s > 0) for item i measuring trait j with method k, and represents the probability distribution of the standard normal distribution.
91
Probability to choose at least category s

P (Yijk s | ijk )
great
1
P (Yijk 1| ijk ) P (Yijk 2 | ijk )
P (Yijk 3 | ijk )
0.8
P (Yijk s | ijk ) = ( ijk isjk ) =
( ijk isjk )
x 1 2 e dx 2
2
0.6
0.4
P (Yijk 4 | ijk )
0.2
0 -6 -5 -4 -3 -2 -1 0 1 2 3
ijk
unpleasant
pleasant
response probability
P (Yijk = s | ijk )
1
good
P (Yijk = s | ijk ) = P (Yijk s | ijk ) P (Yijk s + 1| ijk )
0 -not at all
0.8
very much so - 4
0.6
P (Yijk = 3 | ijk ) = P (Yijk 3 | ijk ) P (Yijk 4 | ijk )
1
0.4
0.2
0 -5 -4 -3 -2 -1 0 1 2 3
ijk
unpleasant
pleasant
92
Factor Analysis for Ordered Categorical Data
Assumption:
* Yijk
latent continuous variable, causes observed responses
split into c parts Each part corresponds to one observed category
* cijk 1, for c 1 ijk < Yijk ( ijk ) * Yijk = s, for sijk < Yijk ( s +1) ijk * 0, for Yijk 1ijk
i = item; j = trait; k = method

93
It can be shown that the latent variable Y*ijk is a function of the itemspecific probit ijk and a residual *ijk = Y*ijk - ijk (see Eid, 1995). This yields the following variance decomposition:
* * Var (Yijk ) = Var ( ijk ) + Var ( ijk )
Because in the graded response model the category characteristic curves are defined on the inverse of the standard normal distribution, the variance of the residual variable has to be 1 (Eid, 1995). As a consequence the variance of the latent variable only depends on the item-specific probit variable :
* * Var (Yijk = Var + Var ( ijk ) ) ( ijk ) = Var ( ijk ) + 1
CFA for ordered categorical data using Graded Response Theory (short)
There is an underlying dimension for the observed ordered categories (Graded Response and CFA) Polychoric (tetrachoric) correlations can be used to estimate model Y* is normally distributed WLSMV estimator in Mplus
95
Alternative
Using robust estimators E.g., MLR in Mplus Five categories needed
96
Cross-Cultural Research
How can one determine if the same construct is measured in different cultures? How can one determine if the questionnaire (the scale) works the same way in different cultures?
97
Multigroup MTMM models

Common language: e.g., USA, UK, and India simultaneous estimation of MTMM models for each culture convergent and discriminant validity coefficients should be approximately the same measurement invariance
absolute: factor loadings, intercepts, and residual variances are identical weak: same items load on the same factors
98
Multigroupmodel Extraversion in a French and a German Speaking Sample
F: Extraversion
Var(F) 1
D: Extraversion
Var(D)
1
E1 1
2
E2 2
2
E2 2
E1 1
Var(1)
Var(2)
Var(1)
Var(2)
99 99
Cross-Cultural Research
Different Languages: French and German speaking parts of Switzerland Guidelines for the translation of instruments
translation and back translation
Determine if structure and discriminant validity of questionnaire is the same across cultures
100
Longitudinal Data
measure of a stable trait or measure of a state? changes in convergent and discriminant validity over time? additional information about stability of traitand method effects available
101
Multioccasion CSC(M-1) Model
102
Multiconstruct-LST Model
103
Multimethod LST Model
104
Clustered data?
IID assumption no longer met!
independent and identically distributed data
biased parameter estimates standard errors and test-statistics no longer trustworthy
105
Multilevel CTUM-Model
Level 2 (Target)
Ert11 Yrt11 1 M21 M31 Mrt1
Level 1 (Rater)
Trait 1
Yrt21 Yrt31
Yrt12
1 M22 M32 Mrt2
Trait 2
Yrt22 Yrt32
Yrt13
Trait 3
Yrt23 Yrt33 ISSAS 2009 - scale construction
1 M23 M33 Mrt3
106
106
Multilevel CTUM-Modell
Level 2 (Target)
Tt11 1 Yrt11 Yrt21 Yrt31
Level 1 (Rater)
Ert11 1 M21 M31 Mrt1
Trait 1
Tt21 Tt31
Tt12
Yrt12 Yrt22 Yrt32
1 M22 M32 Mrt2
Trait 2
Tt22 Tt32
Tt13
Yrt13 Yrt23
Trait 3
Tt23 1 1 Yrt33 Tt33 ISSAS 2009 - scale construction
1 M23 M33 Mrt3
107
107
Variance Components
Var (Yrtik ) = Var (Ttik ) + 2 MikVar ( M rtk ) + Var ( Ertik )

Var (Ttik ) + 2 Var ( Ertik ) MikVar ( M rtk ) Rel (Yrtik ) = = 1 Var (Yrtik ) Var (Yrtik )
CO (Yrtik ) = Var (Ttik ) Var (Ttik ) + 2 MikVar ( M rtk )
2 Mik Var ( M rtk ) MS (Yrtik ) = Var (Ttik ) + 2 MikVar ( M rtk )

108
108
Multilevel CTC(M-1) Model
109
109
References
Scale construction: Clark, L.A. & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319. Haynes, S.N., Richard, D.C.S, & Kubany, E.S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7, 238-247. Spector, P. E. (1992). Summated Rating Scale Construction: An Introduction. Newbury Park,California: Sage Publications.
References
MTMM / SEM: Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley. Bollen, K. A., & Long, J. S. (1993). Testing Structural Equation Models. Sage. Campbell, D. T. & Fiske, D. W. (1959). Convergent and discriminant valuidation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105. Courvoisier, D. S., Nussbeck, F. W., Eid, M., Geiser, C., & Cole, D. A. (2008). Analyzing the convergent validity of states and traits: Development and application of multimethod latent state-trait models. Psychological Assessment, 20, 270-280. Eid, M., Nussbeck, F. W., Geiser, C., Cole, D. A., Gollwitzer, M., & Lischetzke, T. (2008). Structural equation modeling of multitraitmultimethod data: Different models for different types of methods. Psychological Methods, 13, 230-253. Finney, S. J., & DiStefano, C. (2006). Non-normal and categorical data in structural equation modeling. In G. R. Hancock & R. O. Mueller (Eds.). Structural equation modeling: A second course. Greenwich, CT: Information Age Publishing. Hoyle, R. H. (Ed.) (1995). Structural equation modeling. Concepts, issues, and applications. Thousands Oaks: Sage. Kline, R. B. (Ed.) (2004). Principles and practice of structural equation modelling (2nd ed.) New York: Guilford Press. Loehlin, J. C. (2003). Latent variable models: An introduction to factor, path, and structural analysis (4th ed.). Mahwah: Erlbaum. Nussbeck, F. W., Eid, M. & Lischetzke, T. (2006). Analysing MTMM data with SEM for ordinal variables applying the WLSMVestimator: What is the sample size needed for valid results? British Journal of Mathematical and Statistical Psychology, 59, 195-213. Raykov, T., & Marcoulides, G. A. (2006). A First Course in Structural Equation Modeling (2nd ed.). Mahwah: Erlbaum. Schermelleh-Engel, K., Moosbrugger, H., & Mller, H. (2003). Evaluating the fit of structural equation models: Test of significance and descriptive goodness-of-fit measures. Methods of Psychological Research - Online, 8, 23-74. Download: http://www.mpr-online.de/ Schumacker, R. E., & Lomax, R. G. (2004). A beginners guide to structural equation modelling (2nd ed.) Mahwah: Erlbaum. Other: Burns, G. L. & Haynes, S. N. (2006). Clinical Psychology: Construct validation with multiple sources of information and multiple settings. In M. Eid and E. Diener (Eds.) Handbook of Multimethod Measurement in Psychology. Washington, DC: American Psychological Association. Messick, S. (1995). Validity of psychological assessment: validation inferences from persons responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749. Steyer, R., Schwenkmezger, P., Notz, P. & Eid, M. (1994). Testtheoretische Analysen des Mehrdimensionalen ISSAS 2009 - scale construction 111 Befindlichkeitsfragebogens. Diagnostica, 40, 320-328.

Scale Construction

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Scale Construction

Uploaded by

Copyright:

Available Formats

Scale Construction

Methods, Statistics, and Models

Fridtjof Nussbeck, University of Zurich

statistics MTMM models outlook

ISSAS 2009 - scale construction

Measurement and Validity

ISSAS 2009 - scale construction

II. Scale Construction

ISSAS 2009 - scale construction

When to Construct a New Scale?

Phases of test / scale construction

ISSAS 2009 - scale construction

definition of the construct in literature?

emotional reactions (baroque, rational, positive reappraising)

frequency of experiencing positive affect

ISSAS 2009 - scale construction

ISSAS 2009 - scale construction

item generation is a creative act

Ex.: Do you feel happy? never sometimes always

ISSAS 2009 - scale construction

individuals must differ on items

ISSAS 2009 - scale construction

avoid item wordings including negative mood terms

avoid negatives to reverse meaning of an item, but include negative aspects

Multidimensional Mood Questionnaire

items of the PleasantUnpleasant dimension

Steyer, Schwenkmezger, Notz, & Eid (1994)

ISSAS 2009 - scale construction

ISSAS 2009 - scale construction

Do you agree to.

How much does correspond to you

How do you judge

ISSAS 2009 - scale construction

ISSAS 2009 - scale construction

Administration and Item Analysis

ISSAS 2009 - scale construction

Validate and Norm

Mean =1.86 Std. Dev. =0.71 N =481 0 0 1 2 3 4 5

ISSAS 2009 - scale construction

ISSAS 2009 - scale construction

factor analysis for ordinal data

ISSAS 2009 - scale construction

ISSAS 2009 - scale construction

Exploratory Factor Analysis

for all items i

for some items i for some (other) items j

Exploratory Factor Analysis

ISSAS 2009 - scale construction

Criteria for Factor Analysis

ISSAS 2009 - scale construction

Eigenvalues & Explained Variance

ISSAS 2009 - scale construction

ISSAS 2009 - scale construction

Exploratory Factor Analysis

ISSAS 2009 - scale construction

ISSAS 2009 - scale construction

Exploratory Factor Analysis

ISSAS 2009 - scale construction

ISSAS 2009 - scale construction

Confirmatory Factor Analysis

ISSAS 2009 - scale construction

Criteria for Confirmatory Factor Analysis

ISSAS 2009 - scale construction

association of one item with scale (corrected) item-total correlation (discrimination)