Professional Documents
Culture Documents
Reliability
Reliability refers to a measures ability to capture an
Reliability
Criteria of reliability
Test-retest
Test components (internal consistency)
Test-retest reliability
Consistency of measurement for individuals over time
The score similarly e.g. today and 6 months from now
Issues
Memory
If too close in time the correlation between scores is due to memory of item
responses rather than true score captured
Chance covariation
Any two variables will always have a non-zero correlation
Reliability is not constant across subsets of a population
General IQ scores good reliability
IQ scores for college students, less reliable
Internal Consistency
We can get a sort of average correlation
power1
replicability1
When
Later replications are not providing as much information,
Meta-analysis
How
There is no perfect replication (different people involved,
Validity
Validity refers to the question of whether our
Construct-related validity
Convergent
Discriminant
Content validity
Items represent the kinds of material (or content areas) they are
supposed to represent
Are the questions worth a flip in the sense they cover all domains of a
given construct?
outcomes
Concurrent
Criterion is in the present
Predictive
Criterion in the future
interest
Convergent
Correlates well with other measures of the construct
Discriminant
Is distinguished from related but distinct constructs
lead to it)
Related to this is the question of whether the study was sufficiently
sensitive to pick up on the correlation
Internal validity
Has the study been conducted so as to rule out other effects which were
controllable?
External validity
Will the relationship be seen in other settings?
Construct validity
Same concerns as before
Ex. Is reaction time an appropriate measure of learning?
Summary
Reliability and Validity are key concerns in
psychological research
Part of the problem in psychology is the lack of
reliable measures of the things we are interested
in1
Assuming that they are valid to begin with, we
must always press for more reliable measures if
we are to progress scientifically
This means letting go of supposed standards when they