Reliability and Validity

Temilade Adefioye
Content
Overview of Reliability and Validity
Reliability and Validity in Quantitative Research
Reliability and Validity in Qualitative Research
Reliability and Validity in Mixed methods
Brief overview
Reliability and Validity are important concepts in research as
they are used for enhancing the accuracy of the assessment
and evaluation of a research work (Tavakol and Dennick,
2011, p.53)
They have different meanings under the different types of
research i.e. quantitative and qualitative research (Creswell,
2014, p. 201).
Quantitative Research - Reliability and Validity
RELIABILITY refers to the consistency, stability and
repeatability of results i.e. the result of a researcher is
considered reliable if consistent results have been obtained in
identical situations but different circumstances (Twycross and
Shields, 2004, p.36)
VALIDITY is the extent to which any measuring instrument
measures what it is intended to measure (Thatcher, 2010,
p.125).
It is possible for a measurement to be reliable but invalid;
however, if a measurement is unreliable, then it cannot be
valid (Thatcher, 2010, p.125; Twycross and Shields, 2004,
p.36).
Quantitative Research - Reliability and Validity
Example
A table clock that is always five minutes fast
It is reliable because it is always five minutes fast, however, it
is not valid because when compared to a standard format
such as the GMT, it is not correct.
Quantitative Research Types of
Reliability
Stability - This is when a researcher obtains the same result in
repeated administrations or when the same test tools are used on
the same sample size more than once, and when there is a
reliability co-efficient that provides an indication of how reliable
the tool is
Homogeneity - This is a measure of the internal consistency of
the scales. Cronbachs alpha is used to measure the reliability of a
tool.
Equivalence - This is level of agreement among researchers
using the same data collection tool. The ratings of two or more
researchers are compared by calculating a correlation co-efficient.
(Twycross and Shields, 2004, p.36)
Quantitative Research Types of
Validity
Criterion-related validity It is also called Predictive Validity. According to Nunnally
(1978) cited in Thatcher (2010, p.125, 145), predictive validity is when the purpose is to use an
instrument to estimate some important form of behaviour that is external to the measuring instrument
itself . For instance, a written drivers license test can be validated by hypothesizing that the
test will accurately predict driving competence amongst a group of drivers. If this test fails to
accurately predict this competence, then it is not valid.
Content validity It is the extent to which an empirical measurement reflects a specific
domain of content. For instance, in arithmetic operations, the test problem will be content
valid if the researcher focuses on addition, subtraction, multiplication and division but will be
content invalid if the researcher focuses on one aspect of arithmetic alone say, subtraction. It
is also called Face validity (Thatcher, 2010, p.125, 141).
Construct validity It is concerned with the validity of empirical measures and hypothesis
testing of theoretical concepts. It is the extent to which a particular measure relates to other
measures consistent with theoretically derived hypotheses concerning the concepts that are being measured
(Carmines and Zeller, 1979, cited in Thatcher, 2010, p.147).
Qualitative Research Reliability and
Validity
RELIABILITY is referred to as when a researchers
approach is consistent across different researchers and
different projects
VALIDITY is when a researcher uses certain procedures to
check for the accuracy of the research findings
(Gibbs, 2007 cited in Creswell, 2014, p.201).
Qualitative Research Reliability
Strategies
Document procedures and steps taken in the research work (Yin
2009, cited in Creswell, 2014, p.204).
Setting up a detailed protocol and database so that others can
follow the procedure (Gibbs, 2007 cited in Creswell, 2014, p.204).
Check transcripts for errors and obvious mistakes (Creswell, 2014,
p.204).
When using codes, check to see that there is consistency in code
definition (Creswell, 2014, p.204).
In team research, there should be coordinated communication
through regular documented meetings and analysis sharing (Creswell,
2014, p.204).
Cross-check codes developed by different researchers by comparing
results that are independently derived (Creswell, 2014, p.204).
Qualitative Research Validity
Strategies
Triangulate Examining evidence of information from different sources of data
Member checking This is when the final report or specific descriptions from a research work is
taken back to the participants in order to determine if the participant feel that the results are
accurate.
Comprehensive narration This is the use of a rich, thick description to convey findings
Bias Clarification This is when the researcher clarifies the bias he/she brings to the study
Presenting negative or discrepant information that runs contrary to the information in the
findings
Time Spending prolonged time in the field in order to gain an in-depth understanding of the
problem under study.
Peer debriefing Locating a peer debriefer to ask and review questions about the study so that it
will not be only the researcher asking questions about the problem
External auditor Using an external auditor to review the entire project. This individual is not
familiar with the research problem and can be obejective in the assessment of the research work.
(Creswell, 2014, p.201-204, 210-211)
Mixed Methods Research
When using the mixed methods approach, validity can be
carried out by using a convergent approach i.e. a strategy
from the quantitative method e.g. content validity and
another from the qualitative method e.g. triangulation
(Creswell, 2014, p.223, 225)
End
Comments?
Questions?
Observations?
References
CRESWELL, R. (2014). Research design: qualitative, quantitative, and
mixed methods approaches. USA: Sage Publications
TAVAKOL, M. & DENNICK, R. (2011). Making sense of
Cronbachs alpha. International journal of Medical Education, 2,
pp.53-55
THATCHER, R. (2010). Validity and reliability of quantitative
electroencephalography (qEEG). Journal of Neurotherapy, 14, pp.
122-152.
TWYCROSS, A. & SHIELDS, L. (2004). Validity and reliability -
What's it all about? Part 2 Reliability in quantitative studies.
Paediatric Nursing, 16 (10) p. 36