Running head: IDENTIFYING AND DEALING WITH VALIDITY THREATS

Maxwell Exercise 6.1 Identifying and Dealing With Validity Threats

Karin Fonseca
Valdosta State University

IDENTIFYING AND DEALING WITH VALIDITY THREATS

Maxwell Exercise 6.1 Identifying and Dealing With Validity Threats

In this chapter Maxwell states that research bias as well as reactivity are two specific validity threats that
must be addressed, furthermore the main idea is to how the research can rule out specific threats to the
interpretation and conclusion of the phenomena being studied. Because I am a visual person, I developed
a Matrix to better explain any biases that can be present in the study and how do I seek to deal with them.
Validity Matrix for a Study of Attendance Decline in Congregations in America. Determining Factors and Tools That Strengthen
Organizational Leadership Church to Growth in the 21st Century.
What do I need to
know?

Why do I need to
know this

What kind of data

will answer the
questions

Analysis Plans

Validity Threats

Possible strategies for

dealing with validity threats

What factors describe

church leadership in
the 21st century?

-Membership
Attendance Growth
-Leadership styles and
strategies
Organizational
leadership
-Congregations
challenges in the 21st
century

Documents/Reports/
Videos
Previous Research
Interviews
Questionnaires
Observations
Field Notes
Leadership/ Statistical
Reports

Audio/video taping
-Transcriptions
-Theory analysis
-Search for Themes
-Coding
-Use of CAQDAS
-Matrix
-Observations
-Field Notes
-Memos
-Connecting
Strategy

-Deep Culture of Faith might

- Intense review of theories,

documents and expert material
-Intense self-critique of researcher
biases
-Use of open ended script
minimizing research bias
-Development of ongoing
relationships to better understand
the culture and participants natural
and real points of view
-Critical analysis of data provided
by participants
-Multiple face-to-face interviews
with member check review
-Multiple observations with
external member that can review
recordings and provide feedback
-Participants review and
authorization of data provided

influence on the accuracy of

participants true opinion.
-Group interviews might limit
the full expression of self.

IDENTIFYING AND DEALING WITH VALIDITY THREATS

3
- Questionnaires
- Past, present and future
documented experiences from
participants
- Triangulation of data
-Review with research committee
in order to review for discrepant
evidence and negative cases
-Comparison to other cases

What strategies and

tools did successful
leaders employ to
adapt to the rapid
changes in society at
CrossPointe church
while reporting growth
within the community?

-Leadership style,
strategies and tools
linked to growth and
success of this
particular organization
-Congregation
Adaptation to change
in the 21t Century
-The success story of
growth in this
particular institution
while many
congregations have
attendance decline
-To better understand
the culture of this
community and how
this impact on the
growth.

Documents/Reports/
Videos
Interviews
Group Interviews
Questionnaires
Observations
(programs, services,
committee meetings)
Field Notes
Statistical Reports

Audio/video taping
-Transcriptions
-Theory analysis
-Search for Themes
-Coding
-Use of CAQDAS
-Matrix
-Observations
-Field Notes
-Memos
-Connecting
Strategy

-Deep Culture of Faith might

influence on the accuracy of
participants true opinion.
-Group interviews might limit
the full expression of self.
-Potential for lack of
variability in feedback of
group interviews

- Intense review of theories,

documents and expert material
-Intense self-critique of researcher
biases
-Use of open ended script
minimizing research bias
-Development of ongoing
relationships to better understand
the culture and participants natural
and real points of view
-Critical analysis of data provided
by participants
-Multiple face-to-face interviews
with member check review
-Multiple observations with
external member that can review
recordings and provide feedback
-Participants review and
authorization of data provided
- Questionnaires
- Past, present and future
documented experiences from
participants
- Triangulation of data
-Review with research committee
in order to review for discrepant
evidence and negative cases
-Comparison to other cases

IDENTIFYING AND DEALING WITH VALIDITY THREATS

How the organizational

leadership strategy of
Cross Pointe Church
seeks community
growth in the 21st
century?

-Leadership style,
strategies and tools
linked to growth and
success of this
particular organization
-Congregation
Adaptation to change
in the 21t Century
-The success story of
growth in this
particular institution
while many
congregations have
attendance decline
-To better understand
the culture of this
community and how
this impact on the
growth.

Documents/Reports/
Videos
Interviews
Group Interviews
Questionnaires
Observations
(programs, services,
committee meetings)
Field Notes
Statistical Reports

Audio/video taping
-Transcriptions
-Theory analysis
-Search for Themes
-Coding
-Use of CAQDAS
-Matrix
-Observations
-Field Notes
-Memos
-Connecting
Strategy

-Deep Culture of Faith might

How does the Pastor at

CrossPointe seek
growth trough the
leadership team and
members of this
organization?

-This individual plays a

fundamental role in the
organization and for
this reason it is
important to
understand hir
perceptions of his
influences and
strategies.
-Leadership style,
strategies and tools

Documents/Reports/
Videos
Interviews
Group Interviews
Questionnaires
Observations
(programs, services,
committee meetings)
Field Notes
Statistical Reports

Audio/video taping
-Transcriptions
-Theory analysis
-Search for Themes
-Coding
-Use of CAQDAS
-Matrix
-Observations
-Field Notes
-Memos
-Connecting

-Pastor and Leadership team

influence on the accuracy of

participants true opinion.
-Group interviews might limit
the full expression of self.
-potential for lack of
variability in feedback of
group interviews

reactivity to researcher
-Deep Culture of Faith might
influence on the accuracy of
participants true opinion.
-Group interviews might limit
the full expression of self.
-potential for lack of
variability in feedback of
group interviews

- Intense review of theories,

documents and expert material
-Intense self-critique of researcher
biases
-Use of open ended script
minimizing research bias
-Development of ongoing
relationships to better understand
the culture and participants natural
and real points of view
-Critical analysis of data provided
by participants
-Multiple face-to-face interviews
with member check review
-Multiple observations with
external member that can review
recordings and provide feedback
-Participants review and
authorization of data provided
- Questionnaires
- Past, present and future
documented experiences from
participants
- Triangulation of data
-Review with research committee
in order to review for discrepant
evidence and negative cases
-Comparison to other cases
- Intense review of theories,
documents and expert material
-Intense self-critique of researcher
biases
-Use of open ended script
minimizing research bias
-Development of ongoing
relationships to better understand
the culture and participants natural
and real points of view
-Critical analysis of data provided

IDENTIFYING AND DEALING WITH VALIDITY THREATS

linked to growth and
success of this
particular organization
-Congregation
Adaptation to change
in the 21t Century
-The success story of
growth in this
particular institution
while many
congregations have
attendance decline
-To better understand
the culture of this
community and how
this impact on the
growth.

How does the

Leadership Team
perceive the
importance of the
growth within their
community at
CrossPointe and how
are they dealing with
it?

-These individuals play

an important role in the
organization and for
this reason it is
important to
understand their
perceptions of their
influences and
strategies.
-Leadership style,
strategies and tools
linked to growth and
success of this
particular organization
-Congregation
Adaptation to change
in the 21t Century
-The success story of

5
Strategy

Documents/Reports/
Videos
Interviews
Group Interviews
Questionnaires
Observations
(programs, services,
committee meetings)
Field Notes
Statistical Reports

Audio/video taping
-Transcriptions
-Theory analysis
-Search for Themes
-Coding
-Use of CAQDAS
-Matrix
-Observations
-Field Notes
-Memos
-Connecting
Strategy

by participants
-Multiple face-to-face interviews
with member check review
-Interview with previous coworkers of the Pastor
-Interviews with previous church
members
-Multiple observations with
external member that can review
recordings and provide feedback
-Participants review and
authorization of data provided
- Questionnaires
- Past, present and future
documented experiences from
participants
- Triangulation of data
-Review with research committee
in order to review for discrepant
evidence and negative cases
-Comparison to other cases

-Deep Culture of Faith might

influence on the accuracy of
participants true opinion.
-Group interviews might limit
the full expression of self.
-potential for lack of
variability in feedback of
group interviews

- Intense review of theories,

documents and expert material
-Intense self-critique of researcher
biases
-Use of open ended script
minimizing research bias
-Development of ongoing
relationships to better understand
the culture and participants natural
and real points of view
-Critical analysis of data provided
by participants
-Multiple face-to-face interviews
with member check review
-Multiple observations with
external member that can review
recordings and provide feedback
-Participants review and

IDENTIFYING AND DEALING WITH VALIDITY THREATS

growth in this
particular institution
while many
congregations have
attendance decline
-To better understand
the culture of this
community and how
this impact on the
growth.

6
authorization of data provided
- Questionnaires
- Past, present and future
documented experiences from
participants
- Triangulation of data
-Review with research committee
in order to review for discrepant
evidence and negative cases
-Comparison to other cases

Reflection:
Although this exercise is only for practicing, it allowed me to further analyze the development of the research as it adds to the
previous exercise 5.1. By understanding the validity threats the researcher might be faced will, allows for developing strategies to
further understand their own possible biases and be able to explain how they may have influenced the study. Furthermore it allows for
possible strategies on how to deal with the validity threats and allows for the conclusions to be tested verifying and avoiding existence
of significant threat to those conclusions. By doing this exercise the researcher can identify the possible biases, but fundamentally
develop and plan strategies to deal with them in a timely manner. This process is critical to the end result of the research and the
possible explanations on the phenomena being studied.