RESEARCH METHODOLOGY AND DATA ANALYSIS

[MPU/PPU0020; MPPU1060/1074]
SEM I 2018/2019
OVERVIEW OF MIXED METHOD RESEARCH
METHODOLOGY
DR NOR FADILA MOHD AMIN
28 OCTOBER 2018 (SUNDAY)
8.30 AM – 10.30 AM
SEMINAR ROOM 1, LEVEL 3, BLOK D05
FACULTY OF SOCIAL SCIENCES & HUMANITIES
 OVERVIEW OF
MIXED METHOD RESEARCH METHODOLOGY

RESEARCH PARADIGM/
PROCESS PHILOSOPHY
07 01
DATA
COLLECTION TYPES
06 02

INSTRUMENTATION 05 03 CHARACTERISTICS
04
SAMPLING
what is research
PARADIGM
 Thomas Kuhn (1962)
A philosophical way of thinking
 IN EDUCATIONAL RESEARCH
A researcher’s ‘worldview’
the perspective, school of thought, set of shared beliefs
that informs the meaning & interpretation of
research data
(MacKenzie & Knipe, 2006)
A basic set of beliefs or worldview
That guides research action or an investigation
(Guba & Lincoln, 1994)

Human constructions
Which deal with first principles or ultimate indicating
Where the researcher is coming from so as to
construct meaning embedded in data
(Denzin & Lincoln, 2000)
 PARADIGM is important
They provide beliefs and dictates, which for scholars in
a particular discipline, influence :

§ What should be studied

§ How it should be studied
§ How the results of the study should be interpreted
 PARADIGM is important
has significant implications for every decision made in
the research process, including METHODOLOGY
and METHODS
 Ontology Epistemology Theoretical Methodology Method
What is How can I Perspective How do you What
reality? know reality? Which go about techniques do
PARADIGM approach do finding? you use to find
you use to out?
know
something?
Positivism There is a Reality can be Positivism § Experimental Usually
single reality measures and Post-positivism research quantitative,
or truth (more hence the focus § Survey could include :
realist) is on reliable and research § Sampling
valid tools to § Measurement
obtain that & scaling
§ Statistical
analysis
§ Questionnaire
§ Focus group
interview
Constructivist There is no Therefore, the Interpretivism § Ethnography § Qualitative
single reality reality need to (reality needs to § Grounded interview
/Interpretive or truth. be interpreted. It be interpreted) theory § Observation
Reality is is used to § Phenomenolo § Phenomenolo participant/
created by discover the gy gical research non
individuals in underlying § Symbolic § Action participant
groups (less meaning of interaction research § Case study
realist) events and Critical inquiry § Discourse § Narrative
activities Feminism analysis § Theme
identification
 Ontology Epistemology Theoretical Methodology Method
What is How can I Perspective How do you What
reality? know reality? Which go about techniques do
PARADIGM approach do finding? you use to find
you use to out?
know
something?
Pragmatism Reality The best Deweyan § Mixed Combination of
constantly method is one pragmatism methods any the above
renegotiated, that solves Research § Design based and more, such
debated, problems. through design research as data mining,
interpreted in Finding out is expert review,
light of its the means, usability testing,
usefulness kin change is the physical
new underlying aim prototype
unpredictable
situations
 How We Make Sense

NUMBERS PERSONAL  EXPERIENCE

A MIXED METHODS APPROACH

 HOW YOU GOING
TO DESIGN YOUR STUDY?
 Q1 Do you have mixed methods study?
(or simply a quantitative study or a
qualitative study)
§ Do you have a quantitative database? (close-ended)
§ Do you have a qualitative database? (open-ended)
§ Do you plan on bringing the two databases together
(integration)
§ What procedures (design) will you use?
 What I am NOT looking for are :
§ Keeping the database separate
§ the collection of BOTH quantitative and qualitative
data (not content analysis)
§ Multiple forms of quantitative data or multiple forms
of qualitative data (multi-method)
§ Only the transformation of qualitative data into
quantitative (in analysis)
§ Only formative/summative evaluation
§ Only adding in qualitative data into an experiment
§ Doing mixed methods because it’s trendy
 Q2 Does your research problem or
question merit a mixed methods study
When is mixed methods suitable for a problem or
question?
When qualitative research or quantitative research is
insufficient to fully understand the problem
Other types of problems, when: 
Other types of problems, when :

• We need to explore before we administer
instruments
• We need to explain our statistical results
by talking to people
• We need to see if our quantitative results
and our qualitative results match
• We need to enhance our experiments by
talking with people
• We need to develop new instruments by
gathering qualitative data
Q3 What is the overall intent of your
study?

My goal:

• To learn whether you are 
quantitatively‐ or qualitatively‐
oriented by the way you state
your intent
K
e
y
wQuantitative Qualitative
ofactors explore

rdeterminants generate
relationship develop
d
causes create
sinfluences meaning
effects personal experiences
i
n

y
o
 Your Keywords:

1. Will help me think about what mixed 
methods design you will likely use
2. Will help me think about your discipline
field background that you bring to mixed 
methods
3. Will help me think about what skills you
have to begin conducting a mixed methods
study
Q4 What quantitative and qualitative
dataI will ask you to list your quantitative and 
will you collect to address this
intent?
qualitative data sources:

List your quantitative and qualitative data sources :

Quantitative Data Qualitative Data
Site and sample?
How many?
Types of data to 
collect
Specific 
instruments/protoc
ols
Types of 
scales/questions
 I will ask you to list your quantitative and 
qualitative data analysis steps:
List your quantitative and qualitative data analysis steps :

Quantitative  Qualitative  
Analysis Analysis
Getting data ready 
for analysis
Preliminary analysis
Steps in analysis
(including statistics,
themes)
 List Multiple Types of Quantitative and 
Qualitative Data to be Collected and Analyzed

QUANTITATIVE RESEARCH QUALITATIVE RESEARCH
• Quantitative data  • Qualitative data collection 
collection(closed‐ (open‐ended)
ended) – Interviews
– Instruments – Observations
– Behavioral checklists – Documents
– Records – Audio‐visual materials

• Quantitative data  • Qualitative data analysis
analysis – Use text and image data,
– Use numeric data, • For coding
• For description • For theme development
• For comparing groups • For relating themes
• For relating variables

27
 Are These Procedures Realistic and Feasible?

• Time
• Economically
• Skills
• Useful for stakeholders
Q5 Why are you collecting both
quantitative and qualitative data?
Basic reasons for using mixed methods
(called the rationale in mixed methods):

• Need different, multiple perspectives, or more complete 
understandings
• Need to confirm our quantitative measures with qualitative 
experiences
• Need to explain quantitative results
• Need better contextualized instruments, measures, or 
interventions to reach certain populations
• Need to enhance our experiments
• Need to gather trend data and individual perspectives from 
community members
• Need to evaluate the success of a program by using a needs 
assessment AND a test of the success of the program
Q6 Do you plan on using a theory? Do
you plan on advancing your
philosophical stance?
 Frame the study within theory/philosophy:

Paradigm/Worldview (epistemology, ontology, axiology,

methodology)

Theoretical lens
(e.g., feminist, racial, social science theories)

Methodological approach (e.g., ethnography,

experiment, mixed methods)

Methods of data collection (e.g., interviews,

checklists, instruments)

Adapted from Crotty M. (1998)

32
 Further questions about philosophy:

• What is a philosophy?

• Which philosophies are typically
used as the foundation in mixed 
methods research?
Philosophies to consider in mixed methods:

• Pragmatism
• Transformative worldview
• Critical realism
• Dialectic pluralism

• Multiple worldviews/paradigms
• Match worldview to design
• Worldview comes from your scholarly
community
 Further questions about theory:

• What is a theory?

• How do you find a theory?

• How will you use it in your mixed methods
study?
Q7 What mixed methods design will
you use?
Parsimonious set of designs:  Basic and Advanced

Basic Designs
Convergent Design
Explanatory Sequential Design Advanced Designs
Exploratory Sequential Design Basic       
Designs     
Advanced Designs
Intervention Design
Transformative Design
Multiphase Design
Basic mixed methods designs

Convergent Parallel Design

Quantitative Data Quantitative
Collection and  Results
Analysis
Merge
Results Interpret or Explain 
For Comparison Convergence/Divergence
Qualitative Data Qualitative
Collection and  Results
Analysis

Explanatory Sequential Design

Quantitative Data Determine  Qualitative Data Interpret How 

Quantitative Quantitative Qualitative Qualitative Data 
Collection and  Collection and 
Results Results to  Results Explains 
Analysis Analysis Quantitative Results
Explain

Exploratory Sequential Design
Quantitative Data
Use Results to Collection and  Interpret How
Qualitative Data Qualitative Form Variables, Analysis Based on Quantitative Quantitative Results
Collection and  Results Instruments,  Variables, Results Provide New Results,
New, Better Instruments,
Analysis Interventions Instruments,  and Better Interventions
Interventions
 Advanced mixed methods designs
Intervention Mixed Methods Design
Experimental Study

Qualitative
Data Collection, Experiment Group Pre‐test Intervention Post‐Test Qualitative
Analysis, and  Data Collection,
Results Control Group Pre‐test Post‐Test Analysis, and Results
(Exploratory) (Explanatory)
Qualitative Data Collection,
Analysis, and Results (Convergent)

Social Justice Design (using an Explanatory Sequential Design example)

Quantitative  Qualitative Data Interpret How 

Theory Research Quantitative Qualitative Qualitative Data 
Data Collection Collection and  Results
Questions Results Analysis
Explains 
e.g., survey Quantitative Results
And Calls for Action
Promote Social Justice

Multistage Evaluation Design (using an Exploratory Sequential Design example)
Single Program Objective
Summative
Formative Program
Formative Needs  Theory/ Instrument  Program
Program Revision
Assessment Conceptual Framework Development Evaluation
Assessment
(qualitative (based on qualitative (based on  (pre‐post 
(qualitative
data collection) results) quantitative tests) quantitative
data collection)
tests)
How do you choose your design?

• First select your basic design, then
add if you have an advanced design
• Consider your scholarly community:
quantitative or qualitative? (how to begin 
your study)
• Consider your skills:  quantitatively‐strong
or qualitatively‐strong (what will be 
emphasized)
• Consider your resources (collect at one time
or space out)
• Consider the complexity of the design (what 
is the easiest? the hardest?)
Q8 Can you draw a diagram of your
design?
 Convergent Design Wittink et al. (2006)

Source: based on Wittink et al.(2006)

 Intervention Design (Rogers, Day, Randall,, & Bentall , 
2003, study to improve the management of anti‐psychotic 
medication)

Flow of the experiment

Experimental Methodology
qual qual Overall
before QUAN QUAN after results
intervention Pre- Intervention Post- & intervention and
Follow-up Follow-up interpretation
Measure Measures

Procedures: Procedures: Procedures: Procedures:

• One-on- •Three groups: control group, compliance • One-on-one semi- • Discuss treatment
one semi-structured Intervention group, alliance intervention structured interviews – effectiveness
interviews group – group comparisons exiting the trial, • Discuss themes in
• Thematic analysis •Outcome measures: 1) attitudes toward participants from two context of
medication 2) adherence to treatment experimental conditions; interventions
3) avoidance of relapse • Thematic analysis and outcomes
•DAI measure completed 3 times (pre, post,
& follow up) Products: Products:
• Transcripts Discussion
Products: Products: • Themes and quotes
• Transcripts • Numerical item scores
• Developed intervention • Change scores
treatment • Test statistics
 Q9 Can you write a good purpose
statement (or study aim) that fits your
design?
 How to write a mixed methods study aim:

• This mixed methods study will address [overall content 
aim]. A convergent parallel mixed methods design will  Intent
be used, and it is a type of design in which qualitative 
and quantitative data are collected in parallel, analyzed  Design
separately, and then merged. In this study, 
[quantitative data] will be used to test the theory of 
[the theory] that predicts that [independent variables] 
will [positively, negatively] influence the [dependent 
variables] for [participants] at [the site]. The [type of  Data
qualitative data] will explore [the central phenomenon]  Collection
for [participants] at [the site]. The reason for collecting 
both quantitative and qualitative data is to [the mixing  Rationale
reason].

46
Q10 Can you write research questions
for your mixed methods study?
Write 3 types of questions:

• Quantitative questions or 
hypotheses

• Qualitative questions

• A mixed methods question
 Basic design mixed methods questions:

Convergent Design – To what extent do the 
quantitative and qualitative 
results converge?
Explanatory Design – In what ways do the 
qualitative data help to 
explain the quantitative 
results?
Exploratory Design – In what ways do the 
quantitative results generalize 
the qualitative findings?
 Advanced design mixed methods questions:

• Intervention  Design – How do the qualitative findings 
provide an enhanced understanding 
of the quantitative results?

– How do the qualitative findings 
provide an enhanced understanding 
• Transformative Social Justice  of the quantitative results in order 
Design to explore inequalities?

– How to the different phases in the 
project to address the overall 
• Multistage Design research goal?
Place these steps in order for a
Now place these steps in order
good mixed methods study
for a good mixed methods project:

• Problem
• Theory and/or philosophy
• Purpose (study aim)
• Rationale for gathering both
quantitative and qualitative data
• Research questions (quantitative,
qualitative, mixed)
• Types of data/analysis
• Definition of mixed methods
• Type of mixed methods design
• Diagram of the design
THANK YOU