1

Writing Proposals with

Strong Methodology and
Implementation
Kusum Singh, Virginia Tech
Gavin W. Fulmer, National Science Foundation

Goals
2

Encourage you to seek funding from NSF

for your research.
Help you develop rigorous methodology,
data collection and analysis plans that
will make your proposal competitive.
Help you consider the level of detail
appropriate for implementation projects.

Describing Your Projects

Methodology

Expectations for Methods in

DRL
The DRL Programs welcome research using a

variety of evidence.

The program is open to qualitative,

quantitative, and mixed methods.

Methods must be rigorous and appropriate to

the proposed research questions or
hypotheses.

Design, methods, and analytic techniques

should have a coherent and logical link.

Research methods should be described in

adequate detail.

Details of Methods to
Include 1

Provide a rationale for your research design

Make it clear how the research design and

analyses answer the research questions (RQs)

Include a description of study population

and sampling method, sample size,
expected effect size

Power analysis should inform sample size

decision

Details of Methods to
Include 2

Instruments or protocols to be used

Data analysis plans

Validity, reliability, and triangulation of

measures
Reviewers are cautious about development of
new measures
Statistical Models, procedures for analysis of
text/video/observation data

All of these need to have a rationale for

them that connects to your RQs

Quantitative research
7

Research design (e.g. experimental,

quasi-experimental and nonexperimental designs, issues of internal
& external validity)

Measurement (e.g. data to be collected,

constructs, measures, validity &
reliability of measures)

Data analysis (e.g. statistical decisions,

models & procedures)

Qualitative Research
8

Identify the methodology as a systematic

research design (e.g. case study, discourse
analysis etc.)
Describe how and what data will be collected
Consider issues of validity, and triangulation
Include plans for analysis of textual data
(coding scheme, themes etc.)
Find good balance between planned
approach to analysis and flexibility to
respond to findings

Find the Expertise You Need

9

Content experts are not necessarily

methods experts; so partner with
research methodologists

Sooner is better than later (in proposal

writing stage)
Especially necessary if design is complex or
you use innovative methods

Find a colleague

As co-PI or as consultant

10

Common Missteps in
Methods -1

Overly generic language and description

We will use constant comparative methods.

We will use HLM.

Lack of consistent link between the theory,

the RQs, the data collected, and the analyses

Reviewers will notice.

Methods and planned analyses inadequate to
answer RQs.
Try developing a matrix of RQs, data/measures,
and analyses even if only for you during
planning

11

Common Missteps in
Methods -2

Too little or too much data without clear

analysis plan

Reviewers will wonder if you understand

the task.

Method is novel and not well understood

in field

Needs more detail, examples and citations

to justify that it is appropriate

Summary of Main Points

12

Articulate clearly your research questions or

research hypotheses
Think about the most appropriate and rigorous
methods to answer your research questions
Give a clear and concise description of the
research methods
Include your rationale for research design
decisions
Include a research methods expert in your team
Articulate clearly why your research is
important and how it would contribute to theory
and practice

13

Describing an
Implementation

Details of Implementation
14

There are important implementation

issues that need to be addressed if your
project includes

Curriculum development
Professional development
Interventions

15

For All Implementation

Projects

Consider the method(s) used to gauge the

quality of the implementation

Whether as Fidelity of Implementation (FOI),

Intended/Enacted Curriculum, or other
approaches

Be specific on the STEM content, ages/grades,

settings
Be clear on the roles of the team

Who will lead PD or curriculum, who will oversee

implementation?
Who will collect evaluative data on
implementation?

16

Issues for Curriculum

Development

Specify the STEM content of interest and age

range(s) for which you are developing
curriculum
Specify the role(s) of the PI team, outside
experts, participating teachers, or others
Identify the process for development, revision,
and field-testing

Provide justification for the design process you will

use
Make sure the measures match the
materials/curriculum under development

17

Issues for Professional

Development

Be specific on the professional development (PD)

Identify the model for PD you will use

STEM content, grades, and school settings

Role(s) of the PI team, outside experts, participating
teachers, or others
Format of professional development (e.g., online,
workshops)
Duration and location of PD
Evaluation
Train-the-trainer
Master teacher
Professional Learning Community

Provide justification for the model, the format, and

your teams expertise

Issues for Intervention

18

Describe development history and its

prior use

Provide evidence, if any, for interventions

potential effects

Describe in detail:

Population and sample;

Setting, duration, and content;
Design process, if the intervention will be
revised iteratively

Consider Generalizability
19

If you are developing a new curriculum/PD model:

How will the intervention, curriculum, or the

professional development developed in your setting
apply to new settings that may differ from the study?

If you are applying an intervention, PD model, or

curriculum adopted from another setting:

How well does that intervention apply to your

setting?
Will promising prior results be replicable in this
project?

Evaluation Plan
20

Evaluation should be useful for improving the

research project

Design and content of the plan should be

appropriate to what would enhance or benefit the
project

Formative or summative, internal or external

may be appropriate, depending on the project.
For example, advisory committees are
appropriate for the evaluation of projects.
Go to specific session on Project and
Program Evaluation later in the conference
for more details.

21

Any Questions??
Dont be shy.

22

Thank you!
Feel free to contact Kusum Singh for follow-up and tips for finding a
good methodologist:
ksingh@vt.edu