INTRODUCTION TO DESIGNING NEEDS

ASSESSMENT SURVEYS









An AEA Mini Workshop presented by

James W. Altschuld The Ohio State University
Yi-Fang Lee National Chi Nan University
Jeff White University of Louisiana at
Lafayette
Hsin-Ling Hung University of North Dakota

Annual Meeting of the American Evaluation
Association
November , 2010
San Antonio, Texas

OBJECTIVES/AGENDA
Objectives
Understand Needs Assessment
Learn where versions of surveys fit the picture
Quick overview of survey parameters in NA
Get a hands-on feel for the process
Have the opportunity to discuss/comment on the
experience
Hopefully to have some fun on your brief journey
into the world of NA

Also to provide you with a glimpse into the Needs
Assessment Kit (Altschuld, Eastmond, King,
Kumar, Stevahn, and White, 2010)
AGENDA
Introduction (1 minute)
Questions (3-5 minutes)
Basic terms and concepts (8-10 minutes)
Survey overview (15 -18 minutes)
Types and where they fit the process
Samples of questions (scaled or otherwise)
General analysis strategies
Wording problems
Scenario overview, hands-on work, discussion,
etc. (rest of time allotted for the session)



BEGINNINGS
Questions for you
How many have done
NAs?
Have you used surveys in
the process?
For what purposes?
Briefly describe the
survey?
How did you analyze the
data?
Overall success of the
effort?
Did you use other methods
as well what were they?
Any other comments about
what you did?

Terms/concepts
Need
-definition with attention to use
of words (need & needs)
-not mixing solutions with
needs
-definitional problem
immediately for wording
survey items
-examples of needs (see
partial table from book 1)
-types of needs (book 1)
-levels of needs
-Wow, issues at the outset
Needs assessment
-three phase model (see
schematic)
-types of surveys and where
they might fit
-why you still might have to
include solutions in survey
despite above admonition


DEFINITIONS & ISSUES
Need: the measurable discrepancy between what
is or the present state of affairs in regard to the
group and situation of interest and the what
should be or desired state of affairs (Witkin &
Altschuld, 1995).

Issues: measurable discrepancy is the key
-needs not solutions (premature closure on
solutions)
-verb vs. noun concept (misuse of the word)
-desired, likely to occur, ought to occur, etc.
-wish and want lists
-many types of needs

MORE TERMS AND CONCEPTS
NA is a systematic set of procedures
undertaken for the purpose of
setting needs-based priorities and
making decisions about
organizational improvement and
allocation of resources (Witkin &
Altschuld, 1995).

Issues
-context for the NA
-readiness for an assessment
-NA is an organizational activity
-political aspects to the activity
-systems concept and how to
think about it

Lots of subtle aspects of need and
NA
Examples
Types
Why do it and how does it relate
to evaluation

Source: from Needs Assessment Kit , by J. W. Altschuld and D. D. Kumar , 2010,
Thousand Oaks, CA: Sage.
Source: from Needs Assessment Kit , by J. W. Altschuld and D. D. Kumar , 2010,
Thousand Oaks, CA: Sage.
Source: from Needs Assessment Kit , by J. W. Altschuld and D. D. Kumar , 2010,
Thousand Oaks, CA: Sage.
SURVEY CONSIDERATIONS
Sampling
Level 1, 2, and/or 3
Wording issues
Where in the process
For NAC?
Phase 1, 2, 3
What type of need
Short term vs long term
Maintenance
Collaborative
Other
Double, triple scales or more
Content
Behaviors
Attitudes
Content (continued)
Degree of achievement
Satisfaction with services
Importance/value of services
Motivation to resolve
problems
Frequency of use
Observations
Feasibility of resolving
problems
Literature based content
Impediments to solutions
List goes on and on
Formatting
Scales (what type)
Open-ended
Bars

Examples of Surveys but first
Table 3.1 Comparisons of Survey Usage for Organizing the NAC and for
Conducting Needs Assessments
Table 3.1 (Continued)
Table 3.1 (Continued)
Source: from Needs Assessment Kit 2, by J. W. Altschuld and J. N. Eastmond, 2010, Thousand Oaks, CA: Sage.
Table 3.2 Four Basic Survey Formats for Kicking Off Initial NAC Discussions
Source: from Needs Assessment Kit 2, by J. W. Altschuld and J. N. Eastmond, 2010, Thousand Oaks, CA: Sage.
Table 3.5 Structure of the NAC Survey
Table 3.5 (continued)
Source: from Needs Assessment Kit 2, by J. W. Altschuld and J. N. Eastmond, 2010, Thousand Oaks, CA: Sage.
Humann (1997)
Example of First Student Survey
OSEA (2005)
Example of Follow-Up Survey
Lee, Altschuld, & White (2007)
NOW THE HANDS-ON EXERCISE
Scenarios/orientation
Pretend that you are the needs
assessor
Individually and then collectively
What can a NAC tell you?
Questions
Content
Discussion focusing
Sketch out areas where they
could give you guidance/input
So you need a survey for other
groups
Review item writing rules
Review examples

Considerations for NA
surveys
What content
Levels 1, 2, and/or 3
Relation to other
methods
Use of multiple
scales
Impediments
Open-ended questions