Questionnaire

Design
What is a
Questionnaire?

A set of Questions
designed to generate
the statistical
information from a
specific demographic
needed to accomplish
the research objectives
Surveys are more than a collection
of unambiguous questions
How questions are specified and
put together will influence the
respondents willingness to
participate & the responses they
provide
 Purposes of the Questionnaire
Ensures standardization and comparability of the data
across interviews everyone is asked the same questions
Increases speed and accuracy of recording
Facilitates data processing
Allows the researcher to collect the relevant information
necessary to address the management decision problem
Improper design can lead to?
Incomplete information
Inaccurate data
Higher costs
 Criteria to consider
Does it provide the necessary
information
Does it consider the respondent
Does it meet editing, coding and
data processing requirements
 Designing the Questionnaire
Determine survey objectives: Plan what to measure.
Decide on format. E.g. personal interview, telephone,
self.
Formulate questions to obtain the needed information
Decide on the wording of questions
Decide on the question sequence and layout of the
questionnaire
Using a sample, test the questionnaire for omissions
and ambiguity
Correct the problems (pretest again, if necessary)
 The Major Decisions in
Questionnaire Design
1. Content - What should be asked?
2. Wording - How should each question be phrased?
3. Sequence - In what order should the questions be
presented?
4. Layout - What layout will best serve the research
objectives?

The most difficult step is specifying

exactly what information is to be
Content: Questions must meet 5 requirements

1. Are the questions relevant. Do they

pertain to the research problem
2. Are the questions accurate. Do they
accurately depict the attitudes,
behaviors, etc. intended to investigate
3. Do respondents have the necessary
information?
Qualify respondents
4. Do respondents understand and
interpret the question correctly
5. Will respondents give the information?
 How should the questions be asked?
Format: How much freedom do we give respondents in answering questions
Open-ended questions
What do you look for most in a job?
Is there anything else you would like to add about the product
 Open Ended Questions: key advantages
Wide range of responses and information
can be obtained
Answers based on respondents not
researchers frame of reference
consumers terms
Lack of influence. Don't channel
respondents thinking
Can help interpret closed-ended questions
- why
Particularly useful as introduction to
survey or topic
When its important to measure the
 Open-ended questions: Key disadvantages
Ability and/or willingness of respondent to
answer
Interviewers ability to record answers
quickly or summarize accurately & probe
effectively
Interviewers attitude influences response
Time consuming (interview sessions,
tabulation, classification, assignment,
validation)
Difficulty in coding
Require respondents to be articulate
Respondents may miss important points
Closed-ended questions (Fixed-alternative responses)
What do you look for most in a job?
____ Work that pays well
____ Work that gives a sense of accomplishment
____ Work where you make most decisions by
yourself
____ Work that is steady with little chance of
Advantages
being laid off.
Ease of understanding
Requires less effort on part of interviewer and
respondent
Ease of tabulation & analysis
Less error prone
Less interviewer bias
Less time consuming
Closed-ended questions (Fixed-alternative
questions)
Disadvantages

Middle/Neutral categories often selected

inappropriately (ignorance, safety)
Less opportunity for self-expression or subtle
qualifications
Less involving for respondents
Order of response categories can have major
Key tradeoff
impact on results
Want to get respondent to address issues our
research is concerned with (Forced response)
and at same time give respondent opportunity
to honestly opt out of question (i.e., Dont Know,
No Answer, Neither Agree nor Disagree) so as
not to dilute data collected
 Dichotomous Questions
Should the Alberta Government give consumers an
energy rebate?
1. Agree
2. Disagree
Advantages
Easy to administer and
tabulate
Disadvantages
Prone to large amounts of error since polarized responses
prevent gaining information on the range of variation
Fail to communicate any intensity of feeling
 Multiple Choice Questions
Are all possible alternatives included?
Too many alternatives
Position Bias

Scaled Response Questions

Closed ended questions where the response
choices are designed to capture an intensity of
feeling (Likert, Staple, Semantic differential)
Easy to code and more powerful statistical
tools
Main problem: Respondent
misunderstanding
 Question Wording
Can have major impact on how respondent
interprets question
All respondents should interpret in the same way
Things to avoid
Complexity: use simple, direct, conversational language
leading questions -- that suggest or imply certain answers
loaded questions -- suggest social desirability, or are emotionally
charged. Have you purchased a high quality Sony TV this year
Ambiguity and vagueness: Words such as often, occasionally,
usually, regularly, frequently, many, should be used with
caution. If these words have to be used, their meaning should be
explained properly.
Which province is bigger Manitoba or Alberta? would your
answer be based on population or area?
 More things to avoid

long-worded questions

double-barreled questions. Questions

that refer to two or more issues within the
same question. Where respondent may
agree with only 1 part of multipart
statement.
Do you think Nike offers better pricing
and variety than other brands
Making implicit assumptions

Jargon
 More things to avoid
burdensome questions - that may tax the
respondents memory
How many tubes of toothpaste have you
purchased in the last 3 months
Have you purchased toothpaste in the
past week
Embarrassing, sensitive, or threatening
questions:
Have you charged more on your credit
card than you should
Use third person do you think most
 Question Sequence & Layout Decisions

Initial stages
Screening or qualifying Questions: Have you
been to the movie theatre this month?
Need to gain & maintain respondents
cooperation
Make questionnaire simple for interviewer to
administer
opening questions should be interesting, simple,
and easy to answer.
which theatre did you go to last?
Transition Questions
What aspects of the theatre did you like
best
Questions directly related to research
objectives which require more effort and get
respondent
Difficult andthinking
Complicated
about
Questions
topic
The following 10 questions relate to the
characteristics of theatres
Respondent now committed and can see an end
in sight
Classifying and Demographic Questions
What is your average annual household
income
May not be answered (sensitive or threatening)
 Other Considerations
Questions should flow logically from one to the
next
General questions should be asked before more
specific ones
Earlier questions should not influence response to
later ones
Questions should flow from factual and
behavioural questions to attitudinal and opinion
questions
Questions should flow from the least sensitive to
the most sensitive.
1. initial questions - screening and rapport
Questions should flow from unaided (which brand
questions
do you prefer) to aided questions (which brand do
2. Middle - product specific questions
you prefer Tide, Cheer, Gain)
 Other Considerations
Use multiple questions instead of one
Similar questions together
consistent mindset for respondents
Develop a logical flow
Use transitions between sections (E.g. In this
section we ask questions about X)
Distinguish between instructions, questions and
responses
Distinguishing question and responses:
CAPS or BOLD or Underline versus lower case or unformatted

INSTRUCTIONS LAYOUT A:
Do you agree, disagree or have no opinion that this company has:
A good vacation policy - disagree/not sure/agree.
Good management feedback - disagree/not sure/agree.
Good medical insurance - disagree/not sure/agree.

INSTRUCTIONS LAYOUT B:
Does this company have:
___ Disagree Not Sure Agree
A good vacation policy 1 2 3
Good management feedback 1 2 3
Good medical insurance 1 2 3

Matrix question
Filter and pivot questions should be used
as necessary.

A FILTER question is one that

screens out respondents who are
not qualified to answer a second
question.

A PIVOT question is a type of filter

question that is used to determine
what version of a second question to
ask.
 Developing a logical flow
If the questionnaire deals with several topics,
complete questions on a single topic before
moving on to a new topic
If topics are related, ask questions on related
topics before asking questions about unrelated
topics
If you ask questions about behaviors over some
time period, follow chronological order backward
in time
When changing topics, use some transitional
phrase
 QUESTIONNAIRE LAYOUT
The layout and physical attractiveness of a
questionnaire are important aspects
Questionnaires should be designed to
appear as short as possible
Questionnaires should not appear
overcrowded
Leave lots of space for open ended
questions
Questionnaires in booklet form are often
recommended
OPENING
Provide first name (at least)
Provide name of company doing research
Provide reason for survey and topic
State that no selling will be involved and no
personal data other than for statistical
purposes
Tell respondent approximate time to
complete
Reinforce that respondents time is
AND appreciated
CLOSING
Thank for time
Invite to participate
Ask if they had a positive experience and
remind them that their opinions count
 Pretesting and Correcting Problems
Purpose of pretest: To ensure that the
questionnaire meets the expectations in terms of
the information that will be obtained
Is question necessary does it serve a purpose,
will info be used
Missing important variables does it provide the
info needed
Match questions to objectives
Pretest Specific Questions For
Variation
Meaning
Task difficulty
Respondent interest and attention
Ambiguous, ill-defined, loaded, double-
barreled questions
Pretest the Questionnaire
 Questionnaire Design Flow Chart
Step 1: Specify what information will be sought
Step 2: Determine type of questionnaire and method of
administration
Step 3: Determine the content of individual questions
Step 4: Determine form of response for each question
Step 5: Determine wording for each question
Step 6: Determine sequence of questions
Step 7: Determine physical characteristics of questionnaire
Step 8: Revise steps 1-7, revise if necessary
Step 9: Pretest questionnaire, revise if necessary
Step 10: implement
A QUESTIONNAIRE
IS ONLY AS GOOD
AS THE QUESTIONS
IT ASKS