BASICS OF MARKET RESEARCH

Why do market research?

The decision to conduct market research should be based on the need for information in order to
make a specific decision. Often decision makers make judgments based on “a sample of one”,
that is, their own thinking or opinions. By taking the time to simply ask for other’s opinions,
leaders will make better decisions. The group conducting the research should be clear about:
1. The purpose of the research – What do you want to know? Avoid the urge to “cover the
waterfront”, and focus on a few specific topics.
2. Why the information is important, and how it will be used. If you don’t have the
resources or intention of acting on the research results you should not do the research.
Market research is a means to an end, not the end itself. It should lead your group to
action.

How can the Alumnae Relations Committee help?

The Alumnae Relations Committee provides consulting support, including:

• Historical information. The ARC can tell you when research was last conducted
with the same or similar groups, and possibly provide copies of the research tool used
and the research results. This will help you avoid unnecessarily re-surveying the
same population, or “re-inventing the wheel” while writing your research instrument;
• Help identifying the best time to conduct research. By coordinating your research
effort with schedules for fundraising, other research, and general communications, we
can help you to avoid over contacting alumnae or conducting research simultaneously
with other groups;
• Help estimating your research costs and resource needs. The ARC can steer your
research efforts toward low-cost solutions that still meet your objectives;
• Survey templates and training materials. The ARC maintains a database of tools
and training materials to assist you;
• Access to free or inexpensive on-line survey tools. Conduct free/inexpensive web
surveys through our sources;
• Review of survey questions. The ARC is available to review your survey questions
and provide advice and feedback;
• Help with taking action based on research results. Research provides the
information for your group to take action. The ARC can help you focus on your next
steps.

The Alumnae Relations Committee is an important resource in helping you learn about your
constituents through market research. The ARC does not conduct research for individuals or
groups, analyze research results, or plan programs based on research results.

To contact the ARC, email alumnaesurvey@mtholyoke.edu, or call the Association at 413-538-

2300 and ask for contact information for ARC staff or committee members.

ARC, August 2005

What market research results CAN do for your group

Well-designed and conducted market research can help you make smarter decisions by providing
you with information about your constituents, including their feelings, motivations, plans,
beliefs, behavior and personal backgrounds. This information can be used in a variety of ways,
including to identify opportunities, develop programs, and improve performance.

Many times research will help steer your path by making educated guesses about the best course
of action.

What market research CAN’T do for your group

Market research will not answer all of your questions and solve all of your problems. It does not
replace a group’s need for self-examination, or for strategic planning.

Top 6 tips:
1. Don’t reinvent the wheel. Make sure the information you need isn’t already available. If
you’re not sure what information may be available to you, contact the ARC at
alumnaesurvey@mtholyoke.edu.
2. Clearly define the primary research goal, and stay focused on it throughout.
3. Have one person who is in charge – the ultimate decision maker. No “design by
democracy”.
4. Plan your project. Dedicate resources for every stage, before beginning.
5. Don’t ask questions if you cannot act on the answers.
6. Involve the ARC. Make use of our expertise at designing usable research, as well as our
historical information, survey templates, training materials, free web survey tools, and
ability to coordinate your research.

You’ve decided you do need market research…now what?

Get organized
Decide who is responsible for your market research effort, and who is in charge. If the
responsible party is a group, select a single person who has final say.
Line up your resources before you begin. Plan who will do each of the following:
• Contact the ARC for recent research results, training tools and templates;
• Determine the purpose of your research, and how you will conduct it (email survey, etc.);
• Identify who you want to survey and get the necessary contact information;
• Write the research instrument (most often a questionnaire);
• Contact the ARC for assistance with questionnaire review and revision;
• Coordinate your research timeline through the ARC;
• Manage data collection and data entry (if necessary);
• Analyze the data and review the results;
• Share data and research results with the ARC;
• Determine a course of action.

ARC, August 2005

Plan to use any professional resources available to you. Also, determine in advance how you
will pay for research costs, if any. The ARC can help steer you your research efforts to low-cost
solutions.

Decide on the focus for the research

If you haven’t already done so, define the primary goal of your research. Ask “What is the big
decision our group needs to make, and what information do we need in order to make that
decision?” That information is “What you want to know”.

There is a tendency when developing surveys to think “we can add just one more topic…”.
Resist this urge. The advantage to web surveys, the method most often used by the Alumnae
Association, is that they are easy and inexpensive to conduct. Better to do several focused
surveys that one massive, all-encompassing survey.

Think through how you will analyze the results before beginning the research
There’s nothing worse than doing a survey, sitting down to analyze the results, and then realizing
you asked the wrong questions, or didn’t ask some important additional questions. Furthermore,
without adequate planning, research can be misleading by providing misinformation. For
example, a seemingly logical question to ask when following up after an event is about the
quality of individual speakers or programs. However, equally or perhaps more important would
be to ask about the importance of the speaker or program to the overall event. You could have a
series of high-quality speakers or programs, but if none of them are considered important to the
overall event future attendance could suffer. By planning your analysis you may anticipate some
of these issues.

Consider confidentiality
In designing your research you need to decide whether or not the identity of respondents will be
kept confidential, and what measures you will take to ensure confidentiality.

Ensuring respondents that their individual responses will be kept confidential can encourage
them to be more candid and honest with their feedback. Respondents can still be given the
opportunity to have their specific concerns or problems addressed by including a question giving
them the option of providing their name and number if they would like to be contacted.

How to ensure respondent confidentiality

Web surveys conducted via the organization the Alumnae Association uses ensure
confidentiality – you will not even know who responded to the survey.

Ensuring confidentiality with a mail survey will include at minimum that responses be mailed
back to an independent third party. Although the Association would be happy to provide this
service, most alumnae would not consider us to be an independent third party.

Ensuring confidentiality with a phone survey, focus group or in-depth interview will involve
contracting out for data collection.

ARC, August 2005

With any methodology, it is important to consider whether answers to the survey questions
themselves will result in unique responses which can identify individual respondents. For
example, if you conduct a survey of international alumnae and ask for country of residence and
date of graduation, there’s a very good chance you could identify many individual respondents
by their response to these two questions. You may not have any intention of doing so, but many
respondents will realize that their confidentiality has been compromised, and may terminate the
survey, change their responses, or just be angry that you haven’t delivered on your promise of
confidentiality.

Decide who you want to survey - your target population

The target population for your research depends on what you want to know. Carefully match the
target population to the research focus. It could be frustrating for a new graduate to be asked
about career and education histories since graduation, or for a graduate 15 years out to be asked
about the quality of the food in their headquarter dorm.

Determine whether you need Quantitative or Qualitative Research

Quantitative Research is any research method that produces information that can be usefully
analyzed numerically, typically by obtaining data from a scientific sample of the population.
Depending on how they are designed, examples can include telephone surveys, web-based email
surveys, or mail surveys. U.S. Census, labor and health statistics and sales or income trends are
also examples of quantitative information. Quantitative research would be useful for answering
questions such as:
• What proportion of our alumnae of color population would be interested in an
alumnae of color affinity group?
• How does interest in an alumnae of color affinity group vary by geographic region?
• What is the demographic profile of engaged alumnae vs. the profile of
inactive/disenfranchised alumnae?

Qualitative Research is any research method that produces information based upon description
and interpretation rather than numeric analysis. Examples include focus groups, in-depth
interviews, and one-on-one interviews. Qualitative research would be useful for answering
questions such as:
• How alumnae make decisions;
• Whether our alumnae or subgroups of our alumnae have needs that are unmet by
existing products or services;
• What possible barriers exist to use of Association or Club products or services, and
reactions to specific program ideas or service offerings.

Decide how you will do your research – the research methodology

Evaluate the pros and cons of each research method with your information needs in mind.
Consider the cost/benefit of the different methodologies. For example, do the advantages of a
phone survey outweigh the significant cost?
Our recommendation is to consider a web survey first. If the number of email addresses for the
target population is low, or the target population leans heavily toward individuals who do not

ARC, August 2005

have web access, then consider another methodology, or a combination of methodologies (for
example, web survey for those with access, paper survey for those without).
Methodology Pros
Web/online Very inexpensive
survey
Can do “in-house”
(quantitative)
Better response rates
No data entry
Allows for skip patterns, attractive
formatting
Helps avoid the bias that can be a
problem with phone surveys if
volunteers conduct the interviews
Mail survey Often less expensive than telephone
or in-person contact
(quantitative)
Helps avoid the bias that can be a
problem with phone surveys if
volunteers conduct the interviews
Qualitative Can gather nuances, follow up
research - Can be good for teasing-out why
Focus groups, alumnae aren’t attending events,
In-depth aren’t engaged, etc.
interviews
Phone Ability to influence response rate by
survey encouraging reluctant respondents to
participate
(quantitative)
Ability to probe on open-ended
questions to ensure complete,
unambiguous responses
Ability to clarify questions for
respondent as needed
Avoid bias
Data can be biased in a number of ways, including:
ARC, August 2005
Cons
Email may be seen as spam or
invasive
Missed demographics – those
without email or web access
Respondents may provide
ambiguous responses
Not an effective way to reach
disenfranchised alumnae
Still relatively expensive and time-
consuming
Lower response rates
Respondents may skip questions or
provide ambiguous responses
Not an effective way to reach
disenfranchised alumnae
Expensive
Time consuming
Use of volunteers (especially
alumnae or students) increases
likelihood of bias in data. Often
works best for an “outsider”.
Can be expensive and time-
consuming
Use of volunteer callers (especially
alumnae or students) increases
likelihood of bias in data
If being conducted “in-house”, need
to data enter paper responses or
merge computer files for analysis
 • By drafting a questionnaire that does not allow for a balance range of responses or in
some other way leads the respondent,
• By surveying a segment of the population that for one reason or another is not
representative of the larger population that is the target of the research, and
• By “leading” or otherwise influencing the responses of the people interviewed.
If you are considering a telephone survey utilizing students or alumnae volunteers as
interviewers, carefully consider the extent to which this could bias the responses you receive.
Alumnae may respond more favorably (or less) to a fellow alumna or student, precisely the effect
you would desire during a fund raising campaign, but undesirable during a survey. Respondents
may not provide true, accurate responses when speaking to volunteers or students and this could
significantly alter the survey results.
Avoid using volunteers callers who are intimately involved with the research topic. For
example, if your survey is about your local book club, book club volunteers should not do the
calling.
The Alumnae Relations Committee can help you identify ways in which your methodology,
questionnaire or other aspect of your research may bias your data.

Design the research instrument (questionnaire)

• Provide an introduction and reason for the research.
o Have the survey endorsed by the most powerful person affiliated with your group
o Write an introduction, and consider also sending an advance letter or email, that
clearly states the purpose of the research and motivates recipients to participate.
Be clear why alumnae should complete the survey - how it will help them.
o If you are offering an incentive for completing the survey be sure to mention the
incentive during the introduction.
o If responses will be confidential be sure to mention this, but don’t promise more
than you can deliver.
• Based on the survey objectives, identify eligibility and exclusion criteria.
o The questionnaire should be designed with eligibility and exclusion questions as
close to the beginning of the questionnaire as possible. Ineligible individuals
should be skipped to the end of the questionnaire.
o For example, if the target population is alumnae who have attended at least one
club event, and we cannot determine when generating the sample list which
alumnae have attended at least one club event, then we need to ask that question
during the survey. After the introduction, ask “How many club events have you
attended in the past x years?”. Respondents who choose “None” would be
skipped to the end of the questionnaire, where you will end with a phrase like
“Thank you for participating in this research.”
• Limit the number of questions.
o Be sure each question contributes to the overall goal of the research, and helps to
inform the specific decision the group needs to make.
• Use precise, unambiguous questions.

ARC, August 2005

 o Develop questions for a “user-friendly” questionnaire, especially if the
questionnaire will be self-administered (email or mail survey).
o Be sure the questions are easily understood and cannot be interpreted in multiple
ways. For example, “Where were you born?” could generate responses including
country of birth (“Sweden”), town and state (“Melrose, MA”) or hospital
(“Cooley Dickinson”).
o Use appropriate language. Avoid projecting personal bias into the wording of
question and answer categories. For example, instead of “Has the Alumnae
Association adequately served your needs as an international alumna?” consider
“To what extent has the Alumnae Association served your needs as an
International alumna?” followed by a scale.
o Avoid vague or relative terms, like “usually” or “large”. When using relative
terms, define them.
o Avoid double-barred questions. For example, “How would you rate the time and
location for the recent event?”
o Asking precise, unambiguous questions may mean asking multiple questions
rather than one.
• Design an attractive questionnaire.
o Ensure the questionnaire looks appealing, that it is easy to read and respond to,
and that questions won’t be missed (for example, some respondents will fail to
compete side two of a double-sided mail questionnaire).
o Provide clear instructions throughout the questionnaire as needed. This may
include instructions at the beginning of the questionnaire (general instructions),
between different sections in a questionnaire (transitional), and before subsets of
questions (such as “use the following scale to complete questions 10-12”).
• Use the appropriate question format.
o Your questionnaire will most likely include a mix of open-ended and close-ended
questions. Try to limit use of open-ended questions, as they are harder to analyze.
o Open-ended questions:
ƒ Allow respondents to express their thoughts fully, in their own words.
ƒ Are especially useful when it is difficult to anticipate the likely range of
responses on a given issue, such as when the intricacies of an issue are still
unknown.
ƒ Require significant additional time and expense to analyze quantitatively.
May be difficult to compare and interpret responses. Responses may be
ambiguous or incomplete.
ƒ Example: “What could the Alumnae Association offer that would interest
you in coming to campus for a program?”
o Close-ended questions:
ƒ Respondents choose from a list of answers.
ƒ Easy to analyze.

ARC, August 2005

 ƒ Respondents may resent being forced into pre-selected choices. A
combination of close- and open-ended questions can offset this.
ƒ Response categories must be exhaustive, while simultaneously not being
too long. When appropriate, remember to include “I don’t know / Not
applicable” or “Other” as a response option. Response categories should
also be mutually exclusive.
ƒ Decide whether respondents will be restricted to a single response or
allowed to provide multiple responses.
ƒ Example: “How likely would you be to attend the following types of
programs through your local club or regional program, if it was on a topic
of interest to you?
A weekend program
1 – Very likely
2 – Somewhat likely
3 – Not likely”
• Group questions based on topic, and order groupings logically. Place questions which
may be objectionable (such as a question about household income) toward the end of the
questionnaire.
• Include skip-patterns as necessary. For example, begin a section of questions about an
alumna’s most recent class reunion by asking whether the respondent attended her most
recent class reunion. If she did not, instruct her to skip to the next appropriate question
(or program the questionnaire to skip her to the next appropriate question).
• Make sure to ask questions to gather any demographic information that you would like to
use to analyze your data. Since your results from a web survey will come back to you
stripped of information which would allow you to identify the specific respondent, you
would need to ask, for example, about year of graduation or country of residence if you
would like to analyze using that information.
• End the questionnaire by thanking respondents for their participation.

Pilot-test the instrument

Questionnaires based heavily on one of the Alumnae Relations Committee’s survey templates
may not need to be pilot tested. The ARC can recommend to you whether you need to pilot-test
your questionnaire.
If you decide or the ARC recommends that you pilot-test your questionnaire:
o Test the draft questionnaire on a small number of respondents, preferably with
respondents who are generally similar to the population to be surveyed. For
example, do not pilot test a questionnaire about reunions with volunteers from the
Classes and reunion committee. Their understanding and involvement with
reunions will give them a substantially different perspective on the questionnaire.
o Solicit feedback on questionnaire length, and general flow.
o Solicit feedback on question structure. Were any questions difficult to understand
or otherwise problematic?

ARC, August 2005

 o Based on pilot-test feedback, make final questionnaire revisions.

Conduct the research

Web surveys
Following are steps for conducting a typical web survey:
• Contact the ARC for the website and passwords for our web survey service
• Review the training guide to help you understand how web surveys work, and how to
do yours
• Set up your survey. Enter your questionnaire.
• Proof the questionnaire, both for grammar and programming errors
• Complete several surveys before going “live”, to test the functionality
• Upload or enter your sample list
• Send the email invitations with link to the survey. In the body of the email specify a
timeframe for completing the survey
• Follow up a week later with a reminder sent to nonrespondents (the web site has the
ability to send reminders to nonrespondents only)
• Monitor number of respondents. If response to the survey is particularly low,
consider another reminder, perhaps with better incentives.
• Download data, and proceed to analysis.
• If you offered an incentive, don’t forgot to follow through and award prizes.

Mail surveys
Following are steps for conducting a typical web survey:
• Make the appropriate number of copies of your cover letter and questionnaire.
• Print mailing labels from your sample list. If you intend to send a reminder postcard,
you may want to print two batches of labels at this point.
• Insert cover letters and questionnaires into envelopes. Include a postage-prepaid
return envelop, preferably addressed to an independent third part. Label envelopes,
apply postage and mail;
• Follow up approximately 2 weeks later with a reminder postcard.
• Monitor number of respondents. If response to the survey is particularly low,
consider another reminder, perhaps with better incentives.
• Data enter results into a format that will enable you to produce some statistics on the
data. The ARC can provide recommendations how to go about this.
• If you offered an incentive, don’t forgot to follow through and award prizes.

Other research methodologies

If you plan to conduct a focus group, in-depth interviews, or a phone survey please contact the
Alumnae Relations Committee directly so that we can provide you with individualized assistance
and advice.

Analyze the results

If you conducted a web survey, download data results. If you conducted a mail survey you will
need to data enter your results into a system that can provide some statistical analysis.

ARC, August 2005

The Alumnae Relations Committee does not provide assistance with data analysis. However, we
offer the following advice:
• Consider whether you need and can afford professional help, either in the form of a
consultant/firm, or alumnae, professors or graduate students with experience
analyzing research data;
• Think through carefully what you can and cannot do with your data. For example, if
you ask a question and allow alumnae to select more than one answer, you cannot add
together the percentages who selected different responses. For example, in a survey
of alumnae about the alumnae travel program the researchers asked respondents to
indicate the type(s) of travel program that interested them. The result was:
o Land, 58.9%
o River cruise, 47.8%
o Cruise, 32.8%
o Nature, 30%
o Adventure, 16.9%
The data analysis, however, concluded that “80.6% of the respondents preferred some
type of cruise”. In fact, the total may have been no higher than 47.8%, as the same
people who responded that they were interested in a cruise may also have responded
that they were interested in a river cruise.

Report on the results

Be sure to send the Alumnae Relations Committee a copy (preferably in electronic format) of the
questionnaire used, and the research results (including raw data, if possible).

Share the research results, and (if appropriate) your plans

We suggest that you share the research results, and your plans based on them, with your
constituents. You can say, in essence, “We asked, you responded, and we will (improve our
services/expand on such and such)...” Follow-through like this will not only increase your
effectiveness, it will improve the likelihood of participation in future research, as non-
respondents will see the cost of their non-response (no voice in changes.)

Act on the results

Conducting research creates an expectation in your constituents that something will change.
Don’t disappoint them. If you follow our recommendations, you will:
• Contact your constituents in advance of your research to let them know you are doing
research, and why;
• Conduct your research program, analyze the results, and plan your course of action;
• Share the results and your plans with your constituents,
• Act on the results.

Acting on the results is the most important step. Remember, research is not an end in itself, it is
a tool to lead you to effective action.

ARC, August 2005