15

Chapter Fifteen

Evaluating an Integrated
Marketing Program
Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-1

15

Pretesting for Effectiveness

Rocket analogy
Decision Analyst
http://www.decisionanalyst.com
CopyScreen
CopyCheck

What are the pros and cons of

testing ads and marketing
communication pieces at various
stages of development?

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall

15-2

Evaluating an Integrated
Marketing Program

15

Chapter Overview

Matching methods with objectives

Message evaluations
Evaluation criteria
Behavioral evaluations
Evaluating public relations
Evaluating the IMC program

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall

15-3

Evaluation Categories
Message evaluation techniques
Physical design
Cognitive elements
Affective elements

Respondent behavior evaluations

Conative elements
Measurable with numbers
Customer actions
Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall

15-4

Evaluation and IMC Objectives

Match objectives
Pre- and posttest analysis
Levels of analyses

Short-term
Long-term
Product-specific
Corporate level
Affective, cognitive, or conative

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall

15-5

FIGURE

15.1

Message Evaluation Techniques

and When to Use Them
Message Evaluation Method

Concept testing
Copytesting
Recall tests
Recognition tests
Attitude and opinion tests
Emotional reaction tests
Physiological tests
Persuasion analysis

When the Test Is Normally Used

Prior to ad development
Final stages, or finished ad
Primarily after ad has been launched
After ad has been launched
Anytime during or after ad development
Anytime during or after ad development
Anytime during or after ad development
After ad has been launched

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall

15-6

Concept Testing

Prior to ad development
Average cost 30-second TV ad $350,000
Focus groups
Concept testing instruments
Comprehension tests
Reaction tests

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall

15-7

AFLAC
Concept Testing
Before launching the AFLAC duck advertising campaign, the agency
conducted concept tests to determine which idea was the best.

Click here to play clip

from AFLAC

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall

15-8

Copytesting
Used when marketing piece is finished
or in final stages
Methods used

Portfolio test
Theater test
Focus groups
Mall intercept

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall

15-9

Copytesting
Copytesting can be
used to determine if
viewers comprehend
this ad and what their
reaction to it is.

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-10

Copytesting
Criticisms of copytesting
Some agencies not using
Stifles creativity
Focus groups not good judge

Support of copytesting
Issue of accountability
Majority support because clients want
support for ad decision
Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-11

Recall
Tests

Day-after recall (DAR)

Unaided recall
Aided recall
Incorrect answers
Use primarily after ads launched
Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-12

FIGURE

15.3

Items Tested for Recall

Product name or brand

Firm name
Company location
Theme music
Spokesperson
Tagline
Incentive being offered
Product attributes
Primary selling point of communication piece
Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-13

Recall Tests: Do Viewers Remember?

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-14

Sample DAR Test

30-Second TV Advertisement for Pet Food
25.0%

20.0%

15.0%

10.0%

5.0%

0.0%
Brand name

Theme music

Spokesperson

Test Ad

Competitor A

Tagline

Incentive

Product
Attribute

Competitor B

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-15

Sample DAR Test

30-Second TV Advertisement for Pet Food
24.6%
25.0%
21.4%
18.3%

20.0%

16.3%
14.6%

15.0%

Overall Recall 12.9%

9.4%

8.5%

10.0%

5.0%

0.0%
Males

Females

Pet
Owners

Dog
Owners

Ages 18- Ages 36- Ages 51+

35
50

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-16

Recall Decay
Magazine Ad vs. Television Ad
100% 100%
86%

100%

75%
65%

80%

43%

60%
40%
20%
0%
DAR

Two days later

Magazine

Eight days later

Television

Source: Magazines Canadas Research Archive

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-17

Recall Tests
Factors That Influence Scores
Attitude towards advertising
Prominence of brand name
Brand used by respondent
Institutional ads

Respondents age

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-18

Recognition Tests
Respondents shown marketing piece
Often used with recall tests
Good for measuring
Reaction
Comprehension
Likeability

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-19

Recognition Tests
Expression of persons interest
Ad liked + 75%
Ad interesting + 50%
Brand used + 50%

Affected by ad size, color, length

Scores do not decline over time

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-20

Recognition Tests

Can be used to measure

Reaction
Comprehension
Likeability
Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-21

Attitude and Opinion Tests

Used in conjunction with other tests
Recall tests
Recognition tests

Closed-ended questions
Open-ended questions
Roper Start ADD + IMPACT

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-22

Emotional Reaction Tests

Affective advertisements
Used for material designed to solicit emotions
Difficult to measure emotions with questions
Warmth monitor
Emotional reaction tests are self-reported
instruments

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-23

Sample Graph from a Warmth Meter

30-Second TV Advertisement

Sample Ad Rating
Warmth Meter

Ad section that elicited negative emotions

Target Audience

Total Audience

Start

10 seconds

20 seconds

30 seconds

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-24

Physiological Arousal Tests

Measure fluctuations in a persons body

Psychogalvanometer sweat
Pupillometric test pupils of eyes
Psychophysiology brain waves and
currents
Cannot be faked easily

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-25

Persuasion Analysis
Appraise persuasiveness of marketing item
Requires pre- and posttests
ASI Market Research studies

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-26

FIGURE

15.6

Copytesting Principles of PACT

Testing procedure should be relevant to objectives.
Researchers should agree on how the results will be used in
advance.
Multiple measures should be used.
The test should be based on some model or theory of human
response to communication.
Testing procedure should allow for more than one exposure.
In selecting alternate ads to include in the test, they should be at
the same stage in the process as the test ad.
The test should provide controls to avoid biases.
Sample used for the test should be representative of the target
sample.
Testing procedure should demonstrate reliability and validity.
Source: Based on PACT document published in the Journal of Marketing, (1982) ,Vol. 11, No. 4, pp. 4-29.
Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-27

FIGURE

15.7

Behavioral Measures

Sales
Response rates
Redemption rates
Test markets
Purchase simulation tests
Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-28

Sales and Response Rates

Measuring sales with UPC codes
Scanner data
Retailers
Manufacturers

Sales changes can be caused by other factors

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-29

Difficulties in Evaluating
Advertising

Influence of other factors on behavior

Delayed impact of advertising
Consumers change their mind in the store
Whether brand is in evoked set
Goal of ad may be to build brand equity,
not increase sales

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-30

FIGURE

15.9

Responses to Marketing Messages That Can Be Tracked

Changes in sales
Telephone inquiries
Response cards
Internet inquiries
Direct marketing responses
Redemption rate of sales promotion offers
Coupons, premiums, contests, sweepstakes

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-31

FIGURE

1 5 . 10

Methods of Measuring Interactive Marketing

60.0%

51.0%

50.0%
41.1%
40.0%
30.0%

36.5%

44.5%

40.7%

34.2%
26.6%

24.7%

20.0%

16.3%

12.2%

10.0%

4.6%

I
RO

n
Im
pr
es
sio
Re
ns
de
m
pt
io
n
ra
te
Re
s
sp
on
se
ra
te
s

ar
gi
M

ge
ne
ra
t io
of
n
en
ga
ge
m
en
t
Le
ng
th

Sa
le
s

Le
ad

Br
an
d

aw
ar
en
es
s
Cl
ic
kt
hr
Cu
ou
st
gh
om
s
er
kn
ow
le
dg
e

0.0%

Source: Adapted from Larry Jaffee, Follow the Money, Promo, Vol. 20, No. 11 (November 2007 Sourcebook), pp. 5-10.

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-32

Online Metrics
Adknowledge
MarketMatch Planner
Campaign Manager
Administrative Campaign Manager

Audience demographics

MediaMetrix basic demographics

NetRatings GRP and other rating instruments
SRI Consulting Psychographic information
NetGuide Web site ratings and descriptives
BPA Interactive Web traffic audit data

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-33

Test Markets
Used to assess:
Advertisements
Consumer and trade promotions
Pricing tactics
New products
Cost-effective method of evaluation prior to launch
Resembles actual situation
Design test market to model full marketing plan
Length of test market is a concern
Competitive actions must be considered
Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-34

Purchase Simulation Tests

Bias in purchase intention questions

Simulated purchase tests
Research Systems Corporation
Does not rely on opinions and attitudes

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-35

F I G U R E 1 5 . 11
Evaluating Public Relations

Number of clippings
Number of impressions
Advertising equivalence technique
Comparison to public relations objectives

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-36

Evaluating the IMC Program

Greater demand for accountability

ROI of advertising and marketing
Difficult to measure ROI 70%
Difficult to define ROI 70%

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-37

TAB LE

15.2

Definitions of ROI for Marketing

Definition of ROI

Percent Using

Incremental sales from marketing

66%

Changes in brand awareness

57%

Total sales revenue from marketing

55%

Changes in purchase intentions

55%

Changes in market share

49%

Ratio of advertising costs to sales

34%

Reach/frequency achieved

30%

Gross rating points delivered

25%

Post-buy analysis comparing the media plan to its delivery

21%

Source: Paul J. Cough, Study: Marketers Struggle to Measure Effectiveness, Shoot, Vol. 45, No. 29
(August 20, 2004), pp. 7-8.
Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-38

FIGURE

1 5 . 12

Measures of Overall Health of a Company

Market share
Level of innovation
Productivity
Physical and financial resources
Profitability
Manager performance and development
Employee performance and attitudes
Social responsibility

Source: Pete Drucker, Management: Tools, Responsibilities, Practices, New York: Harper and Row, 1974.
Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-39

International Implications
Assessment of IMC Programs
Domestic results
Results in other countries
Overall organization

Individual ads and promotional programs

Local culture
Across national boundaries
Multinational regional offices

Copyright 2010 by Pearson Education, Inc. publishing as Prentice Hall 15-40