External Validity Is More Than
Skin Deep
Some Answers to Criticisms of ,
Laboratory Experiments
LEONARD BERKOWITZ University of Wisconsin
EDWARD DONNERSTEIN University of Wisconsin
ABSTRACT: We discuss some criticisms of laboratory
experiments in psychology, giving special attention to
the claim that these experiments lack external validity
(since it is -widely assumed that ecological validity
makes for external validity). We suggest that represen-
tative designs are inadequate for testing causal hy-
potheses, that ecological validity may facilitate the for-
mulation of population estimates but is not necessary
for causal hypothesis testing, and that experiments are
not conducted to establish population estimates. More-
over, the meaning that subjects assign to the laboratory
setting and their actions, rather than the laboratory
setting's mundane realism, affects the generalizability
of the laboratory results. We emphasize, as other writers
have, that whether laboratory results are generalizable
to other situations is an empirical question. Research
on aggression, especially in regard to the "weapons ef-
. feet," is employed to illustrate the possible extension
of laboratory findings to other, more natural, situations.
Most criticisms of laboratory experiments in psy-
chology fault them for their "artificiality" or, in
more sophisticated terms, for their lack of external
validity. As everyone knows, the great majority of
psychology's experiments employ a very limited
sample of participants (typically, college students)
placed in a fairly unique setting (a university lab-
oratory) and usually working on tasks bearing little
resemblance to their everyday activities. Given the
unrepresentativeness of these subjects and situa-
tions, the critics ask, how can the findings be gen-
eralized to the "real world" of ordinary people
engaged in their daily lives? In one form or an-
other, this question has been raised by professionals
and outsiders, by trained social scientists, and by
persons having only a slight acquaintance with the
field. There can be no easy answer. Anyone fa-
Vol. 37, No. 3, 245-257
Copyright 1982 by the American" Psychological Association, Inc.
0003-066X/82/3703-0245$00.75
miliar with the longstanding debate that has
swirled about the topic of experimental validity
is well aware of the complex issues bound up in
this matter. Nonetheless, recent discussions of lab-
oratory experiments vis-a-vis naturalistic investi-
gations in the areas of industrial-organizational,
developmental, and social psychology suggest that
it is again important to offer a brief defense of
experimental research throughout all of psychol-
ogy. This argument will not go into all of the con-
siderations that might be mentioned. We will focus
on what we regard as some of the chief objections
to laboratory experiments, first examining some of
the general criticisms that have been leveled
against these studies and then applying our re-
buttals to the special area of research on aggression.
We will argue, first, that the logic of an experiment
does not require that the settings and subjects rep-
resent real-world conditions and, second, that the
external validity of a study (in the sense of the
generalizability of its results) is not necessarily gov-
erned by physical representativeness.
The Supposed Limited Validity of
Experimental Social Psychology
Harre and Secord (1972) have provided an excel-
lent illustration of the attacks on laboratory ex-
perimentation in their discussion of what they term
the "information-processing model" in experimen-
tal social psychology. In this paradigm, they con-
Requests for reprints should be sent to Leonard Berkowitz,
Department of Psychology, University of Wisconsin, Madison,
Wisconsin 53706.
AMERICAN PSYCHOLOGIST • MARCH 1982 • 245
tended, researchers neglect the social contexts
within which people are embedded in their usual
lives and ask their subjects to react to some stimulus
object on the basis of extremely limited informa-
tion. Byrne's (1969) well-known studies of inter-
personal attraction are held up as particularly hor-
rible examples of this isolation of subjects from
their social surroundings. The experimenter in
these investigations told the subjects about the at-
titudes supposedly held by a (fictitious) stranger
and then required the participants to rate their
liking for that individual. According to Harre and
Secord
What is glaringly wrong here is the contrast between the
laboratory situation, in which the action of the partici-
pants is confined to a series of dispassionate judgments
based upon written descriptions, and the face-to-face
situation where a person reacts to another person with
warmth and friendliness and a feeling of spontaneous
liking . . . The dissimilarity between the life situation
and the laboratory situation is so marked that the labo-
ratory experiment really tells us nothing about the gen-
esis of liking and friendship among real people, (pp. 51-
52, italics in original)
Here, in extreme form, is the oft-repeated claim
that laboratory experiments lack external validity:
Byrne's procedure is supposedly so unrepresenta-
tive of the development of liking in naturalistic
settings that we cannot possibly generalize from
this research to a "face-to-face situation where a
person reacts to another person with warmth and
friendliness and a feeling of spontaneous liking."
Other writers have voiced similar doubts. In his
survey of the history of social psychology, Gordon
Allport (1968) maintained that "many contem-
porary studies seem to shed light on nothing more
than a narrow phenomenon studied under specific
conditions." Even if the experiment is successfully
repeated, he complained, "there is no proof that
the discovery has wider validity" (p. 68). Several
books that object to the strong experimental em-
phasis in contemporary social psychology (e.g.,
Gilmo.ur & Duck, 1980; Strickland, Aboud, & Ger-
gen, 1976) have now been published. What is es-
pecially disturbing in these attacks is the wide-
spread equation of experimental value with
ecological validity; the research supposedly is of
little benefit unless its conditions and subjects faith-
fully mirror "real people in the real world."1
The Bases of the Misgivings About
Experimental Validity
The criticisms of laboratory experiments stem
from the critics' assumptions about human behav-
246 • MARCH 1982 • AMERICAN PSYCHOLOGIST
ior as well as from their conceptions of the purpose
of experiments. At the risk of oversimplifying, it
will be helpful to take a brief look at some of their
thinking.
Opposing the classic systematic designs (which
control and manipulate a limited set of variables
through on,ly a restricted range of values of these
variables), Egon Brunswik (1955) advocated rep-
resentative designs employing a representative
sampling of situations and of subjects. He insisted
on this "ecological validity" (the notion he intro-
duced to psychology) because of his probabilistic
view of behavior and especially of perception. For
him the classic procedures exaggerate the impact,
of proximal cues and unduly minimize the signif-
icance of the surrounding distal cues. The role of
these latter cues could only be assessed adequately,
Brunswik argued, if they were allowed to vary
naturally. It seems to us that Campbell and Stanley
(1963) also adopted a probabilistic position in their
discussion of external "validity. This validity for
them was a matter of inductive inference: an in-
ference could be drawn from a research sample
to a larger population only to the degree that the
given sample was representative of that popula-
tion. Since behavior was affected by a wide variety
of factors, proximal and distal, these possible in-
fluences ought to be present in the sample to the
extent that they occurred in the broader population
to which the researcher .wanted to generalize.
Other critics have based their objections to lab-
oratory experimentation on their conception of
human nature. They remind us that people seek
meaning, govern their actions according to social
rules, and strive to fulfill their inherent capacities.
These proclivities are supposedly neglected or even
denied by experimental research. Along with
Harre and Secord (1972) and others, Gadlin and
Ingle (1975) held that the human subject of psy-
chology experiments is manipulated by the exper-
imenter much as if he or she were an inert, "ma-
nipulable object." The result, according to Schultz
(1969) and others, is that psychology has developed
an inadequate, overly mechanistic image of the
1
Much of our argument pertains to other areas of psychology
as well as to social psychology. Developmental psychology in
particular has seen a rapidly mounting insistence on the eco-
logical validity of research under the impetus of such influential
writers as Bronfenbrenner (Note 1). We wonder if this insistence
might have become too rigid and perhaps even somewhat dog-
matic—Theeditor of one of the leading developmental journals,
in discussing criteria for the selection of manuscripts, once
stated, "In the case of laboratory experimental studies . . . the
ecological validity of such studies will be a major consideration."
human as a stimulus-response machine. The "lim-
itations of the experimental method," we are told,
yield "misleading findings and one-sided concep-
tualizations" (Wachtel, 1980, p. 407).
Some Replies
DOES METHODOLOGY DETERMINE CONTENT?
A number of experimentally oriented psychologists
have answered these attacks. To cite only a few,
Kruglanski (1976) insisted that the content of psy-
chological theorizing is not inevitably affected by
experimental methodology. We can see this in the
type of theorizing that is now dominant in exper-
imental social psychology. Contrary to the claim
that experiments portray people as relatively un-
thinking, almost robotlike reactors to the events
impinging on them, the most influential analyses
of human behavior in experimental social psy-
chology today emphasize the role of the individ-
ual's thought processes and assume that he or she
is an active seeker after meaning and often tries
to control what is happening. One can even go
further and argue that experiments give us a truer
image of human complexity than do uncontrolled,
naturalistic investigations. Where the latter can
only hint at the richness of human nature, the rel-
ative precision of laboratory studies shows us much
more clearly how people can be deliberate and
thoughtful on some occasions and impulsive, non-
thoughtful, and even irrational at other times. Ex-
periments do not always picture humans as invol-
untary reactors to external stimuli by any means,
but people do respond in such an automatic fashion
under some conditions, and this can be revealed
in well-controlled experiments. More will be said
about this later.
THE MAIN PURPOSE OF AN EXPERIMENT
Other psychologists have questioned the probabi-
listic approach to the study of human behavior
taken by Brunswik and others. Their argument
must be given attention because it essentially holds
that the main purpose of experimental research is
not necessarily best served by an overriding em-
phasis on representative designs. Postman (1955)
made this point in a symposium on Brunswik's
perspective. He noted that representative designs
"necessarily entail the abandonment of classical
notions of control; we must take the interaction
among variables as we find it" (p. 222). This means
we can only make statements about correlations
between distal or proximal stimuli on one hand
and the organism's responses on the other on the
basis of such designs, but cannot draw any definite
conclusions from this research about the processes
that mediate between the external stimuli and the
subsequent responses. The lesson should be clear.
If the investigator wishes to estimate the magni-
tude of these correlations in some target popula-
tion, he or she should turn to representative designs
high in ecological validity. But if the researcher
is primarily interested in "testing the implication
of a theoretical proposition," systematic experi-
ments are desirable (Postman, 1955, pp. 222-223).
To phrase this somewhat differently, laboratory
experiments are mainly oriented toward testing
some causal hypothesis (and according to Postman,
ought to deal chiefly with mediational processes)
and are not carried out to determine the proba-
bility that a certain event will occur in a particular
population.2
Kruglanski (1975) offered much the same view
as to why experiments are conducted. He main-
tained that many objections to laboratory experi-
ments actually confuse two very different types
of psychological research: (a) particularistic in-
quiries in which attention is focused mainly on the
accuracy of statements about specific instances and
the intended generalizations are restricted in scope,
and (b) universalistic (or theoretically oriented)
investigations in which the researcher tentatively
claims a universal scope of generality for the find-
ings "across all sorts and varieties of theoretically
irrelevant conditions." In a universally oriented
laboratory experiment
the hypothesis tested concerns the causal relations among
general constructs . . . of which the specific manipula-
tions and measures constitute arbitrary operational def-
initions, and the subject sample is an arbitrary group
from the general universe (e.g., the class of all humans)
to which the hypothesis is assumed to apply. (Kruglanski,
1975, p. 105)
2
In an interesting article published after this manuscript was
written, Henshel (1980) offered another argument in support
of "artificial" laboratory experiments. Along with several phi-
losophers of science, Henshel began by differentiating between
the objective of verification (which is aimed at the testing of
some hypothesis) and the objective of discovery (which seeks
to uncover "new regularities under conditions that do not exist
anywhere in natural settings" but that are capable of existing).
His thesis is that the experimental control occurring in labo-
ratory studies facilitates such discoveries, and thus artificiality
is no problem for this objective. Henshel points to the research
on biof eedback and the possible acquisition of language by non-
human primates as only two examples of the fruitful pursuit
of discovery by laboratory experiments. These phenomena
could not have been uncovered if investigators had confined
themselves to naturally occurring variations in conditions.
AMERICAN PSYCHOLOGIST • MARCH 1982 • 247
 The experimentalist pursuing this type of re-
search need not be especially worried about the
representativeness of the sample and laboratory
conditions. He or she can have greater confidence
in the validity of the causal relations that are un-
covered, Kruglanski reminded us, when these re-
lationships are "replicated with alternative oper-
ational definitions of the same constructs and with
other samples" (p. 105), but the chief question is
whether the experimental situation "captures the
intended essence of the theoretical variables"
(p. 106).
Following the distinction drawn by Carlsmith,
Ellsworth, and Aronson (1976), we can say that
Kruglanski called for a greater emphasis on ex-
perimental realism than on mundane realism.
Mundane realism exists when the experimental
setting and procedure resemble things that occur
in the "real world," that is, when there is a su-
perficial appearance of reality. Many experimental
social psychologists would rather engage their sub-
jects fully in the laboratory events than strive for
this surface resemblance to naturalistic settings.
They assume, with some justification, that exper-
imental realism is more apt to "capture the in-
tended essence of the theoretical variables" than
is a mere mundane realism, and they are more
interested in these variables than on how faithfully
the laboratory conditions seem to duplicate the
physical and social characteristics of particular
real-world settings. Littman (1961) used an inter-
esting phrase to make the same observation when
he said that psychology was a "socially indifferent
science." Psychology should be concerned with
psychological processes, Littmari argued, not with
specific social content. Its focus should be on the
development of universalistic theoretical state-
ments regarding these mediating processes, prop-
ositions that apply to humans everywhere, and
should give little attention to particularistic reports
of the behavior of certain groups in limited situ-
ations. Those who would adhere to this general
prescription for psychology must necessarily give
greater weight to the adequacy of systematic ex-
perimental designs than to the representativeness
of the subjects and settings in the laboratory re-
search. . same processes that influenced the extent to which
Although such a statement might appear com-
monplace when it is phrased abstractly, our own
research interests can help give this point greater
impact. Consider what laboratory experiments on
the effects of filmed violence can or cannot ac-
complish. Our discussion clearly implies that most
of these investigations do not tell us the likelihood
248 • MARCH 1982 • AMERICAN PSYCHOLOGIST
that ordinary viewers will behave aggressively af-
ter they watch a violent TV program in their own
homes—The settings, subjects, and stimulus ma-
terials employed in these experiments are hardly
representative of naturalistic TV viewing. What
the studies usually do indicate, however, is that
observed aggression can influence people in the
audience to display stronger aggression than they
otherwise might have exhibited. Employing care-
ful, systematic designs, the experiments have dem-
onstrated such a causal effect far more clearly than
great numbers of field surveys employing repre-
sentative designs could. Then too, this argument
also maintains that laboratory experiments in the
area of aggression should not be faulted for their
supposed failure to copy real-world conditions
faithfully. Some of the criticisms of this line of
research maintain that the experimental procedure
is not sufficiently realistic. The critics insist on
mundane realism rather than on experimental re-
alism, and they place too much emphasis on rep-
resentativeness and ecological validity.
EMPIRICAL EVIDENCE
REGARDING THE GENERALIZABILITY
OF UNREPRESENTATIVE RESEARCH
Even though laboratory experiments need not be
preoccupied with attempts to mirror naturalistic
conditions, many of their findings can be gener-
alized to other settings. As the critics should know,
many laboratory results have been duplicated by
investigations carried out in a more representative
manner. Take the Byrne research attacked by
Harre and Secord as being unduly "artificial."
These experiments have shown that attitude sim-
ilarity affects the degree of liking that people have
for each other. Other studies have also found that
attitude similarity predicts the development of lik-
ing among university men residing in a student
rooming house (Newcomb, 1961) and people kept
togethervfor 10 days in a small room (Griffitt &
Veitch, 1974), as well as the interpersonal pref-
erences expressed by white and black job appli-
cants who did not know they were participating
in an experiment (Rokeach, 1968, pp. 68-75). The
Byrne's subjects said they liked a fictitious stranger
evidently also operate, to a considerable degree at
least, in a wide variety of more realistic situations
involving face-to-face interaction.
Similar observations can be made from other
areas of psychology. Dipboye and Flanagan (1979)
examined almost 200 laboratory experiments and
300 field studies in industrial-organizational psy-
chology and concluded that there probably is no
difference in the degree to which we can gener-
alize from either of these two kinds of investiga-
tions. Research carried out in a supposedly artifi-
cial setting need not have less external validity than
research conducted under more lifelike circum-
stances. As they commented later, "Too often the
assumption is made that because a study was con-
ducted in a field setting, it is inherently more ex-
ternally valid than a laboratory study" (Dipboye
& Flanagan, 1980, p. 388).
Bass and Firestone (1980) have offered an ex-
cellent addendum to this comment. It is important,
they noted, to distinguish between the represen-
tativeness and generalizability of psychological re-
search. Many writers "imply that research which
is not representative in terms of subject populations
or setting characteristics cannot be generalized"
(p. 463), However, Bass and Firestone pointed out,
the psychological attributes of the research may
affect the generalizability of the results far more
than the study's objective features such as the de-
mographic characteristics of the 'sample or the
physical nature of the situation the subjects are in.
These attributes can include the participants' def-
inition of their task and their interpretations and
judgments.
THE MEANING OF A SITUATION AS A DETERMINANT
OF THE GENERALIZABILITY OF THE BEHAVIOR
We have now come to our central thesis: The
meaning the subjects assign to the situation they
are in and the behavior they are carrying out plays
a greater part in determining the generalizability
pf an experiment's outcome than does the sample's
demographic representativeness or the setting's
surface realism.
Social scientists have long recognized that con-
sistent interpretations can lead to behavioral con-
sistency. If an individual interprets two objectively
different situations in much the same manner, he
or she will tend to act in the same way on both
occasions. As just one example, Dweck, Goetz, and
Strauss (1980) recently argued that when individ-
uals make the same causal attributions in seem-
ingly disparate settings, they will generalize their
experiences from one situation to the other. We
are simply extending this idea and are noting that
two very different persons may act in the same
way because they have essentially the same un-
derstanding of what is going on and what they are
doing. Moreover, even people who are relatively
unusual in such objective characteristics as age, sex,
education, and class background may arrive at the
same interpretation of the events around them as
those persons with more ordinary qualities.
This is not to say, of course, that laboratory ex-
periments are necessarily as externally valid as
field studies conducted in naturalistic settings.
Rather, with Dipboye and Flanagan (1979), we
argue only that the generalizability of laboratory
findings in comparison with the results of natu-
ralistic investigations is an empirical matter. In
some areas of psychology, such as developmental,
industrial-organizational, and social psychology,
there is an ever-mounting insistence on field stud-
ies as if these investigations are inherently more
valid than laboratory experiments. No such flat
statement is warranted: "There are limits on the
generalizability of all findings, which can be re-
vealed only through systematic testing with dif-
ferent subjects, settings, and responses" (Flanagan
& Dipboye, 1980, p. 465).3
We might perhaps forgive statistically oriented
researchers for their automatic equation of exter-
nal validity with demographic representativeness,
but more interesting is the implicit view of ex-
perimental subjects held by many of the other crit-
ics of laboratory experimentation. Although they
profess a true appreciation of the thinking human
being, in this important respect they actually seem
to give greater weight to physical characteristics
than to the subjects' interpretations. They may be
too similar to Watsonian behaviorists in this re-
gard.4
3
Carlsmith et al. (1976) put this very well when they observed
that we cannot be sure that the same variables were operating
in the laboratory setting and a natural situation even when the
former is a close analogue of the latter. "One cannot guarantee
generalizability simply by providing an experiment that has a
high degree of mundane realism. This does not increase our
confidence in our ability to generalize from the results, for in
the final analysis the question is an empirical one" (p. 86). These
two sentences should be sent to every critic who has faulted an
experiment solely because it seemed to lack ecological validity.
4
One of Bronfenbrenner's (Note 1) criticisms of laboratory
experimentation is based on the experiment's supposed ambi-
guity. Many laboratory studies, he suggested, are so unclear to
the participants that they do not show consistent behavior from
one time to another. We contend that such a high-level of
ambiguity is not a necessary property of all experiments. Indeed,
many experimentalists, at least in social psychology, strive to
have their subjects formulate a very clear and definite under-
standing of the laboratory events. The experimental setting may
not bear a physical resemblance to a particular natural situation
(i.e., it may lack ecological validity), but it can still have a
definite meaning for the participants. In any caSe, laboratory
behavior need not always have a lower test-retest reliability
than behavior in more naturalistic situations.
AMERICAN PSYCHOLOGIST • MARCH 1982 • 249
Experimental Artifacts
Some criticisms of laboratory research recognize
the importance of the participants' perceptions but
contend that subjects frequently do not look at the
laboratory situation the way the researcher intends.
Indeed, it is sometimes suggested that the subjects
are apt to view the experiment in such a different
manner that their reactions may not be governed
by the variables of interest to the researcher but
could actually be produced by other factors. These
factors, which (in the Campbell and Stanley^ 1963,
terminology) are said to lessen the experiment's
internal validity, include demand characteristics
and the participant's evaluation apprehension. We
will briefly discuss these matters because critics
often refer to these presumed artifacts as reasons
for dismissing laboratory findings.
DEMANP CHARACTERISTICS
Starting with a very influential paper first pub-
lished almost a generation ago, Martin Orne (1962)
has advanced a motivational conception that has
been more widely accepted and exposed to less
critical scrutiny than probably any other motiva-
tional analysis in contemporary psychology. Orne
was struck by the extent to which laboratory sub-
jects are willing to comply with the experimenter's
requests: They act in all sorts of seemingly silly or
meaningless ways and sometimes even agree to do
things that otherwise are repugnant to them
merely because the experimenter asks them to do
so. The reason for this acquiescence, according to
Orne, is that the subjects "share (with the exper^
imenter) the hope and expectation that the study
in which they are participating will in some ma-
terial way contribute to science and perhaps ulti-
mately to human welfare in general" (p. 778).
They supposedly go along with the experimenter's
explicit requests because they identify with the
goals of science and want the experiment to be
successful. Moreover, Orne insisted that out of this
concern for science, the subjects try to determine
the true purpose of the experiment so that they
can respond in a way that will help to validate the
experimental hypothesis. They then react to cues
in the situation that indicate what the hypothesis
is—the demand characteristics of the situation—
rather than to the experimental variables manip-
ulated by the investigator.
Future historians of social science surely will
marvel at how this reasoning has been accepted
in the absence of clear evidence supporting it. The
250 • MARCH 1982 • AMERICAN PSYCHOLOGIST
great majority of laboratory subjects obviously do
follow the experimental rules spelled out to them.
We cannot be sure, however, that this compliance
stems from the participants' identification with sci-
ence, as Orne claimed, rather than from other
motives such as an inclination to obey the dictates
of legitimate authority (such as the experimenter),
the desire for the experimenter's approval or other
benefits that might be obtained, and even intel-
lectual curiosity regarding the experiment itself.
Whatever the reason for the subjects' acquiescence
to the explicit requests made of them, this com-
pliance does not in itself necessarily imply that
they will strive to seek out the experiment's "true
purpose" or that they will be especially astute in
appertaining the hypothesis or that they will be
particularly intent on confirming the hypothesis.
(For example, assuming they are committed to sci-
ence, could they not try to be "good subjects" by
displaying their true dispositions rather than by
behaving "falsely" in order to do what the re-
searcher expected?) In bur view, Orne's conception
of demand characteristics is based on a chain of
speculation that does not withstand close exami-
nation.
Nor, for that matter, is there much empirical
support for Orne's contention that subjects typi-
cally try hard to confirm the experimenter's hy-
pothesis. According to a number of studies now,
as reviews by Weber and Cook (1972), Kruglanski
(1975), and Silverman (1977) have shown, there
is little reason to believe that the subjects' behavior
in the laboratory is governed by a pervasive and
powerful motive to prove the experimenter's hy-
pothesis. First of all, our own experience and other
researchers' reports indicate that there is not as
much awareness of the research hypothesis in
many experiments as the critics have claimed. (Of
course, the experimenter can produce this aware-
ness in his or her subjects at the end of the session
by insistently asking them about the deceptions
that had been practiced and what the experiment
was "really" about.) Then tod, in a number of stud-
ies, when the participants were deliberately made
•aware of the experiment's purpose at the start of
the session, many of these people subsequently
"leaned over backwards" to avoid confirming the
supposed research expectation (see Kruglanski,
1975, p. 117). They apparently were more inclined
to be negatiyistic than conformistic. And further-
more, as Weber and Cook (1972) and Silverman
(1977) have pointed out, at least some of the pub-
lished demonstrations of subject cooperativeness
could have grown out of the participants' desire
to present themselves in a healthy light and not
from their supposed desire to confirm the re-
searcher's hypothesis. In sum, while some subjects
may occasionally try to live up to the research
expectation, as they understand it, we should not
automatically assume that any given laboratory
finding is necessarily produced by demand char-
acteristics; the motive to confirm the experi-
menter's hypothesis is probably too rare and too
weak to be counted on as a ready determinant of
the experimental outcome.
EVALUATION APPREHENSION
Even though very few subjects have any particular
interest in confirming the investigator's expecta-
tions, most of them probably want to look good to
the experimenter. In Rosenberg's (1969) phrase,
their behavior is dictated to some degree by eval-
uation apprehension. Rosenberg noted that many
people believe psychologists are generally able to
assess the intelligence, competence, and emotional
adjustment of those with whom they come into
contact. Undergraduate students face to face with
a psychologist in a laboratory experiment might
well think that they are being-evaluated and, to
alleviate their apprehension, try to impress the
experimenter by appearing smart, sophisticated,
or healthy.
We have little doubt that this type of desire is
very widespread. The Weber and Cook (1972) sur-
vey of the research into subject artifacts reported
clear evidence that this motivation is prevalent
throughout the usual subject population. Moreover,
when this motive is pitted against the supposed
wish to confirm the researcher's hypothesis, most
participants prefer to look good, as Carlsmith et
al. have commented (1976, p. 282).5 This is an
important point and we will return to it later in
our discussion of the research on aggression. For
our present purposes, however, we will observe
only that the subjects' evaluation apprehension
should diminish as they learn that every experi-
menter (or every psychologist for that matter) is
not interested in assessing their personality or com-
petence.
The Validity of Research on Aggression
Recent years have seen a rising tide of doubt re-
garding the validity of the procedures typically
followed in laboratory experiments on aggression
(e.g., Kane, Joseph, & Tedeschi, 1976; Tedeschi,
Smith, & Brown, 1974). Additionally, a number
of psychologists who are ordinarily quite sympa-
thetic to laboratory research have expressed serious
misgivings about the experiments on aggression.
They accept many of the pro-experiment argu-
ments that have been offered here in the abstract
but seem to regard the standard laboratory studies
of aggression as beyond the pale and lacking in
external validity. In the remainder of this article
we will attempt to demonstrate that the points we
made earlier regarding the external validity of lab-
oratory experiments in general also apply to many
experiments on aggression in particular.
THE CONSTRUCT VALIDITY OF THE
LABORATORY MEASURES
Many of the criticisms of experiments on aggres-
sion focus on the laboratory measures of aggression.
These objections generally insist that the measures
do not really reflect the subjects' aggressive incli-
nations, either because (a) the participants are only
complying to the demand characteristics in the
situation or (b) the measures do not capture what
the critics regard as the most important aspects of
aggressive behavior. Let us consider each of these
charges in turn,
Demand characteristics. The procedures fol-
lowed in most aggression experiments are now so
familiar that few readers of the laboratory reports
are surprised by the experimental manipulations.
The comparatively sophisticated audience is gen-
erally aware of what is done in this research and
why. Quite a few members of this audience then
assume that the laboratory subjects also must have
caught on to the nature and purpose of the de-
ceptions that had been practiced. We will not ar-
gue about how many participants are fooled, al-
though in our experience there are not as many
"aware" subjects in the typical experiment as the
critics suppose. (University students often know
about the possibility of deception in experiments,
but if the cover story is adequate, they are usually
not quite sure what, if any, deceptions might have
been used in any given study.) What is more im-
portant to us here is the inference that is then
frequently drawn from this supposedly high level
5
In his summary of the relevant literature, Silyerman (1977)
suggested that demand characteristics are most likely to operate
(if at all) when the subjects are not sure what response will place
them in a favorable light. On the other hand, when evaluation
apprehension and demand cues are in opposition, Silverman
concluded that subjects are apt to try to enhance their own
image rather than the experimenter's.
AMERICAN PSYCHOLOGIST * MARCH 1982 • 251
of subject awareness: It is often claimed that once
the participants realize what the experiment is
really about, they become motivated to act the
way they think the experimenter desires because
they want to help confirm the experimenter's hy-
pothesis.
This demand characteristics argument can be
readily illustrated by the research on the "weapons
effect" initiated by Berkowitz and LePage (1967).
Assuming that weapons have an aggressive mean-
ing for many people in our society, Berkowitz and
LePage hypothesized that the mere sight of weap-
ons would function like classically conditioned
stimuli to elicit involuntary, aggression-facilitating
reactions. In keeping with their analysis, deliber-
ately provoked subjects administered more electric
shocks to their frustrater when they were in the
presence of guns than when only neutral objects
were nearby. Other researchers held, however,
that the Berkowitz-LePage findings were artifac-
tual and arose because of the demand cues in their
study (e.g., Page & Scheldt, 1971). The experi-
mejntal provocation supposedly informed the sub-
jects that they were expected to "get even" with
their tormentor, and more important, the presence
of the weapons was presumably a clear commu-
nication telling the participants that they should
deliver many shocks to the other person. The Ber-
kowitz-LePage subjects had only given the re-
searchers the kind of behavior they wanted. In
spite of Berkowitz's (1971) attempted rebuttal, it
now seems to be widely believed that the weapons-
effect studies (and other aggression experiments
as well) are greatly contaminated by the subjects'
compliance to demand cues.
As we commented earlier, much of the empir-
ical support for this conception is questionable and
rests on ambiguous correlational observations. As
an example, in the Page and Scheidt (1971) re-
search attacking the weapons effect, many of the
participants who had delivered a high number of
shocks in the presence of the guns verbalized the
weapons hypothesis when they were questioned at
the end of the session. This correlation of behavior
with awareness does not prove that the awareness
had preceded and caused the action. Many persons
seek to justify their aggression by inventing suit-
able reasons after they have attacked their victim
(Brock & Pallak, 1969). This justification could
conceivably have been provided when the exper-
imenter interviewed the subjects at the end of the
session, asking them about the possible reasons for
the presence of the guns and their suspicions. They
252 • MARCH 1982 • AMERICAN PSYCHOLOGIST
could now believe the notion implanted in them
by the questioning because it explained and ex-
cused what they had done.
If the claims made for demand characteristics
are somewhat dubious, however, other possible
artifacts-such as evaluation apprehension might
still operate. Indeed, along with Weber and Cook
(1972) and Carlsmith et.al. (1976), our experience
suggests that many of the students seem to be much
more intent on looking "good"-—on appearing
"healthy" and well-adjusted—than on confirming
the experimenter's hypothesis. This is shown by
the Turner and Simons (1974) study of how subject
sophistication and evaluation apprehension com-
bine to affect the participants' reactions to the sight
of weapons. In this experiment the investigators
established different levels of sophistication in their
subjects, by having a confederate give each naive
participant certain information before the session
began. Cross-cutting this manipulation, some of
the men were also induced to have'a high level of
evaluation apprehension by telling them the re-
search was concerned with psychological adjust-
ment, while the others were made to be less ap-
prehensive about the purpose of the study. All of
the subjects were deliberately provoked by a part-
ner and then each participant had an opportunity
to shock this person while two guns were visible
on a nearby table. According to Turner and Si-
mons, both experimental variations had a signifi-
cant effect on the number of shocks that were ad-
ministered. The greater the subjects' awareness
that the investigator was interested in their re-
sponse to the weapons, the fewer the shocks they
gave. Rather than being more compliant to the
experimental demands, the sophisticated subjects
tended to hold back on the shocks they delivered.
The reason for this restraint is obvious: Aggression
is frowned on in our society and the subjects did
not want to exhibit "bad" behavior. We can also
see this in the evaluation apprehension findings:
The subjects who were made to be concerned
about their psychological adjustment gave fewer
shocks than their less apprehensive counterparts.
At the very least, these findings (and others that
might also be mentioned) clearly indicate that sub-
ject compliance to demand cues is nowhere near
as serious a problem in aggression experiments as
some critics have supposed. The available evidence
suggests that many of the people taking part in
psychological experiments are inclined to be re-
strained in attacking their victims if they suspect
that the researcher is interested in their aggressive
behavior, probably because of evaluation appre-
hension. This inhibiting influence is yet another
restriction on the representativeness of the labo-
ratory setting, but it also means that whatever high
levels of aggression are obtained come about in
spite of the subjects' awareness and not because
they know the study's true purpose.
We believe that some of the objections to the
usual laboratory measures of aggression are also
unwarranted, The most common criticism raises
the matter of the supposed unrepresentativeness
of" the laboratory responses ("Who gives electric
shocks in real life?"), but other critics have ques-
tioned the conception of aggression that is implied
in these responses. In our view both of these points
can be answered with essentially the same argu-
ment: The essential feature of the laboratory be-
havior for the subjects, the meaning their actions
have for them, is that they intentionally are hurting
their victims. This intentional harm fits what we
regard as the best definition of "aggression" and
also provides the measure with a fair degree of
external validity.
Injury as an aggressive goal. In saying that
aggression should be defined as the deliberate in-
jury of another we are of course also maintaining
that the aggressively instigated individual has a
particular goal (to hurt someone) and is reinforced
when this objective is attained. For some investi-
gators (e.g., Patterson & Cobb, 1973; Tedeschi et
al., 1974), the aggressors mainly desire to coerce
their victims. They may hurt their opponents phys-
ically or only threaten to do so, insult them ver-
bally, or attempt to deprive them of some valued
resource. Moreover, although the aggressors may
do these things either in the heat of anger or in
an unemotional pursuit of some other goal, what-
ever they do and in whatever way, their funda-
mental objective supposedly is to influence other
people's behavior. This may be too narrow a con-
ception. Along with other writers (e.g., Feshbach,
1964; Moyer, 1976; Rule & Nesdale, 1974), we sug-
gest that there are different kinds of aggression
and that they are not all equally reinforced by the
same outcomes. All aggression is not carried out
simply to coerce someone. Furthermore, it also
appears that humans who have been angered by
some aversive event are apt to be especially mo-
tivated to inflict harm so that knowledge of the
intended target's appropriate injury is particularly
rewarding.
Laboratory experiments have yielded observa-
tions in accord with this thesis (e.g., Rule & Nes-
dale, 1974). Angered subjects often try to hurt the
one they believe had provoked them. Seeking this
injury, moreover, they also are especially gratified
by information about their tormentor's suffering.
Under suitable conditions, such "pain cues" can
stimulate heightened aggression. Thus, according
to Baron (1977), strongly angered subjects are more
punitive than they otherwise would have been on
learning that their tormentor is in pain. Since the
suitable injury of the tormentor is the provoked
person's goal, neutral stimuli previously associated
with the provocateur's suffering can acquire the
capacity to evoke impulsive, aggressive reactions.
In an experiment by Swart and Berkowitz (1976),
angered subjects displayed the strongest impulsive
aggression when they encountered stimuli that had
earlier been paired with information about their
tormentor's injury. Furthermore, findings pub-
lished by Feshbach, Stiles, and Bitter (1967) in-
dicate that this rewarding pain feedback infor-
mation can also reinforce nonaggressive responses
as well. \
Taken together, these results show that angry
aggressors are motivated to hurt the person who
had offended them. Their attacks in response to
the provocation are not only influence attempts,
nor for that matter, are they only efforts at re-
storing self-esteem. Whatever other outcome may
also be pursued, these aggressive reactions are at
least partly oriented toward inflicting injury. We
conclude from all this that laboratory measures
involving the deliberate harm of another individ-
ual have construct validity as aggressive responses.
Correlations of laboratory measures with other
measures of aggression. Further attesting to the
validity of the laboratory measures of aggression,
there is also evidence that these scores are often
associated with other indexes of aggressiveness.
Williams, Meyerson, and Eron (1967) provided
their third-grade boys and girls with an opportu-
nity to punish a peer for mistakes made on a joint
task, using a slightly modified Buss procedure, and
then related the laboratory scores to an indepen-
dently validated index of each child's customary
aggressiveness as rated by the children's classmates.
The youngsters of both sexes who were seen by
their peers as being most aggressive in their daily
social encounters administered significantly more
intense punishment on the laboratory apparatus,
for longer durations, with shorter latencies, and
with greater frequency than did their less aggres-
sive counterparts. Shemberg, Leventhal, and All-
man (1968) reported similar findings in a study
AMERICAN PSYCHOLOGIST • MARCH 1982 • 253
with high School boys and girls. Again using the
Buss procedure, those teenagers who were rated
by their counselors as being most aggressive gen-
erally gave their victims the most intense punish-
ment. A more recent unpublished investigation by
Leyens (Note 2) in Belgium demonstrates that
these results are also obtained when the partici-
pants are clearly told that their responses will harm
the other person. The school children in this study
were informed that their high-level responses
would prevent their partners from doing what they
had wanted. After each youngster had been frus-
trated by the other child, those who were rated by
their peers as being the most aggressive were the
most intense in their aggressive reactions with the
laboratory apparatus. Finally, Malamuth and Check
(Note 3) found that male subjects with the highest
scores on a scale measuring acceptance of violence
against women were later most aggressive in the
Buss procedure toward a woman who had insulted
them. These same men also considered themselves
angrier with this woman and expressed a stronger
"desire to hurt her with punishment."
All of these observations seem to form a nice,
coherent whole. Taken together, they suggest that
aggression should be defined as the deliberate in-
jury of another and that most laboratory measures
purporting to assess the subject's aggressiveness
actually do fit this definition, at least in part. We
would go even further. Not only do we regard the
laboratory responses as "aggression" but, we be-
lieve, so do most of the subjects. Although the re-
sponses they make in the experiment certainly do
not physically resemble the attacks they display
in their daily lives, all of these actions appear to
have much the saine meaning to the participants;
they know they are intentionally hurting someone
else. This common meaning relates the laboratory
behavior to the conduct shown in more naturalistic
settings.
THE GENERALITY OF THE WEAPONS EFFECT
Another way to support our thesis is to demonstrate
that the findings obtained in the university labo-
ratory are paralleled by observations made in very
different settings with a variety of behavioral mea-
sures. We can do this with the weapons-effect re-
search mentioned earlier dealing with the aggres-
sion-facilitating influence of the mere sight of
weapons.
After the publication of the first experiment re-
porting such an effect (Berkowitz & LePage,
254 • MARCH 1982 • AMERICAN PSYCHOLOGIST
1967), several investigators attempted to replicate
the initial results, while still others sought to show,
as we have seen, that these findings arose because
of the subjects' compliance to demand cues. A sur-
vey of this literature (conveniently summarized by
Turner, Simons, Berkowitz, & Frodi, 1977) tends
to confirm Berkowitz's initial contention; although
there have been some failures to obtain the weap-
ons effect (e.g., Buss, Booker, & Buss, 1972), other
researchers have found that the sight of weapons
can lead to heightened aggression. Moreover, these
positive results have been obtained in Europe as
well as in the United States, with young children
as well as with adults, in naturalistic situations as
well as in the experimental laboratory, and with
a number of different measures of aggression.
Charles Turner has carried out the most com-
prehensive series of studies of the weapons effect.
Two of his experiments utilizing nursery school
children (cited in Turner et al., 1977, p. 361) dem-
onstrated that play with toy guns stimulated the
youngsters to high levels of verbal and physical
antisocial behavior. (The "antisocial behavior" re-
corded in these studies excluded actions that are
generally appropriate for these toys and had more
to do with such acts as pushing and shoving.)
Turner and Goldsmith noted that other studies of
children's aggressive play have yielded very sim-
ilar results, so the findings seem to be quite reliable.
Yet another investigation, employing adults who
did not realize they were taking part in an exper-
iment, also showed that the sight of a gun can
produce increased aggressiveness even in a fantasy
setting (cited in Turner et al., 1977, p. 360). Young
meri participating in spring carnivals at a western
university threw more wet sponges at a clown-tar-
get when a gun was visible in the booth than when
no weapon was present. But it is not only playful
aggression that can be affected by the sight of a
weapon, according to still another field study with
adults. In this experiment (cited in Turner et al.,
pp. 359-360) the aggressive responses were the
blasts of an auto horn sounded by impatient mo-
torists as they waited for a (deliberately) stalled
pickup truck to get out of their way at an inter-
section. Turner and his associates hypothesized that
these aggressive reactions would be most likely to
occur when the frustrating truck bore an aggres-
sive stimulus. The stimulus they placed on the
truck in some instances was the combination of
(a) a rifle prominently displayed on a gun rack at
the back of a truck and (b) a bumper sticker pro-
claiming Vengeance! The investigators assumed
that this latter hostile message would enhance the
rifle's aggressive meaning and found, as they had
predicted, that this combination drew the most
horn blasts from the thwarted motorists behind the
truck.
Space limitations keep us from saying very much
more about this line of research. However, to add
just one observation, we can also note that the ex-
ternal validity of the weapons-effect studies is also
corroborated by Turner's experiments on the role
of memory in this phenomenon. Extending Ber-
kowitz's initial analysis, he argued that besides
having an aggressive meaning, weapons can also
serve as "retrieval cues for earlier aggressive epi-
sodes." If this is the case, he reasoned, aggressive
stimuli that are most easily imagined (i.e., that
have the greatest imagery value) should be most
likely to remind people of earlier aggressive epi-
sodes in their lives and therefore should be the best
elicitors of aggressive responses from those who are
disposed to be aggressive. An experiment by
Turner and Layton (1976) yielded supporting ev-
idence for this hypothesis.
Conclusions
We should be very clear as to just what we are,
or are not, claiming for laboratory experiments in
general and aggression experiments in particular.
Our central argument has to do with the external
validity of their findings—the extent to which the
laboratory results can be generalized to other sam-
ples, settings, and specific behaviors. External va-
lidity, we have argued, does not necessarily re-
quire ecological validity. The experimental results
may tell us something about the conduct of a broad
range of people in natural situations even though
the subjects and laboratory settings are not phys-
ically representative of this population or the real-
world situations in which they are embedded. The
meaning the laboratory participants impart to the
experimental stimulation and the responses they
make determines, at least in large part, the extent
to which their laboratory reactions parallel the
behavior they exhibit in other settings.
The experimenter would do well to strive for
experimental rather than mundane realism (to use
the Carlsmith et al., 1976, terminology) to facilitate
the desired interpretation of the laboratory events.
However, no one can guarantee at the outset that
the subjects' understanding of what is happening
to them in the social psychological laboratory will
resemble the way they interpret supposedly similar
Occurrences in other settings. They may assign
much the same meanings to both sets of stimuli
or they may not. Many people may ordinarily
think of a gun only as an object used for hurting
others, but filled with evaluation apprehension in
the laboratory (because they believe their adjust-
ment is being tested), they may regard a nearby
pistol in a different manner—for example, as
wicked and dangerous. They may therefore re-
spond quite differently in these two situations.
Thus, we are not insisting that the laboratory find-
ings are necessarily generalizable to the world out-
side. No blanket statement can be made one way
or another-about experiments as a whole. Just as
it is incorrect to say laboratory behavior will not
arise in other settings that are physically very dif-
ferent, so it is equally wrong to hold that the sub-
jects' laboratory reactions will definitely occur on
more naturalistic occasions. Appropriate question-
ing is vital to ensure that the participants have
interpreted the experimental treatments in the
desired way.
Even if the crucial aspects of the experiment are
understood as the investigators wanted, the highly
controlled nature of the laboratory procedures usu-
ally rules out the operation of other factors that
can govern the specific behavior in the outside
world. Experiments in the area of aggression usu-
ally attempt to minimize the subjects' inhibitions
against aggression, whereas these restraints are
often quite strong in natural social settings. Be-
cause of the lessened inhibitions in the former
cases, it may actually be somewhat easier to see
the weapons effect in the laboratory than in most
social situations (assuming the participants do not
believe that the experimenter is assessing their per-
sonality). In other words, even if we confine our-
selves to psychological influences, the laboratory
setting is not necessarily representative of the social
world within which many people act. As a con-
sequence, we cannot use our laboratory findings
to estimate the likelihood that a certain class of
responses will occur in naturalistic situations. Sup-
pose that 60% of the subjects in a sample exhibit
heightened aggressiveness over some baseline level
when a weapon is present. Even if these people
were representative of the persons in a larger pop-
ulation, we could not say, of course, that 60% of
this broader group would react in the same way
in a more realistic situation. Experiments are not
conducted to yield such an estimate.
Here we have to emphasize the point made by
AMERICAN PSYCHOLOGIST • MARCH 1982 • 255
virtually every defender of laboratory experimen-
tation. These investigations basically seek to test
causal hypotheses: Can alterations in Variable X
lead to changes in Variable Y? Can the mere pres-
ence (versus the absence) of a weapon produce
stronger displays of open aggression than there
otherwise would have been? Laboratory experi-
ments cannot be truly representative designs and
must be artificial in nature. Because of this lack
of representativeness, they cannot be used to for-
mulate population estimates. However, to the ex-
tent that this artificiality stems from control over
irrelevant variables; the artificiality is the strength
and not the weakness of experiments. To some
degree, then, the attacks on laboratory experimen-
tation reveal a misunderstanding of just why ex-
periments are carried out.
REFERENCE NOTES
1. Bronfenbrenner, U. Ecological validity in research on human
development. In, External Validity in the Study of Human
Development. Symposium presented at the meeting of the
American Psychological Association, Washington, D.C., Sep-
tember 1976.
2. Leyens, J-P Personal communication, summer, 1980.
3. Malamuth, N., & Check, J. V. The effects of mass media
exposure on acceptance of violence against women: A field
experiment. Paper presented at the meeting of the American
Psychological Association, Montreal, September 1980.
REFERENCES
Allport, G. W. The historical background of modern psychol-
ogy. In G. Lindzey & E. Aronson (Eds.), The handbook of
social psychology (Vol. 1). Reading, Mass.: Addison-Wesley,
1968.
Baron, R. A. Human aggression. New York: Plenum, 1977.
Bass, A. R., & Firestone, I. J. Implications of representativeness
for generalizability of field and laboratory research, findings.
American Psychologist, 1980, 35, 463-464. (Comment)
Berkowitz, L. The "weapons effect," demand characteristics,
and the myth of the compliant subject. Journal of Personality
and Social Psychology, 1971, 20, 332-338. '
Berkowitz, L., & LePage, A. Weapons as aggression-eliciting
stimuli. Journal of Personality and Social Psychology, 1967,
7, 202-207.
Brock, T. C., & Pallak, M. S. The consequences of choosing to
be aggressive. In P. G. Zimbardo (Ed.), The cognitive control
of motivation. Glenview, 111.: Scott, Foresman, 1969.
Brunswik, E. Representative design and probabilistic theory in
a functional psychology. Psychological Review, 1955, 62,
193-217.
Buss, A., Booker, A., & Buss, E. Firing a weapon and aggression.
Journal of Personality arid Social Psychology, 1972,22,296-
302.
Byrne, D. Attitudes and attraction. In L. Berkowitz (Ed.), Ad-
vances in experimental social psychology (Vol. 4). New York:
Academic Press, 1969.
Campbell, D. T., & Stanley, J. C. Experimental and quasi-
experimental designs for research. Chicago: Rand-McNally,
1963.
Carlsmith, J. M., Ellsworth, P. C., & Aronson, E. Methods of
research in social psychology, Reading, Mass.: Addison-Wes-
ley, 1976.
256 • MARCH 1982 • AMERICAN PSYCHOLOGIST
Dipboye, R. L., & Flanagan, M. F. Research settings in indus-
trial and organizational psychology: Are findings in the field
more generalizable than in the laboratory? American Psy-
chologist, 1979, 34, 141-150.
Dipboye, R. L., & Flanagan, M. F. Reply to Willems and How-
ard. American Psychologist, 1980, 35, 388-390. (Comment)
Dweck, C. S., Goetz, T. E., & Strauss, N. L. Sex differences in
learned helplessness: IV. An experimental and naturalistic
study of failure generalization and its mediators. Journal of
Personality and Social Psychology, 1980, 38, 441-452.
Feshbach, S. The function of aggression and the regulation of
aggressive drive. Psychological Review, 1964, 71, 257-272.
Feshbach, S., Stiles, W. B., & Bitter, E. The reinforcing effect
of witnessing aggression. Journal of Experimental Research
in Personality, 1967, 2, 133-139.
Flanagan, M. F., if Dipboye, R. L. Representativeness does have
implications for the generalizability of laboratory and field
research findings. American Psychologist, 1980,35,464-467.
(Comment)
Gadlin, H., & Ingle, G. Through the one-way mirror. The limits
of experimental self-reflection. American Psychologist, 1975,
30, 1003-1010.
Gilmour, R., & Duck, S. The development of social psychology.
London: Academic Press, 1980.
Griffitt, W., & Veitch, R. Preacquaintance attitude similarity
and attraction revisited: Ten days in a fall-out shelter. So-
ciometry, 1974, 37, 163-173.
Harre, R., & Secord, P. F. The explanation of social behaviour.
Oxford, England: Blackwell, 1972.
Henshel, R. L. The purposes of laboratory experimentation and
the virtues of deliberate artificiality. Journal of Experimental
Social Psychology, 1980, 16, 466-478.
Kane, T. R.,-Joseph, J. M., & Tedeschi, J. T. Person perception
and the Berkowitz paradigm for the study of aggression. Jour-
nal of Personality and Social Psychology, 1976,33,663-673.
Kruglanski, A. W. The human subject in the psychology ex-
periment: Fact and artifact. In L. Berkowitz (Ed.), Advances
in experimental social psychology (Vol. 8). New York: Ac-
ademic Press, 1975.
Kruglanski, A. W. On the paradigmatic objections to experi-
mental psychology. American Psychologist, 1976, 31, 655-
663.
Littman, R. A. Psychology: The socially indifferent science.
American Psychologist, 1961, 16, 232-236.
Moyer, K. E. The psychobiology of aggression. New York:
Harper & Row, 1976.
Newcomb, T. M. The acquaintance process. New York: Holt,
Rinehart & Winston, 1961.
Orne, M. T. On the social psychology of the psychological ex-
periment: With particular reference to demand characteris-
tics and their implications. American Psychologist, 1962, 17,
776-783.
Page, M., & Scheidt, R. The elusive weapons effect: Demand
awareness, evaluation apprehension, and slightly sophisti-
cated subjects. Journal of Personality and Social Psychology,
1971, 20, 304-318.
Patterson, G. R , & Cobb, J. A. Stimulus control for classes of
noxious behaviors. In J. F. Knutson (Ed.), The control of
aggression: Implications from basic research. Chicago: Al-
dine, 1973.
Postman, L. The probability approach and nomothetic theory.
Psychological Review, 1955, 62, 218-225.
Rokeach, M. Beliefs, attitudes and values. San Francisco: Jos-
. sey-Bass, 1968.
Rosenberg, M. The conditions and consequences of evaluation
apprehension. In R. Rosenthal & R. Rosnow (Eds.), Artifact
in behavioral research. New York: Academic Press, 1969.
Rule, B. G., & Nesdale, A. Differing functions of aggression.
Journal of Personality, 1974, 42, 467-481.
Schultz, D. P. The human subject in psychological research.
Psychological Bulletin, 1969, 72, 214-228.
Shemberg, K. M., Leventhal, D. B., & Allman, L. Aggression
machine performance and rated aggression. Journal of Ex-
perimental Research in Personality, 1968; 3, 117-119.
Silverman, I. The human subject in the psychological labo-
ratory. New York: Pergamon, 1977.
Strickland, L. J., Aboud, F. E., & Gefgen, K. J. (Eds.). Social
psychology in transition. New York: Plenum Press, 1976.
Swart, C., & Berkowitz, L. The effect of a stimulus associated
with a victim's pain on later aggression. Journal of Person-
ality and Social Psychology, 1976, 33, 623-631.
Tedeschi, J. T., Smith, R. B., Ill, & Brown, R. C., Jr. A rein-
terpretation of research on aggression. Psychological Bulletin,
1974, 81, 540-562.
Turner, C.,W., & Layton, J. F. Verbal imagery and connotation
as memory-induced mediators of aggressive behavior. Jour-
nal of Personality and Social Psychology, 1976, 33, 755-763.
Turner, C. W., & Simons, L. A. Effects of subject sophistication
and evaluation/apprehension on aggressive responses to weap-
ons. Journal of Personality and Social Psychology, 1974, 30,
341-348.
Turner, C. W., Simons, L. S., Berkowitz, L., & Frodi, A. The
stimulating and inhibiting effects of weapons on aggressive
behavior. Aggressive Behavior, 1977, 3, 355-378.
Wachtel, P. L. Investigation and its discontents: Some con-
straints on progress in psychological research. American Psy-
chologist, 1980, 35, 399-408.
Weber, S. J., & Cook, T. D. Subject effects in laboratory re-
search: An examination of subject roles, demand character-
istics, and valid inferences. Psychological Bulletin, 1972, 77,
273-295.
Williams, J., Meyerson, L., & Eron, L. Peer-rated aggression
and aggressive responses elicited in an experimental situation.
Child Development, 1967, 38, 181-189.
AMERICAN PSYCHOLOGIST • MARCH 1982 • 257