The Falsification Fallacy

CHARLES F. RUDDER
Behavior Studies

The following essay challenges the currently uncontested interpretation of Karl Popper's account of empirical falsifiability as negation by modus tollens, 1 and proposes and alternative interpretation based on Popper's revision of an early assumption that scientific inference is a form of "material implication". In The Logic o f Scientific Discovery 2 Popper reports that shortly after his 1934 publication of Logik der Forschung, Alfred Tarski led him to reinterpret scientific discovery as a consequence of the falsification of a "deduction" from "entailment". 3 D.C. Phillips' use of Popper's thesis in his critique of pluralistic approaches to educational studies, 4 James Garrison's endorsement of Phillips' interpretation of Popper, 5 and Thomas Cook and Donald Campbell's appeal to Popper's falsifiability in defense of null hypothesis testing in social research 6 are presented as examples of the accepted interpretation of Popper's logic in educational philosophy and research. Although each of these writers appropriates Popper's work somewhat differently, their arguments share the common feature of interpreting "deducibility" as "material implication", which, I will argue, leads to an ambiguity when Popper's description of the validity of empirical falsification is interpreted as modus tollens. Following a review of the problematic context in which Popper's account of empirical falsifiability appears in educational studies, the discussion turns to the more specific task of exposing the ambiguity that results from substituting material implication for entailment in Popper's thesis. This result, to which I refer in the title of the paper as the "falsification fallacy", cancels the corroborative role of Popper's argument in Phillips' critique of methodological pluralism and Cook and Campbell's account of the logical validity of null hypothesis testing, and it raises questions about the general credibility of conventional approaches to educational research.

Gary Cziko notes that, "Asked to find a recurring theme in the issues of
Educational Researcher from the late 1970s and throughout the 1980s, one

would have little difficulty discovering a continuing preoccupation and debate concerning the epistemology of educational research. ''7 Disputes over issues such as the relationship between science and philosophy, logic and experience, and rationality and inquiry in which the epistemological debate in educational studie,s is rooted are not recent developments in modern philosophy. The current
Studies in Philosophy and Education 12: 179-199, 1993. 9 1993Kluwer Academic Publishers. Printed in the Netherlands.

180

CHARLESF. RUDDER

significance of these disputes in efforts to delineate the features of social inquiry relative to philosophy and physics has, however, intensified as a consequence of a "post-positivistic" shift in epistemology and philosophy of science. This shift is reflected in the epistemological debate among educational philosophers and researchers as the erosion of the credibility of positivistic accounts of the relation between theoretical validity and observation in educational studies. Phillips has pointed out and documented the fact that positivism is a historically multifaceted and internally diverse movement which cannot be generally criticized without careful attention to what, if anything, is characteristic of the movement as a whole. 8 An undisputed tenet of the empirical tradition in which positivism is rooted is the assumption that modern science is a unique exemplar of human rationality. Following Locke, modern empiricists assume that scientific "knowledge" is a psychological consequence of "learning" governed by a mechanical determinism. This belief sustained attempts, reflected in the current subjective/objective distinction, to isolate valid from invalid experiences. The assumption that perception can somehow directly confirm conceptual judgments led to efforts to restrict warranted theoretical assertions to statements that are empirically verifiable. That is, the observation of events in the world external to the knower is assumed to determine what is objectively "knowable" in contrast to irrational or pre-rational value judgments that are merely subjectively "believed" as a consequence of being rhetorically or emotionally attractive. Despite extensive efforts by the early positivists to defend the reduction of scientific "proof" to observation, warranted assertability has, since Hume, stubbornly resisted an empirical identity. Analytic philosophy based on the verifiability principle of meaning set out in logical positivism and supported by studies in symbolic logic, philosophy of science, and philosophy of mathematics represents the most sophisticated defense of equating human rationality with the scientific method of inquiry. In an attempt to circumvent Hume's skepticism, analytic philosophy adopted Kant's assertion of the priority of meaning over truth and a modification of Kant's distinction between analytic and synthetic judgments. Proceeding from the assumption that verifiability distinguishes meaningful from meaningless discourse and the analytic/synthetic distinction between forms of verification, analytic empiricists define concepts as hypothetical "constructs" embedded in languages that are deductively consistent. A language is assumed to be deductively consistent as a consequence of "well formed" statements in the language being constructable and derivable from explicit rules which, when followed, dictate the logical conclusions of valid arguments. Explicit rules are thus taken as eliminating the ambiguities characteristic of languages in which the rules have not been, or cannot be, made explicit. Scientific theories and hypotheses are analytically characterized as statements in a language the rules of which are explicit enough to dictate a radically restricted range (optimally one) of logically verifiable conclusion. The conclusions of more ambiguous types of discourse are relegated to the realm of meaningless metaphysical or speculative assertion and dismissed as candidates

THE FALSIFICATIONFALLACY

181

for theoretical truth. Hence, analytic arguments always presuppose some conception of human reasoning resembling logical and/or scientific inquiry, an equation of meaning (rational possibility) and empirically or formally verifiable propositions, and an equation of knowledge (rational actuality) with empirically or formally verified propositions. A major consequence of the influence of analytic empiricism in education is the current institutional monopoly of experimental/quasi-experimental research designs modeled after analytic accounts of the relation between cognition and perception. Under the analytic/synthetic distinction analytic terms and statements in experimental hypotheses have no existential reference or import. The semantic neutrality of analytic terms and statements makes their referents negotiable. In educational research this referential negotiability is reflected in the practice of operationalizing statements by assigning them empirical referents. Deductively verifiable experimental hypotheses are then assumed to be tentatively true or warranted if what is asserted when empirical referents are substituted for the terms of hypothetical statements agrees with observation qua observation. Reducing experimental validity to logical necessity and empirical determinism is defended by analytic philosophers of education and researchers as a means of gaining theoretical "clarity" by reducing ambiguity and achieving practical "objectivity" by eliminating value judgments. There is no denying the growing agreement among philosophers, physical scientists, and social researchers, influenced by developments within analytic philosophy and a variety of "pre" and "post-positivist" movements, that the tide of inquiry has turned against logical and empirical reductionism. One consequence of the anti-reductionist turn has been a radical undermining of the empiricist assumptions upon which mainstream educational research was grounded and which have sustained the institutional priority of experimental and quasi-experimental research designs. More specifically, this shift in the assessment of the veridical role of observation in scientific inquiry undercuts the notion that operationalizing the terms of statements insures the objectivity of research hypotheses, pedagogical objectives, and the content and results of evaluation instruments. A common criticism of mainstream educational research influenced by movements within analytic philosophy turns on the assumption that theories as such have no referents and the claim that when they are assigned empirical referents, observation cannot function as a theory neutral source of evidence that a theory is warranted. One consequence of operationalizing hypothetical terms and statements is that it introduces value judgments which in theory analysis scrupulously excludes. The valuational act of assigning referents to terms neutralizes formal validy and weakens the conditions under which theoretical statements can be empirically assessed. 9 This in turn drives applications of analytic empiricism in social and educational research perilously close to a vicious circle. If, for example, the terms of an analytic description of cognitive schemata are assigned empirical equivalents 1~ - when statements are converted from analytic to synthetic - it is unclear how their experimental agreement with

182

CHARLES F. RUDDER

observation (synthetic truth) relates to their theoretical validity (analytic truth). 11 Moreover, since the classes of observations assigned to different educational theories may, and likely will, be mutually exclusive, a history of agreement between various theories and observation will not support the claim that a given theory is a more comprehensive or adequate account of what it purports to explain than any other. When theories represent different initial conceptions of educational phenomena such as intelligence, knowledge, learning, aptitude, achievement, creativity, motivation, etc. and identify different selections and interpretations of events that qualify as instances or indices of these phenomena, no agreement between a data set and its respective theory can settle disputes about which if any theory represents a superior initial conception of educative events. These and related criticisms of analysis support the conclusion that an ideological bias which infects the selection and interpretation of empirical data makes all theoretical claims empirically "underdetermined".12 Many have further concluded that empirical underdeterminism leaves the hegemony of experimental/quasi-experimental strategies in educational research unwarranted and justifies a pluralistic tolerance of a variety of research strategies committed to a corresponding variety of equally underdetermined theories, data sets, and forms of research-based pedagogical practice. Analytic empiricists have generally opposed pluralistic approaches to educational studies, principally on the ground that methodological pluralism leads to an untenable epistemological relativism. 13 In Conjectures and Refutations 14 and his Autobiography 15 Popper identifies his studies in epistemology and philosophy of science with developments which, since the late twenties, have eroded the credibility of analytic epistemology as a consequence of reinterpretations of the relation between theory and observation provoked by relativity and quantum mechanics. 16 In the accumulating agreement between research and Einstein'S alternative to Newton's theory of gravitation, Popper saw a great disparity between the "revisability" of theoretical physics and theoretical studies in psychology and political philosophy. Popper was impressed not simply by what he took to be the falsification of Newton's theory but also, and more particularly, by the falsifiability of Einstein's theory in the event that facts contrary to its predictions should be observed. 17 In Popper's opinion, the theories of Freud, Adler, and Marx, buttressed by ad hoc conjectures that explain away every theoretically unanticipated development, are unfalsifiable by any turn of events. Believing this to be evidence of a fundamental and irreconcilable difference between physical and social inquiry, Popper attempted to formulate a criterion of "demarcation" which would distinguish scientific from extra-scientific investigations. Popper claims to have discovered and solved "Hume's problem", by which he means that he is the first to see that the problem of empirical validity (Hume,s problem) is logical and hence distinguishable from, and solvable in a manner independent of, the psychological mind/body problem (Descartes', Locke's,

THE FALSIFICATIONFALLACY

183

Berkeley's and Kant's problem). 18 Popper's attempt to solve the logical problem of empirical validity without raising the psychological problem of describing the origin and development of concepts and their relation to perception sharply distinguishes his work from positivism and analytic empiricism. In short, Popper's epistemological distinction between scientific and nonscientific communities of inquiry is entirely logical and is not offered as an explanation of cognition. 19 Unlike the verifiability principle which purports to distinguish valid from invalid experiences, Popper's demarcation only applies to theoretical arguments. At any time and for a variety of reasons, Popper notes, rationally competent individuals such as Einstein and Bohr may or may not find current scientific conclusions convincing. Furthermore, he argues, empirical falsification is deductively valid while empirical verification is not. Popper thus identifies falsifiability as a valid and distinctive characteristic of physical theories, and makes it his criterion for distinguishing scientific from nonscientific statements - the former are empirically falsifiable and the latter are not. Conceding the failure of analytic attempts to defend the validity of empirical verifiability, Phillips and Cook and Campbell enlist Popper's account of falsifiability in defense of moderate theoretical and practical interpretations of this development. Substituting falsifiability for the traditional analytic role of verifiability in philosophy of education and research, Phillips attempts to reconstruct an analytic defense of methodological monism, z~ Making the same substitution, Cook and Campbell, rejecting the claim that theories can be experimentally verified for the weaker claim that they can only be eliminated by empirical falsification, reargue the conventional case for experimental/quasiexperimental research designs.

Phillips, whose work is opposed to what he sees as the specter of relativism inherent in calls for pluralistic approaches to educational research, insists that the conclusions which critics of traditional educational research have drawn from legitimate objections to analytic empiricism are too extreme. 21 With the critics of analysis Phillips agrees that observation and experimentation are theory impregnated and, hence, do not provide unequivocal critical tests of the truth of scientific hypotheses and theories. But, acknowledging the theory dependence of "O" statements, he argues, only reemphasizes something we already knew, namely, the universal fallibility of cognition. We have no grounds for absolute confidence in the results of inquiry - we are irredeemably fallible. This, however, merely underscores the established importance of submitting hypotheses and theories to strict logical tests: It seems to me that, while any conclusions we reach on the basis of evidence must always be held less than absolutely, the logic of experimentation is sound and the method is probably the least controversial one in the researcher's armament. The

184

CHARLES F. RUDDER

classic source, of course, is Book III, Chapter VIII of Mill's System of Logic, and the most famous item is his "method of difference" The question that is open for discussion is not the logic, but rather what can be concluded from a successful (or even an unsuccessful) experiment. 22 Here Phillips simply endorses the popular belief that legitimate criticisms of analytic empiricism do not challenge the logic of experimentation and are thus restricted to its failure to account for what can be validly concluded from logically sound experiments. Phillips turns to Popper for an account of the failure of analytic empiricism to explain the role of observation in scientific discovery. Phillips accepts Popper's claim that the critical flaw in logical positivism and linguistic analysis is the assumption that scientific conclusions can be empirically and experimentally verified. Empirical verification, Phillips argues, falls before the logical fallacy "affirming the consequent": Another problem arises if the test has positive results. For as a point of logic, the positive result does not give unequivocal support to the theory or hypothesis under test; the form of inference involved is "affirming the consequent" which is fallacious. If from some theory (T) it is deduced that some consequence will follow under certain conditions, and if the test is carried out and C is found to occur, it cannot be concluded that therefore is true: If T then C C Therefore T Which is not valid. (Consider "If it is raining then it is cloudy; it is cloudy; therefore it is raining".) 23 In a synopsis of Phillips' position, James Garrison relates Phillips' argument a bit more directly to an interpretation of the logical assumptions entailed in what the researcher working from the confirmationist perspective is doing and would like to claim: Confirmation of anything from grand unified theories to the humblest hypothesis consists in drawing a logical implication from some hypothesis (H) or theory (T) to some empirical-experimental conclusion (E) that says something like "If T (or H) is true then E will be observed." If E is, as a matter of empirical fact, observed, the researchers would like to claim T is confirmed or verified. Schematically, this pattern of argument may be represented in propositional logic this way: P1 P2 C T--)E E T

In research the lo~cal sign "--)" is usually meant to convey some necessary connection, for instance, causality. The problem is that this schema is logically invalid, representing what is commonly called "the fallacy of affirming the consequent." Premises one and two (P1 and P2) may be true and the conclusion (C) nonetheless false. 24 Phillips further appeals to Popper in defense of the claim that empirical falsification is a logically irrefragable demonstration that the traditional analytic

THE FALSIFICATIONFALLACY

185

emphasis on logic and observation was not misplaced but only misdirected. Replacing attempts to empirically verify scientific hypotheses by efforts to empirically falsify them, Phillips argues, conforms to the valid negation o f the antecedent term of an implication when the consequent is false known as modus
tollens:

It was by reflecting upon these matters (in part) that Popper concluded that although a scientific theory or hypothesis cannot be proven, it can be decisively refuted - one negative result can show that a theory is untenable. The logical form involved here would be "modus tollens", which is valid: If T then C Not C Therefore not T Thus, "If it is raining then it is cloudy; it is not cloudy; therefore it is not raining", which is logically unassailable. 25 Again, Garrison endorses Phillips' interpretation of Popper when he says: Unlike the schema for confirmation, this form of deductive reasoning known as
modus tollens, is logically valid. Summing up, Popper declares in effect that all

reasoning is either deductive or defective. 26 Cook and Campbell 27 enlist Popper in defense of the claim that the failure of the verificationist program has not undermined the validity of experimental and quasi-experimental research strategies. Cook and Campbell share Phillips' reluctance to inter analytic empiricism, and are more reluctant than Phillips to abandon the role of confirmation in social research. Working from an eclectic concatenation of "utilitarianism" and a phenomenalistic conception of "truth", Cook and Campbell argue that confirmation may be taken as evidence of "usefulness", but it is not evidence of "truth". Like Phillips and Garrison, Cook and Campbell identify the logical vulnerability of confirmation with "affirming the consequent", "The error of referring to data as proving the truth of a theory can be stated in terms of an old class of logical fallacies such as 'affirming the consequent', 'modus tollens' [sic], or 'the error of the undistributed middle term'", 28 which they illustrate in a manner logically identical to the examples noted above: If Newton's theory A is true, then it should be observed that the tides have a period B, the path of Mars a shape C, the trajectory of a cannonball form D. Observation confirms B, C, and D. 2~erefore Newton's theory A is true. 29 Cook and Campbell defend the utility o f such invalid arguments in practical applications o f empirical research. They then turn to Popper in defense of null hypothesis testing which they claim provides a rigorous and logically valid method for deriving the experimental and quasi-experimental conclusions, "disconfirmed" and "not disconfirrned": Popper's falsificationism . . . . . stresses the ambiguity of confirmation. For him, corroboration gives only the comfort that the theory has survived the test, that even

186

CHARLES F. RUDDER

after the most impressive corroborations of predictions, it has only achieved the status of "not yet disconfirmed." This status of"not yet disconfirmed" is rare and precious in any advanced science, but it is far from the status of"being true. ''3~ Falsification, as Cook and Campbell interpret it, provides the scientifically invaluable service o f elimination. A m o n g the plethora o f possibly true theories and hypotheses that are not empirically verifiable are some, which, through a process they characterize as "winnowing", can be disconfirmed as a consequence o f empirical falsification: If observations inconsistent with B, C, and D are found, these validly falsify the truth of Newton's theory A. The argument is thus highly relevant to a winnowing process, in which predictions and observations serve to weed out the most inadequate theories, including the most inadequate causal hypotheses which seem at first to "explain" a given phenomenon. However, if the predictions are confirmed, the theory remains one of the possible true explanations.31 III The descriptions of the fallacy o f empirical verification and the validity o f falsification summarized above share the common feature o f treating scientific inference as a form of the inference " i f p then q" or "p implies q" (p D q) known as conditional or material implication. Recall, for example, Garrison's remark that: Confirmation of anything from grand unified theories to the humblest hypothesis consists of drawing a logical implication from some hypothesis (H) or theory (T) to some empirical-experimental conclusion (E) that says something like "If T (or H) is true then E will be observed.''32 This equation follows logically from the fact that the ambiguity o f affirming the consequent and the validity o f m o d u s tollens are demonstrable if and only if the material implication "p implies q" is true. Tarski defines implication as a relation between a subordinate antecedent and a superordinate consequent: If we combine two sentences by the words "if .... then .... " we obtain a compound sentence which is denoted as an IMPLICATION or CONDITIONAL SENTENCE. The subordinate clause to which the word "/f' is prefixed is called ANTECEDENT, and the principal clause introduced by the word "then" is called CONSEQUENT. By asserting an implication one asserts that it does not occur that the antecedent is true and the consequent is false. 33 The formal consequences o f Tarski's subordinate/superordinate distinction between sentences also hold for relations between sets or classes which can be illustrated diagrammatically (Figures 1 and 2). 34 Given the non-empty sets P and Q, Figures 1 and 2 can be interpreted as representing the following conditions, hereafter referred to as C1, C l a , C l b , C l c , and C2:

THE FALSIFICATION FALLACY

187

@
Fig. 1. Fig. 2. C1.
a.

-,Q ~ P

b.
C,

C2.

When P is a proper subset of Q, P implies Q (Figure 1)35, and: If P, Q is necessary If Q, P is possible but not necessary If not Q, not P is necessary When P is not a proper subset of Q, it is not the case that P implies Q (Figure 2).

The contingency of the fallacy "affirming the consequent" and the validity of modus tollens on the condition that it does not occur that the antecedent is true when the consequent is false (Figure 2 and C2) can be seen by inspecting the truLh table for material implication. Table I displays three conditions (lines 1, 2, 3) under which the relation (D) is true (Figure 1) and one (line 4) under which it is false (Figure 2 and C2). The conditions set out in the truth table under which (p D q) is true (lines 1, 2, 3), reveal to inspection the consistency of two rules of inference, modus ponens (C 1a) and modus tollens (C l c), from which the truth value of one of terms p or q may be determined when the value of the other term is given, and the ambiguity of "affirming the consequent" and "denying the antecedent". TABLEI p (1) (2) (3) (4) T F F T q T T F F pDq T T T F

188

CHARLES F. RUDDER

The validity of modus ponens and modus tollens derives from the circumstance that i f p D q is true and either p is true or q is false, Table I reveals one and only one corresponding condition, q is true or p is false (lines 1 and 3). Schematically:

modus ponens p~q p q

modus tollens p~q -~q -~p

The ambiguity of affirming the consequent is apparent in the circumstance that if p D q is true and q is true, there are two contradictory corresponding conditions, p is true and p is false (lines 1 and 2), making the truth or falsity of p undecidable (Clb). The Logic of Scientific Discovery 36, Popper's English translation of his Logik der Forschung, was published with new appendices and additional footnotes. The latter include a correction of what he came to see as an earlier confusion of the difference between "deducibility" from "entailment" and material implication. At a point in the original work where he identifies the falsification of t, in the statement t ~ p when p is false, as "the modus tollens of classical logic", 37 Popper adds a footnote concerning his use of the symbol "---~": In connection with the present passage and two later passages ... in which I use the symbol "--->", I wish to say that when writing the book, I was still in a state of confusion about the distinction between a conditional statement (if-then-statement; sometimes called, somewhat misleadingly, "material implication") and a statement about deducibility (or a statement asserting that some conditional statement is logically true, or analytic, or that its antecedent entails its consequent) - a distinction which I was taught to understand by Alfred Tarski, a few months after the publication of the book. The problem is not very relevant to the context of this book; but the confusion should be pointed out nevertheless. 38 Popper attaches minor importance to this confusion because he claims to have never deviated from his original conception of falsification. From the beginning, Popper argues, the mode of falsification to which he attributes the validity of scientific inference was based on what he later calls "entailment" and not on material implication. In light of this correction, Popper advises his reader to interpret the symbol "---~" in the statement "t ~ p" as "entails", meaning "p is deducible from t" or "given t it follows that p", and not to interpret it as meaning "if t then p" or "t implies p,,.39 This in turn means that it can not be uncritically assumed that Popper's rejection of empirical verification is an instance of affirming the consequent of an implication or that his description of empirical falsification is logically equivalent to negation by modus tollens. Now, as indicated above, Phillips, Garrison, and Cook and Campbell ignore this disclaimer 4~ and continue to identify Popper's account of scientific inference with material implication. This equation is contrary to Popper's description of deducibility and leads to an ambiguity when Popper's contradic-

THE FALSIFICATIONFALLACY

189

tion of a deduction from entailment is interpreted as modus tollens. The ambiguity results from substituting, in Popper's terms, the "tautological" negation of the antecedent term in a statement when the truth value of the consequent is given by modus tollens for the "non-tautological" falsification of a statement about a relation between terms by contradiction. This result also disqualifies attempts to empirically test theories when the relation between theoretical hypotheses and empirical evidence is represented as theory implies observation on the same grounds that Phillips, Garrison, and Cook and Campbell reject empirical verification 41 because, like affirming the consequent of an implication, it leaves the truth or falsity of theoretical assertions undecidable when conditions contrary to what is theoretically indicated are observed. The ambiguity can be exposed by examining the logical consequences of Popper' s description of "entailment".

IV Popper characterizes entailment as meta-linguistic, non-tautological, and, more importantly in the present context, as conforming to the inequality "greater than or equal to":
The logical content [of a statement p] is defined, with the help of the concept of derivability, as the class of all non-tautological statements which are derivable from the statement in question (it may be called its consequence class). So the logical content ofp is at least equal to (i.e. greater than or equal to) that of a statement q, if q is derivable from p (or, in symbols. 'p --4 q,).42

TO these remarks Popper adds the footnote: 'p ---4 q' means, according to this explanation, that the conditional statement with the antecedent p and the consequent q is tautological, or logically tree. (At the time of writing the text, I was not clear on this point; nor did I understand the significance of the fact that an assertion about deducibility was a meta-linguistic one . . . . Thus 'p --~ q' may be read here: 'p entails q,).43 Popper further elaborates and clarifies the relation that he has in mind, and which he admonishes his reader not to conflate with material implication, when he says, "If q is derivable from p, but not p from q, then the consequence class of q must be a proper sub-set of the consequence class of p; and p then posseses the larger consequence class, and thereby the greater logical content (or logical force)". 44 Despite some residual terminological ambiguity, it is abundantly clear that Popper interprets entailment as a relation (indicated by the symbol "-4") between classes under which the expression p --~ q indicates that the antecedent p is superordinate to a subordinate consequent q by virtue of the consequent being included in, or as Popper puts it above, a proper sub-set of, the antecedent (Figure 3). The relation between the antecedent and the consequent in Popper's descrip-

190

CHARLESF. RUDDER

Fig. 3. p entails q (p ----)q). tion of entailment (Figure 3) inverts Tarski's description of material implication (Figure 1) as a relation between a subordinate antecedent and a superordinate consequent. Popper's account of falsifiability follows directly from class inclusion as it is outlined above. If the consequence class represented by a statement includes members that are not included in the consequence class represented by a more inclusive statement, the less inclusive statement contradicts, 45 and may be interpreted as falsifying, the more inclusive statement. Popper illustrates deducibility from entailment by comparing the statements: r: "All orbits of heavenly bodies are elliptical." s: "All orbits of planets are elliptical. ''46 Because the statement "All orbits of planets are elliptical.", as Popper puts, it "says less" than the statement "All orbits of heavenly bodies are elliptical.", s is deducible from r, and hence, may falsify r, "but not vice versa. ''47 Since the consequence class indicated by the statement r, "All orbits of heavenly bodies are elliptical.", includes - is greater than or equal to - the consequence class of the statement s about planets, if it is assumed or observed that the orbits of some planets are not elliptical (that is, that what is asserted in the statement with the smaller consequence class is false) the less inclusive assertion may, according to Popper, be interpreted as contradicting and thus, falsifying, the more inclusive assertion (Figure 4). 48

Fig. 4. Falsification of entailment (-~(r ~ s)). Note that the falsification of "r entails s" (Figure 4) is the condition, s is not a proper sub-set of r, under which the subordinate s "implies" the superordinate r

THE FALSIFICATIONFALLACY

191

is false (Figure 2 and C2). Note further Popper's claim that falsifying a statement will not empirically falsify a less universal or less inclusive statement which is the condition under which the logical negation by modus tollens is valid (Clc). Thus, for example, falsifying the statement "All orbits of heavenly bodies (H) are elliptical (E)." does not rule out the possibility that the orbits of some heavenly bodies are elliptical, and hence cannot falsify the less inclusive statement "All orbits of planets (P) are elliptical" (Figure 5). Similarly, Popper argues, induction is logically impossible, not because it attempts to affirm the consequent of an implication, but because the empirical truth of a statement cannot be deduced from a less universal or inclusive statement which is the condition under which the logical inference modus ponens is valid (Figure 1 and Cla). From the consequence class of the assertion "All orbits of planets are elliptical.", which does not eliminate the possibility that the orbits of some heavenly bodies are not elliptical, the more inclusive statement "All orbits of heavenly bodies are elliptical." is not deducible (Figure 5).

Fig. 5. Popper's rejection of empirical verification denies the possibility of arguing from a subordinate class to a superordinate class which is valid under material implication. Hence, Popper argues that the contradiction of a general statement by a conflicting statement with a smaller or less universal consequence class is the only form of empirically valid argument that moves from particular (a less inclusive class) to general (a more inclusive class), or as Popper puts it, in the direction of induction 49. In sum Popper's description of empirical falsification as a movement from a statement about a less inclusive class to a statement about a more inclusive class moves in the same direction as modus ponens under material implication and in the opposite direction from modus tollens. Interpreting Popper's "entailment" (E "says more" than H) as the inequality, the logical content or logical force of E is greater than or equal to the logical content of H (E >_ H), his deducibility from "entailment" can be represented syllogistically as a conjunction of the premises E _> H and H >_ P (the logical content of H is greater than or equal to the logical content of P), from which the conclusion, E _>P (the logical content of E is greater than or equal to the logical content of P) is deducible. Falsification can then be represented as the nontautological contradiction of the premise E _>H by the non-tautological falsification of the conclusion E _>P which meets the test for a valid syllogism.

192 Schematically:

CHARLES F. RUDDER

E includes H = E > H H includes P = H > _ _ P E includes P = E ->P But: E includes P is false = -~ (E _>P) E includes H is false = -~ (E > H) The syllogistic form of Popper's example set out above could thus run: "Is elliptical" includes "All orbits of heavenly bodies" "All orbits of heavenly bodies" includes "All orbits of planets" "Is elliptical" includes "All orbits of planets" But: Some planetary orbits are not elliptical. "Is elliptical" includes "All orbits of planets" is false. "Is elliptical" includes "All orbits of heavenly bodies" is false. Only the conclusion E > P ("Is elliptical" includes "All orbits o f planets") is deducible from the premises E > H and H > P. The non-tautological character of the assertion that E > P is false and its falsification of the premise E > H is apparent in the conditions that the falsity of E > P is not deducible from the premises, is not dictated by prior observation, and can falsify the premise E > H if and only if the premise H > P is not falsifiable. 5~ If, for example, a history of observation includes only elliptical orbits, the discovery of non-elliptical orbits would not be logically deducible or dictated by past experience. Granting that considerations other than orbital motion influence what we are willing to call planets, we must nevertheless discount some past experience if we accept a deviation from prior experience of orbits and decide that some orbits are not elliptical. We could, for instance, save the hypothesis by refusing to include objects with non-elliptical orbits among the things we call planets, which, as we will see below, is the only option permitted under the tautological negation of the antecedent term in an implication by modus tollens. Consider, for example, the datum "The orbit of q' is not an ellipse." and the syllogism (Figure 6): The orbit of every planet (P) is elliptical (E) Every q is a member of P The orbit of every q is a member of E Datum: The orbit o f q ' is not a member of E

THE FALSIFICATIONFALLACY

193

In the logic of classes the fundamental relations are having or not having members and being or not being a member of another class. Following Popper, the datum "The orbit of q" is not E." raises a question, namely, whether to include q' in the class P (Figure 6). If q' is excluded from the class P, the syllogism stands as originally asserted. If we do include q' among the members of the class P (Figure 7), assuming that the expansion of the consequence class of P does not alter its math, the datum may be interpreted as "The orbit of some member of the class P is not a member o f E." which contradicts the conclusion and the premise "The orbit of every P is E."

Fig. 6.

Fig. 7. Interpreting the relation between the orbits of planets and ellipses as the material implication "If q is a member of P then the orbit o f q is E." ((q~P) ~ (q~E)), restricts the valid inference from the datum "The orbit of q' is not a member of E." by modus tollens to the conclusion that q' is not a planet. Schematically: (q' e P) D (q' e E) = q' is a planet implies the orbit ofq' is an ellipse -~ (q' a E) = The orbit of q' is an ellipse, is false -~ (q" ~ P) = q' is a planet, is false This is to say that if the material implication "q is a planet implies the orbit of q is an ellipse" is true, negation by modus tollens will only permit the conclusion that however q' may resemble a planet in other respects, if its orbit is not an ellipse, it is not a planet (Figure 6), which does not and cannot contradict, and thus falsify, the assumption that if q is a planet then the orbit of q is an ellipse

194

CHARLESF. RUDDER

(Figure 7). The uncritical equation of Popper's relation between classes which may be falsified by contradiction with material implication must assume that by falsifying the consequent q, the implication p ~ q is falsifiable (Figures 2, 4, and 7). The argument would then take the form "'q' is a planet.' implies 'The orbit of q' is an ellipse'", "The orbit of q' is an ellipse." is false, therefore "'q' is a planet.' implies 'The orbit of q' is an ellipse.'" is false. That is, it must be assumed that the negation of the consequent -, (q' ~ E) in the expression ((q' e P) D (q' ~ E)) can falsify the expression. Schematically:
(q" e P) ~ (q" e E) ".(q'~ E) ". ((q" e P) D (q" e E))

Table I reveals that when q is false two corresponding contradictory conditions (lines 3 and 4), p ~ q is true (Figure 6 and Clc) and p D q is false (Figures 2, 7 and C2), making the falsity of p D q undecidable when it is asserted and only asserted that q is false. Hence, the equation of Popper's contradiction of an entailment with modus tollens under material implication leads to a logical ambiguity similar to the fallacies "affirming the consequent" and "denying the antecedent".

V Given the collapse of analytic verificationism, the ambiguity identified above leaves the assumption that the logical relation between a theory and empirical evidence may be represented as theory implies observation vulnerable on two counts. First, it fails to meet the logical requirements of Popper's account of scientific inference which robs it of the validity that Popper claims for empirical falsifiability. Second, without the corroboration of Popper's logic, it remains open to the critiques of analytic empiricism and alternative proposals for conducting educational research that Popper's thesis has been imported to counter. The uncritical identification of the link between theory and observation with implication leaves the formal relation between theoretical statements and their objects problematic and the corresponding relation between empirical evidence and theoretical validity ambiguous. These difficulties, which resemble the questions raised by developments within the analytic movement discussed earlier in the paper, show up in the arguments of Phillips, Garrison, and Cook and Campbell summarized above. The claim that "In research the logical sign '--+' is usually meant to convey some necessary connection, for instance, causality.''51 bristles with difficulties. Because logical necessity is analytic and connections between theories and observations are synthetic, this assumption is vulnerable to the blurring of the

THE FALSIFICATION FALLACY

195

distinction between necessity, or as Quine puts it "analycity''52, and synthetic truth created by operationalizing theoretical terms and statements. The identification of psychological causality (theory causes observation) and logical implication (theory implies observation) is also meta-linguistically and formally problematic. It is by no means obvious what it would mean for a theory or hypothesis to "cause" an observation and, if it could, how the observation could be interpreted as either confirming or falsifying it. Formally, the equation of causality and implication is problematic as a consequence of the fact, as Rosser notes, that there is no logical expression for the notion of causality. 53 The problems associated with the formal relation between theoretical statements and their objects are compounded by an ambiguity concerning the veridical relation between theories and observation. Consider, for example, Cook and Campbell's illustration of a falsifiable scientific assertion, "If Newton's theory A is true, then it should be observed that the tides have a period B, the path of Mars a shape C, the trajectory of a cannonball form D. ''54 As noted earlier, Cook and Campbell attribute the impossibility of empirical verification to the fallacy "affirming the consequent" and ascribe the validity of falsification in scientific research to the circumstance that "If observations inconsistent with B, C, and D are found they validly falsify the truth of Newton's theory A. ''55 From these conditions it follows that Cook and Campbell equate the experimentally testable relation between a theory A and observations B, C, and D with the material implication A D (B, C, D) and experimentally valid conclusions with the falsification of the theory A by modus tollens when (B, C, D) is false. 56 The ambiguity appears when we attempt to determine what is being falsified. If it is assumed that A D (B, C, D) is true and asserts something like "theory qua theory (T) independent of its being operationalized 'implies' observation (O)", it is just as logically meaningful to say that any statement - say, the fundamental theorem of arithmetic - "implies" the properties of tides, orbits, and trajectories is falsifiable by observations inconsistent with these conditionsY If we do not assume that T D O is true the truth or falsity of T when O is false is ambiguous because both T when T D O is false and -,T when T ~ O is true correspond to -,O. If, as Popper does, we attempt to escape the ambiguity of the relation between theoretical validity and empirical evidence by assigning the object of the empirical test to a connection between a theoretical assertion and its object, it must be assumed that the validity of the relation indicated by the copula, and not the truth value of one of the terms (theory) when the truth value of the other term (observation) is given, is the subject of the empirical test. Assuming that statements which empirical research attempts to validate take the form "theory implies observation" (T D O), and that the relation (D) is the subject of the empirical falsification rather than a theory qua theory (T), arguing from the negation of the consequent (T ~ O, "-,O, therefore -~(T ~ O)) is also ambiguous in the manner described above; that is, both T ~ O is true and T D O is false correspond to -,O. For a practical illustration of the problems identified above, consider the

196

CHARLESF. RUDDER

hypothesis which regularly appears in teacher evaluation instruments that increased time on task results in gains in academic achievement and the assumption that material implication links testable hypotheses to observations. Assuming that threats to statistical validity have been eliminated, a valid negation of what is theoretically indicated by this hypothesis would take the form: T D O = (Increased time on task) "implies" (Measurable gains in achievement) "~O = No measurable gains in achievement -~T = No increased time on task

Now, if the hypothesis T D O is true, the absence of measurable gains in achievement is open to a number of theoretically indicated interpretations such as that time on task was not increased or, perhaps, not properly increased or that gains in achievement have not yet occurred. This is to say that however what is observed conflicts with theoretically determined expectations, it is validly interpretable only in a manner consistent with theoretically indicated auxialiary hypotheses. If, however, we do not assume that the hypothesis T D O is true, the claim that the absence of observable changes (-10) - again discounting threats to statistical validity - is evidence that the hypothesis is false is logically ambiguous because it is open to contradictory interpretations. The circumstance (-~T) interpreted above as contrary to theoretically indicated auxialiary hypotheses 58 when what is indicated by the implication is true (Table I, line 3), and the circumstance (-~(T D O)) interpretable as contrary to hypothetically indicated factual consequences - say, as Harold Rugg argued, 59 it is observed that for some activities increased time on task impedes achievement - making what is indicated by the implication false (Table I, line 4), correspond to the condition "~O. Hence, taking the absence of observable change as evidence that the experimental hypothesis can be validly assessed flounders on a logical fallacy.

VI The collapse of positivism fully justifies attempts such as Cook and Campbell's to reargue the philosophical foundations of social and educational research. Verificationist descriptions of the scientific method of inquiry that have justified the scientific status of educational research and from which experimental and quasi-experimental research methods were derived are, after all, philosophical artifacts. At present, Popper's account of the role of falsifiability in scientific discovery - as stated or with internally consistent modifications - remains a viable alternative to analytic verificationism. 6~ However, apart from falsifiability

THE FALSIFICATIONFALLACY

t 97

interpreted as the contradiction of an assertion by a less general assertion, Popper's account of the relation between theoretical validity and empirical evidence collapses. Hence, a strict conformity to the logical and meta-logical requirements of Popper's account of falsifiability is essential to any assessment of its aplicability in social and educational inquiry. The ambiguity described above makes Popper's account o f falsifiability and its role in scientific discovery logically incommensurable with Cook and Campbell's attempt to reconstruct an analytic defense of quasi-experimental approaches to social and educational research. The same ambiguity removes the corroboration that Phillips claims for Popper's account o f falsifiability in his defense of methodological monism. Moreover, contrary to the claim that "the logic of experimentation is sound ''61 and that "The question that is open for discussion is not the logic, but rather what can be concluded from a successful ... experiment ''62, Popper's logic of scientific discovery suggests the possibility that the logic of educational experimentation may not be sound and might well be open to critical discussion and revision. Further exploration of these issues might enhance our understanding and contribute to the elimination of some sources o f the disappointing absence of agreement about what we have learned from orthodox approaches to social and educational research and the equally disappointing results of its practical applications. 63

NOTES i See, for example, Meehl, Paul, 'Theoretical Risks and Tabular Asterisks: Sir Karl, Sir Ronald and the Slow Progress of Soft Psychology', in Selected Philosophical and Methodological Papers, Minneapolis: University of Minnesota Press, 1991. C. Antony Anderson and Keith Gunderson (eds.). P. 18ff, Rorer, Leonard G., 'Some Myths of Science in Psychology', in Thinking Clearly about Psychology, Vol. h Minneapolis, University of Minnesota press, 1991, pp. 75-76, and De C. Femandes, Sergio L., Foundations of Objective Knowledge, Boston: D. Reidel Publishing Company, 1985, pp. 73-74. 2 Popper, Karl R., The Logic of Scientific Discovery, New York: Basic Books, Inc., 1959, p. 76. 3 'Entailment' is here restricted to Popper's definition of the the term in his account of the relation between the antecedent and consequent in statements that are empirically falsifiable. Expressions such as 'P entails Q' are interpreted throughout the paper as statements indicating that a class of statements, objects, or events P is greater than or equal to - includes all and possibly more than is included in - the class Q. 4 See for example, Phillips, D.C., Philosophy Science and Social Inquiry, New York: Pergamon Press, 1987. 5 Garrison, James W., 'Some Principles of Positivistic Philosophy of Science', Educational Researcher, November 1986, Vol. 15. No. 11. p. 13. 6 Cook, Thomas D. and Campbell Donald T., Quasi-experimentation: Design and Analysis Issues for Field Settings, Chicago: Rand McNally, 1979. 7 Cz~'Lko,Gary, 'Unpredictability and Indeterminism in Human Behavior: Arguments and Implications for educational Research', Educational Researcher. April 1989, Vol. 18, No. 3. p. 17. 8 Phillips, D.C. 'After the Wake: Postpositivistic Educational Thought'. Educational

198

CHARLES F. RUDDER

Researcher. May 1983. Vol. 12, No. 5. pp. 4-12.

9 See, for example, Kenneth Howe, on the 'dogma' of the fact-value distinction in 'Two Dogmas of Educational Research', Educational Researcher. October 1985, Vol. 14. No. 8 pp. 10-18. t0 Flanagan, Owen, The Science of the Mind, Cambridge: MIT Press, 1991. pp. 307-308. Gardner, Howard, The Mind's New Science, New York: Basic Books, Inc., 1987. P. 323. 11 Quine's critique of the analytic/synthetic distinction meticulously examines these issues (Quine, W.V.O., 'Two Dogmas of Empiricism', in From a Logical Point of View. Cambridge: Harvard University Press, 1962). 12 Garrison, James W., 'Some Principles of Positivistic Philosophy of Science', Educational Researcher. p. 13. See also, Laudan, Larry, Science and Relativism, Chicago: The University of Chicago Press, 1990. pp. 33-68. 13 Laudan, Larry, Science and Relativism. 14 Popper, Karl R. Conjectures and Refutations. pp. 35-36. 15 Autobiography of Karl Popper, in The Philosophy of Karl Popper, Library of Living Philosophers, Volume XIV Book I, Paul Arthur Schlipp (ed.), La Salle: Open Court, 1974. p. 72. 16 See also F.S.C. Northrop's introduction to Wemer Heisenberg's Physics and Philosophy. New York: Harper Torch Books, 1962, and Bartley, William W., Unfathomable Knowledge Unmeasured Wealth. La Salle, Illinois: Open Court, 1990. pp. 172-176. 17 Popper, Autobiography, p. 38. is Popper, Objective Knowledge, p. 4. 19 See, for example, 'Epistemology Without a Knowing Subject', Popper, Objective Knowledge, pp. 106-150. 20 In effect Philfips substitutes a 'logical negativism' for logical positivism. 21 Phillips, D.C., Philosophy, Science, and Social Inquiry. 2z Phillips, D.C., 'Toward an Evaluation of the Experiment in Educational Contexts', Educational Researcher. July 1981, Vol. 10, No. 6. p. 18. 23 Phillips, Philosophy, Science, and Social Inquiry. pp. 14-15. 24 Garrison, James W., 'Some Principles of Positivistic Philosophy of Science', p. 13. 25 Phillips, Philosophy, Science, and Social Inquiry. p. 15. 26 Garrison, 'Positivistic Philosophy of Science'. p. 13. 27 Cook, and Campbell, Quasi-Experimentation: Design and Analysis Issues for Field

Settings. z8 lbid, 21. 29 Ibid, 21. 3o lbid, 21. 31 Ibid, 22.

32 Garrison, 'Positivistic Philosophy of Science'. p. 13. 33 Tarski, Alfred, 'Symbolic Logic', in Newman, James R. (ed.), The World of Mathematics, New York: Simon and Schuster, 1956. p. 1913. 34 Rorer, 'Some Myths of Science in Psychology', p. 75. 35 Ibid, 75. 36 Popper, Karl R., The Logic of Scientific Discovery, New York: Basic Books, Inc., 1959. 37 Ibid, 76. 38 Ibld, 76 n. 39 Ibld, 123 and 123 n *1. 4o Cook and Campbell's erronious and contradictory (see note 56 below) identification of modus tollens as a fallacy (reference 28 above) is puzzling. Is this simply a 'slip of the pen' or did they observe Popper's disclaimer and fail to explore its consequences? 41 See also, Meehl, 'Theoretical Risks and Tabular Asterisks', p. 25. 42 Ibid, 120.

THE FALSIFICATIONFALLACY
43

199

Ibid, 120n

44 Ibid, 120. Emphasis added. 45 Popper' Objective Knowledge, p. 126 and p. 297. 46 Popper, The Logic of Scientific Discovery. p. 121. 47 Ibid.
48

Ibid,

121-122.

49 Popper, The Logic of Scientific Discovery. p. 41 and pp. 121-122. 5o Popper's description of deducibility and falsifiability in The Logic of Scientific Discovery summarized above is an enthymeme; that is, one premise of the syllogism - in the example above 'All heavenly bodies' includes 'All planets' - is not formally explicit. (But see Conjectures and Refutations, pp. 202-204). It is imperative in the present context that this omission be noted because both the truth of the conclusion of the syllogism and the falsification of premise E > H when the conclusion is false depend on the t~ruth of the premise H >_P. Consider, for example, 'Every bird has wings.', 'Every bat is a bird.', therefore, 'Every bat has wings.' 51 Garrison, 'Positivistic Philosophy of Science'. p. 13. 5z Quine, 'Two Dogmas of Empiricism'. 53 Rosser, J. Barkley, Logic for Mathematicians. New York: McGraw-Hill Book Company, Inc., 1953. p. 6. See also Standley, Gerald B. New Methods in Symbolic Logic. Boston: Houghton Mifflin Company, 1971. p. 7 54 Cook and Campbell, Quasi-Experimentation. p. 21. 55 Ibid, 21 56 See note 40. 57 See Standley, for example, on the 'oddness' of material implication in New Methods in

Symbolic Logic. p. 7.
58 Although the present interpretation of Popper's falsifiability does not remove the problems associated with auxiliary hypotheses A and initial conditions C (Meehl, 'Theoretical Risks and Tabular Asterisks', p. 20, and Garrison, 'Positivistic Philosophy of Science', p. 13) it does put them in a different light. The falsification of the expression [(T. A . C) ~ O], does not isolate the conjunction (T. A . C). 59 R u g g , Harold, Imigination, New York: Harper & Row, Publishers, 1963. 60 See, for example, Bartley Unfathomable Knowledge Unmeasured Wealth, pp. 164-184 and Meehl, 'Theoretical Risks and Tabular Asterisks'. 61 Phillips, 'Toward an Evaluation of the Experiment in Educational Contexts', p. 18. 62 IbM, 18. 63 Meehl, 'Theoretical Risks and Tabular Asterisks'.