The Daubert test of reliability: Fighting “junk science” in the

courtrooms
2002, Nov 2nd; from Skeptic Report
by Steven B. Loomis

I. Introduction

In continuation of Renata Zilch’s article on the Toxic Mold scare, this is a brief overview of the US
court system’s treatment of scientific and other expert testimony under the standards set forth by the
Supreme Court in Daubert (Daubert v. Merrill Dow Pharmaceuticals, Inc., 509 U.S. 579, 595 (1993)).
The “toxic” mold case cited by Renata Zilch in her article had the testimony of the Plaintiff’s expert
excluded on the grounds that the methodology used was simply too unreliable to be useful to the jury.
See Allison v. Fire Insurance Exchange, 98 S.W.3d 227 (Tex. App.-Austin 2002, n.w.h.).

The Texas standard, as discussed below, is based directly on the standards set forth by the U.S.
Supreme Court in Daubert. This article is only a brief summary of some of the issues involved in expert
testimony and the efforts by the court system in the U.S. to exclude unreliable expert testimony, or
“junk science,” from trials without interfering with the jury’s basic function as a finder of fact.

II. Overview

The U.S. court system places a great amount of faith in the collective ability of a jury to determine
issues on a bewildering array of subjects, with no special training or education. That faith is tested
somewhat when the jury is faced with testimony from purported experts using techniques or methods
of analysis that lie far outside a lay person’s normal sphere of knowledge.

The courts have an inherent reluctance to take any issue of fact away from the jury. The credibility of
any witness – even an expert witness – is a pure question of fact; there is no legal instruction regarding
what a juror may or may not find “credible” or convincing. The court’s authority is limited to issues of
law, and it has no say on the decision regarding what type of evidence or witness should be believed.

However, the courts – and the drafters of the Federal Rules of Evidence – recognized that unreliable or
irrelevant testimony from an expert is not useful to the jury, and in fact ultimately risks confusing them.
There exists also the very real concern that a jury will give too much credence to a supposed “expert”
based on his qualifications alone, and would be unable to effectively evaluate the science behind any
opinions being offered. Finally, there is also the pragmatic fact that many litigants will simply use
“hired guns” – terminology from the old west used as shorthand for experts-for-pay – who will
essentially testify to anything, as long as they are paid enough.

The Federal Rules therefore set the courts up as “gatekeepers” to insure that only opinions that are
backed by a consistent methodology be allowed before the jury. A subjective opinion from an expert
with insufficient objective evidence to back it up is essentially no evidence at all.

The courts, however, recognize that their job as gatekeepers is extremely limited in scope. The court
does not attempt to weed out an opinion that it believes is incorrect, but only looks to the methodology
used to determine whether the result (whatever it may be) was arrived at in a reliable fashion. The
exclusion of an expert’s testimony is supposed to be the exception, not the rule: “Vigorous cross-
examination, presentation of contrary evidence, and careful instruction on the burden of proof are the
traditional and appropriate means of attacking shaky — but admissible – evidence.”

III. Daubert and The Federal Rules of Civil Procedure

Because of the above considerations, the courts have long filled the role of “gate keeper” to ensure that
expert testimony reaching the jury meets a basic level of reliability. Older case law (based on the
common law “Frye Test”) (2) used a fairly stringent standard for determining if an expert could testify
in front of a jury: the testimony was inadmissible unless “the principle it is based on is sufficiently
established to have a general acceptance to the field in which it belongs.” (3)

The Frye test was used by the majority of U.S. Federal courts for over seventy (70) years.(4) While it
was widely accepted, a relatively small number of courts either refused to follow the strict test or
actually repudiated it directly. (5)

The test, poor at separating any new or novel scientific or technological procedure, came under
increasing attack in the Eighties and early Nineties.(6) The Supreme Court in the Daubert finally
declared that the more flexible Federal Rules of Evidence had completely replaced the Frye test in
determining whether an expert’s testimony was admissible. (7)

IV. The Daubert Test for Reliability

Rule 702 of the Federal Rules of Evidence states, in part:

A witness qualified as an expert by knowledge, skill, experience, training, or education, may testify
thereto in the form of an opinion or otherwise, if:
 1. the testimony is based upon sufficient facts or data,
2. the testimony is the product of reliable principles and methods, and
3. the witness has applied the principles and methods reliably to the facts of the case. (8)

The key for the Court in determining whether an expert may testify before a jury is therefore primarily
one of “reliability of method.” The court will not look at the actual opinion held by an expert, but
merely examines his or her methodology to determine whether the procedures used would be expected
to lead to trustworthy results.(9) If an expert relies on unreliable foundational data or his methodology
is not reliable, then his entire opinion is likewise unreliable and should be excluded from the jury. (10)

A. The Daubert Factors

The U.S. Supreme Court set out several specific factors that should be used by the courts in evaluating
any proposed expert testimony. These factors are not exclusive and some or all may not apply in any
given case, but they are always the place to start the reliability analysis. (11) The factors are as follows:

1. Whether the theory or technique has been scientifically tested;

2. Whether the theory or technique has been subject to peer review or publication;
3. The (expected) error rate of the technique used;
4. Acceptance of the theory or technique in the relevant scientific community.(12)

The test is meant to be a flexible one, with no single factor being dispositive.(13) Likewise, the
Supreme Court recognized that not all factors would be useful in all cases, and that other factors may
be more important than any of the listed ones for a specific case. (14) Obviously, an opinion or type of
analysis created specifically for use in a lawsuit is not given the same weight as a method of analysis
that is widely accepted by the scientific community outside the litigation setting. While the Daubert test
is certainly more liberal than the older, Frye standard, it still allows the exclusion of testimony where
the court is convinced that the method used to support the opinion is simply too poorly designed to be
trustworthy.

B. “Other” Factors

Since the Daubert decision was handed down, the federal courts have identified a number additional
factors which have been useful in examining the reliability of expert opinion. These include the
following:

1. Whether experts are proposing to testify about matters growing naturally and directly out of
research they have conducted independent of the litigation, or whether they have developed
their opinions expressly for purposes of testifying. (15)
2. Whether the expert has unjustifiably extrapolated from an accepted premise to an unfounded
conclusion. (16)
 3. Whether the expert has adequately accounted for obvious alternative explanations. (17)
4. Whether the field of expertise claimed by the expert is known to reach reliable results for the
type of opinion the expert would give (Skeptics will be happy to note that even the most
“reliable” astrologer using methodology “widely accepted” by the relevant “scientific”
community can be excluded under this analysis). (18)

The above list of additional factors is taken directly from the commentary to the 2003 version of the
Federal Rules of Evidence regarding Rule 702.
C. Texas Factors

As Renata’s associated article shows, the Texas courts use a test derived directly from the Supreme
Court’s Daubert analysis.(19) Many states have adopted similar language and analysis, as their state
rules of evidence are patterned directly after – if not identical to – the Federal Rules of Evidence. (20)

The Texas Supreme Court, while accepting the analysis and conclusions of the U.S. Supreme Court in
Daubert, declined to simply adopt the Daubert factors.(21) The Texas courts therefore use a slightly
different formulation of the same concepts. In any case involving a challenge to the reliability of expert
testimony, the courts examine the following factors:

1. The extent to which the theory has been or can be tested;

2. The extent to which the technique relies upon a subjective interpretation by the expert;
3. Whether the theory has been subject to publication and/or peer review;
4. The technique’s potential rate of error;
5. Whether the underlying theory or technique has been generally accepted as valid by the relevant
scientific community; and
6. The non-judicial uses that have been made of the theory or technique (i.e., was is developed
simply for litigation?). (22)

Like Daubert, these factors are non-exclusive; the courts are allowed to look at any facts that it feels are
relevant to its examination of the expert’s methodology. (23)
Also like Daubert, the courts are cautioned to “focus solely on the validity of principles and
methodology underlying the testimony, not the conclusions generated.” (24) This is a test designed to
find bad methodology, not exclude opinions that the court believes are incorrect.

V. Application to Mold Cases

Issues of causation in “toxic” mold cases, such as the one cited by Renata, are well suited for the above
type of reliability analysis. In any case of alleged negligence, it is necessary for a plaintiff to prove not
only that the Defendant has breached a duty of care, but that the breach actually caused the damages
complained of. This type of causation evidence is outside the general knowledge of a lay person and is
therefore almost exclusively within the realm of expert testimony.
Causation, in mold cases (or any case involving alleged health problems from exposure to a substance
or chemical), is a two-step hurdle for the plaintiff. First, he or she must show that the substance in
question can cause the type of damages attributed to it. This is called “general” causation – can the
substance cause the type of damages alleged in the general population? (25)

If the plaintiff passes the first hurdle, then he or she must also show (by a preponderance of the
evidence) that the substance in question actually caused his or her specific injuries. (26)
An expert must show a sufficient factual grounding for both of these levels of causation before that
testimony can be considered reliable enough to be put in front of a jury. In the Allison case cited by
Renata, the plaintiff’s expert relied on epidemiological studies, but failed to show that such reliance
met accepted standards within the scientific community.(27) In fact, the expert testified that:

1. he could not give a confidence interval for the results of his study;
2. he could not give a calculation for any risk factors for exposure; and
3. he could not state that the techniques used in the study were generally accepted. (28)

This testimony, combined with the fact that the expert appeared to be using this study merely for
litigation (and in a manner not generally used by his field of study), allowed the trial court to exclude
his entire testimony without any real controversy. The decision was upheld on appeal with a minimum
of text devoted to the issue. (29)
This case helps illustrate that the science behind at least some of the mold cases is built more upon
wishful thinking and the profit motive than actual science. Until and unless the plaintiffs in such cases
can come up with a methodology that passes muster, the “coming wave” of toxic tort litigation risks
going the way of bathtub fusion.

Endnotes
1. Daubert v. Merrill Dow Pharmaceuticals, Inc., 509 U.S. 579, 595 (1993) (emph. (dashes)
added); see also Notes of Advisory Committee, Rule 702, Federal Rules of Evidence.
2. Frye v. United States, 293 F. 1013, 1014 (D.C. Circuit 1923).
3. Daubert, 727 F. Supp. 570, 572 (S.D. Cal. 1989), aff’d 509 U.S. 579 (1993) (internal quotation
marks omitted) (quoting United States v. Kilgus, 571 F.2d 508, 510 (9th Cir. 1978)).
4. See, e.g., E. Green & C. Nesson, Problems, Cases, and Materials on Evidence, p.649 (1983).
5. See, e.g., DeLuca v. Dow Pharmaceuticals, Inc., 911 F.2d 941, 954-55 (3rd Cir. 1990) (rejecting
Frye standard outright).
6. See Daubert, 509 U.S. 590-91.
7. Id. At 591, 593.
8. Rule 702, Federal Rules of Evidence. Note that this is a “post-Daubert” rendering of the Rule.
The text was changed to acknowledge the Daubert decision, but it also avoided listing any
specific factors.
9. See generally, e.g., North Dallas Diagnostic Center v. Dewberry, 900 S.W.2d 90, 95 (Tex. App.-
Dallas 1995, writ denied) (interpreting the Texas standard based on the Federal standard and
Daubert).
10.Merrell Dow Pharmaceuticals, Inc. v. Havner, 953 S.W.2d 706, 714 (Tex. 1997) (also using the
Texas standard).
11.Daubert, 509 U.S. at 591, 593; see also Kumho Tire Co. V. Carmichael, 526 U.S. 137, 140
(1999); Black v. Food Lion, Inc., 171 F.3d 308 (5th Cir. 1999) (holding that a court should first
consider the applicability of the Daubert factors, and only then determine if any other factors
may be relevant).
12.Daubert, 509 U.S. 591-593.
13.Id.
14.Id.
15.Daubert v. Merrell Dow Pharmaceuticals, Inc., 43 F.3d 1311, 1317 (9th Cir. 1995) (on remand).
16.See General Elec. Co. v. Joiner, 522 U.S. 136, 146 (1997) (noting that the trial court “may
conclude that there is simply too great an analytical gap between the data and the opinion
proffered”).
17.See Claar v. Burlington N.R.R., 29 F.3d 499 (9th Cir. 1994) (excluding expert testimony where
expert failed to even consider possible causes for the plaintiff’s condition).
18.See Kumho Tire, 119 S.Ct.at 1175 (Daubert’s general acceptance factor does not “help show
that an expert’s testimony is reliable where the discipline itself lacks reliability, as for example,
theories grounded in any so-called generally accepted principles of astrology or necromancy.”);
see also Moore v. Ashland Chemical, Inc., 151 F.3d 269 (5th Cir. 1998) (en banc) (clinical
doctor was properly precluded from testifying to the toxicological cause of the plaintiff’s
respiratory problem, where the opinion was not sufficiently grounded in scientific
methodology).
19.C.f. E.I. du Pont Nemours & Co. v. Robinson, 923 S.W.2d 549 (Tex. 1995).
20.E.g., id. And cases cited therein.
21.Id.; see also Gammill v. Jack Williams Chevrolet, 972 S.W.2d 713, 726 (Tex. 1998).
22.Robinson, 923 S.W.2d at 557.
23.Id.
24.Dewberry, 900 S.W.2d at 95.
25.E.g., Havner, 953 S.W.2d at 714-15.
26.Id. At 715.
27.Allison, 98 S.W.3d at 236.
28.Id.
29.Id.