Misconceptions That Increase Error

The practice of law includes a heavy emphasis upon examination of prior cases, comparison of analysis and decisions, and distinguishing between the reasoning in this case or that case as it applies to the facts of the instant case. The search for relevant precedents, lining up authorities, and arguing for a position is the major pursuit of attorneys and judges. This most naturally leads to the assumption that science proceeds in the same way. Therefore, when the legal practitioner evaluates scientific information, the natural habit is to look for cases, studies, experiments, or opinion that supports a desired decision and to expect that there will be some opposing cases, studies, experiments, or opinions. Then the conflict will be argued out on the basis of whose authorities appear the most weighty or can be most persuasively presented to the finder-of-fact. This assumption, uncritically accepted and imposed upon scientific data, leads to the frequent question addressed to scientific experts, "Well, doctor, aren't there studies that disagree with you?" Judges may exclude expert testimony by saying they do not want a battle of the experts, that it is controversial in the scientific community, and that the juries already know all of that stuff anyway. The assumption is that science proceeds like a baseball game box score.

Most scientists understand that knowledge is incremental so that we are always building knowledge on what has gone before that is agreed on as accurate. We may never possess full or perfect knowledge about a phenomenon but we must make our decisions on the most scientifically rigorous evidence available. This means we do not rely upon case studies, anecdotes, experience, or elaborate theories built on little or no empirical data. Effective practical applications of the science of psychology are derived from tested and replicated scientific investigations using scientific methods.

Unfortunately, it is the very success of this approach that has led to the climate in which pseudoscience and poor science flourish. People have learned to be impressed by psychology so therefore they accept current fads and unsupported claims. Anything goes in psychology, so lose weight through hypnosis tapes, learn Chinese while you sleep, and find self-actualization by beating drums in the woods.

Scientific psychology proceeds through the application of systematic observation so that through the observation some concepts are supported and others are rejected. Science grows cumulatively by having the systematic observations publicly verified. Findings are presented so that others can replicate them, criticize, extend, or reject them. Ideas that survive this process are understood to be usable. However, it must also be understood that in the process a single result that falsifies a concept must be given more credence than many that may support it (Meehl, 1978). It is not simply a matter of counting noses as the U. S. Supreme Court decision, Daubert vs. Merrell, Dow (61 U.S.L.W. 4805, 113 S Ct 2786, 1993) also recognizes. The decision, as is discussed below, sets forth the primary criterion for what is scientific as falsifiability and replication.

Psychology is the only scientific discipline within the mental health professionals. Psychiatrists and social workers are not trained as scientists and the practice of psychiatry and social work are not scientific disciplines (McHugh, 1994; Saari, 1994). Unfortunately the practice of psychology by clinicians is divorced from the science of psychology and the credible scientific research in psychology has little or no impact on practitioners (Campbell, 1994; Dawes, 1994; Stricker, 1992). Dawes (1989) described the result of this separation:

The major thrust of APA (American Psychological Association) policy has been to convince the American public that its practicing members have a special expertise and power that simply doesn't exist. . . .And the willingness of psychologists, without facing APA sanctions, to hypothesize in court settings child abuse in the absence of physical evidence-but on the basis of interviews, unvalidated tests, and tests that have been shown to be invalid-is appalling. It is one thing to push for professional status and income based on true expertise. Doing so in the absence of evidence for such expertise-or in the face of evidence that it does not exist-is socially fraudulent. (pp. 14-15)

In the justice system reliance upon expert opinion is based on the assumption that there is a real expertise so that an expert has knowledge that can assist the finder-of-fact in reaching the most accurate decision possible. If that expertise does not exist, there will be a large amount of incompetent and error-ridden opinion offered to the courts under the mantle of objective science. It may be that the most appropriate and helpful expert opinion is to show that there is no trustworthy evidence on either side (Meehl, 1989). If asked whether or not there is general agreement in the scientific community, an expert may in good conscience answer yes, there is consensus but it is wrong.

At the same time, it is necessary to assert what can be offered as solidly supported by scientific knowledge. There is no virtue in attempting to maintain some sort of balance, saying, "on the one hand...but on the other," when there is data demonstrating a given direction. In dealing with child sexual abuse, there is a discernible tilt in the direction of supporting defense consideration of false positives and concern about false accusations (Ceci, 1994).