Trial by data: Psychological research as legal evidence

Trial by Data Psychological Research as Legal Evidence ELIZABETH LOFTUS JOHN MONAHAN

ABSTRACT: Psychologists are playing a new and increasingly important role in the courtroom by offering empirical information derived from psychological research. This article first presents some examples of research testimony and then discusses five issues that are raised by the courtroom use of psychological data: internal validity, external validity, consensual validity, probabilistic conclusions, and the role of the psychologist's personal values.

University of Washington University of California, Irvine

The present article addresses what we believe are several important and unexplored issues arising from the increasingly frequent use of research psychologists in the courtroom. First, however, it may be useful to briefly note the history of social science involvement in the judicial process (cf. Monahan, in press-b).

Historical Review The issues to be decided at a criminal or civil trial are commonly conceived as being of two types. There are questions of law and questions of fact, the former being within the province of the judge and the latter within the province of the jury. It is by now routine for psychologists to be called on to assist the jury in determining such facts as whether a given person was mentally ill at the time a crime was committed, whether someone suffered psychological trauma as the result of an auto accident, or whether a defendant is competent to stand trial. There is a different way in which psychologists inform the triers of fact in a trial, however, that has received little scholarly attention. It has to do, not with providing a clinical evaluation of individuals involved in the case at bar, but with offering empirical information of a general sort derived from psychological research. In such cases, the contribution of the psychological expert consists in informing the jury (or judge, if a jury has been waived) of empirical facts that they may be unaware of and that relate to the behavior of people in general, or at least to the behavior of large classes of people. The testimony does not speak directly to the behavior of a given person, such as a defendant or a witness at the trial, as clinical testimony would, but indirectly includes such persons as specific instances of the general findings observed. 270 • MARCH 1980 • AMERICAN PSYCHOLOGIST Copyright 1980 by the American Psychological Association, Inc. 0003-066X/80/3503-0270$00.7S

Accounts of the introduction of social science into the American legal process generally take as their point of departure the 1954 decision in Brown v. Board o] Education? the case in which the United States Supreme Court struck down "separate but equal" public schools. In a footnote of that decision, reference was made to seven works by contemporary social scientists that had found the effect of segregation on black children to be invidious. First on the list was the now famous "doll study" of psychologist Kenneth B. Clark (1950). Clark had testified on this research as an expert witness in one of the trials subsumed under the Brown case. The decision to include social science testimony had been a highly controversial one. The plaintiffs' lawyers had been split on whether even to

This article was written while Elizabeth Loftus was a fellow at the Center for Advanced Study in the Behavioral Sciences and John Monahan was a visiting scholar at Stanford Law School. The writing of this manuscript was facilitated by support from the National Institute of Mental Health and the National Science Foundation to Elizabeth Loftus, and support from the National Science Foundation and the Andrew Mellon Foundation to the Center. Requests for reprints should be sent to Elizabeth Loftus, Department of Psychology, University of Washington, Seattle, Washington 98195, or to John Monahan, Program in Social Ecology, University of California, Irvine, California 92717. 1 Brown v. Board of Education, 74S Ct. 686 (19S4).

Vol. 35, No. 3,270-283

mention it in the brief. "Jesus Christ," stated William Coleman, a lawyer on Thurgood Marshall's NAACP team, "Those damned dolls! I thought it was a joke" (quoted in Kluger, 1976, p. 312). The debate within the Supreme Court itself was intense. In a memo to his fellow justices while they were considering the case, Robert Jackson wrote that he did not doubt that segregation was painful to blacks. However that may be, and if all the woes of colored children would be solved by forcing them into white company, I do not think we should import into the concept of equal protection of the law these elusive psychological and subjective factors. They are not determinable with satisfactory objectivity or measurable with reasonable certainty. If we adhere to objective criteria the judicial process will still be capricious enough. (Quoted in Kluger, 1976, p. 689)

Chief Justice Earl Warren, seeking to defuse the controversy he had created by including the footnote, later commented that the studies were "merely supportive" and "not the substance" of the ruling. "It was only a note, after all," he said (quoted in Kluger, 1976, p. 106). To gauge the extent to which social scientists have been accepted in the legal system since their introduction in 1954, it is instructive to compare the Brown decision with the 1978 Ballew v. Georgia,- a recent United States Supreme Court case that held that state criminal trials before a jury of five persons violated the constitution. No fewer than 25 studies on the effects of group size on jury decision-making were cited in Ballew, including many from such mainline sources as the Journal of Personality and Social Psychology and Psychological Bulletin. Indeed, Justice Blackmun's opinion reads as though it were written for a psychology journal: Statistical .studies suggest that the risk of convicting an innocent person (Type I error) rises as the size of the jury diminishes. Because the risk of not convicting a guilty person (Type II error) increases with the size of the panel, an optimal jury size can be selected as a function of the interaction between the two risks. Nagel and Neef [1975] concluded that the optimal size, for the purpose of minimizing errors, should vary with the importance attached to the two" types of mistakes. After weighing Type Terror as ten times more significant than Type II, perhaps not an unreasonable assumption, they concluded that the optimal jury size was between six and eight. As the size diminished to five and below, the weighted sum of errors increased because of the enlarging risk of the conviction of innocent defendants, (p. 4220)

The opinion in Ballew goes on to critique the "methodological problems" with research studies reporting no difference between juries o f , various

sizes, including the fact that "only students composed the juries" in some cases (p. 4223). The use of social science data is justified "because they provide the only basis, besides judicial hunch, for a decision about whether smaller and smaller juries will be able to fulfill the purpose and functions of the Sixth Amendment" (p. 4220). The Court concluded that the data "lead us to conclude that the purpose and functioning of the jury in a criminal trial is seriously impaired, and to a constitutional degree, by a reduction in size to below six members" (p. 4222). From one obscure footnote in 1954, social science data have come to occupy 14 paragraphs of a 3 2-paragraph, Supreme Court opinion. More significantly, from a position of being "merely supportive" in Brown, research findings "have led us to conclude . , ," in Ballew. Yet the controversy surrounding the use of social science data is no less intense today than it was 25 years ago. While the decision in Ballew, as in Brown, was unanimous, only three justices would sign it. Three others agreed with the decision, but for different reasons. And three, including Chief Justice Warren Burger, wrote their own concurring opinion specifically rejecting "heavy reliance upon numerology derived from statistical studies." What is "the only basis . . . for a decision" to one third of the Supreme Court is "numerology" to another third. And- the final third appear to have no opinion at all on the issue. As these cases illustrate, there are two ways in which research psychologists can play a role in judicial decisions. A psychologist may be asked to testify in person at a trial as an "expert witness," as was Dr. Clark in one of the cases subsumed under the Brown- decision. Alternatively, the published research of a psychologist may be cited by a judge of an appeals court, without the psychologist having personally testified at the trial whose verdict is being appealed; this was what occurred in the Ballew case. It is the former procedure that we shall address in this article, since the use of studies without the authors having been subject to cross-examination at trial raises complex jurisprudential issues (see Chief Justice Burger's concurring opinion in Ballew), and in any event, psychologists have little control over how others use their published findings. We shall exclude

-Ballew v. Georgia, 98 S. Ct. 1029 (1978).

AMERICAN PSYCHOLOGIST • MARCH 1980 • 271

testimony involving clinical examination, not because it lacks a research foundation—it may or may not be empirically based—but because a substantial body of literature examining clinical testimony is already available (Brodsky, 1977; Brodsky & Robey, 1973; Ziskin, 1975).

Expert Testimony The purpose of all evidence offered at trial, including expert testimony, is to facilitate the acquisition of knowledge by the jury, or trier of fact, thus enabling them to reach a final determination. The system of evidence used in American courts rests on two axioms: Only facts having rational probative or evidential value are admissible, and all facts having such value are admissible unless some specific policy or rule forbids. The trial judge has broad discretion to decide whether a particular piece of evidence, including the evidence of expert testimony, has sufficient probative value to be introduced. The judge must weigh this value against whatever prejudicial effects the evidence might have, and then make a decision on its admissibility in a particular case. The judge, then, is the one to decide whether the jury can receive appreciable help from the expert testimony of a psychologist. The Federal Rules of Evidence (1975) provide this guide to judges: If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise. (Rule 702, p. 80)

The rule is broadly phrased and quite liberal. It allows the testimony not only of persons who possess "scientific" and "technical" knowledge but also of those who possess other kinds of "specialized" knowledge. It recognizes that one can attain expertise in many ways. Thus, the rule allows the testimony not only of "credentialed" experts, such as physicians and physicists, but also of a broad group of "skilled" witnesses, such as bankers or landowners testifying to land values. A leading appellate decision on the subject of expert testimony regarding eyewitness reliability is the 1973 decision in United States v. Amaral.3 While the case concerns eyewitness identification, which we shall consider in detail later, the issues addressed by the court are relevant to the varieties of psychological expert testimony that are the focus of this article. 272 • MARCH 1980 • AMERICAN PSYCHOLOGIST

By way of background, the defendant, Manuel P. Amaral, was charged in 1973 with the robbery of two national banks. Later that year he was tried. At his trial, defense counsel moved to introduce the testimony of a psychologist with regard to the effect of stress on perception and, more generally, regarding the unreliability of eyewitness identification. The trial court refused to admit the testimony on the ground that "it would not be appropriate to take from the jury their own determination as to what weight or effect to give to the evidence of the eyewitness and identifying witnesses and to have that determination put before them on the basis of the expert witness testimony as proffered" (p. 1153). Amaral was found guilty. He appealed his conviction on 'several grounds, one of which was the refusal of the trial court to admit the expert testimony. The U.S. Court of Appeals for the Ninth Circuit held that the trial judge did not err in excluding the testimony. The Court noted that defense counsel had uncovered no confusion or uncertainty about identity in any of the various witnesses. And, while it acknowledged that stress might affect perception, it noted that not all the witnesses were under similar conditions of stress. One witness saw the robber as he sat in his car blocking the exit from the bank parking lot, another saw him from the safety of her house as he returned from robbing the bank, and still another saw him as he entered the bank and approached the teller. Further, the Court felt that defense counsel could use cross-examination to inquire into the witness's capacity and opportunity for observation, his or her attention and interest, and any distraction or division of attention. Amaral stood convicted. The Amaral decision is important in that it reiterated the general principles regarding expert testimony: (a) The witness must be a qualified expert; (b) the testimony must concern a proper, subject matter; (c) the testimony must be in accordance with a generally accepted explanatory theory; and (d) the probative value of the testimony must outweigh its prejudicial effect. In addition, the Court noted that the trial judge has the broad discretion of whether to admit the testimony or not. In Amaral, the judge simply exercised proper discretion when he excluded the testimony. 3 United States v. Amaral. 1973).

F. 2d. 1148 (9th Cir.

In his analysis of Amaral, Woocher (1977) remarks that the appellate decision failed in one important way: It did nothing to establish any guidelines to aid the trial judge in exercising his or her discretion. How is the trial judge to decide whether the expert witness is truly qualified, or whether the testimony concerns a proper subject matter, or whether it conforms to a generally accepted explanatory theory? What standards should the judge use to-weigh its probative value against its potentially prejudicial effects? In terms of the first criterion, the judge can examine evidence of the proposed expert's knowledge, experience, training, or education. In the case of eyewitness identification, in addition to holding an advanced degree in the general field of perception and memory, an expert should have conducted research and published in that area. Deciding whether the testimony is a proper subject matter for expert testimony has been a more troublesome task. To be proper, it must be beyond the knowledge and experience of the average layperson. In fact, members of the Supreme Court of Iowa * recently argued that the expert testimony on this subject is not beyond the knowledge and experience of a juror and thus is not a proper subject of expert testimony. It is true that many jurors have some commonsense knowledge of the factors that may cause mistakes in identification. Most people, for example, know that the longer the period of time between an incident and one's recollection, the worse the memory —the major factor at issue in the Iowa case. But there is ample evidence that jurors have intuition about the reliability of eyewitness accounts that are unsupported by scientific research. For example, a substantial percentage of people are not fully informed about the problems of crossracial identification or how stress affects memory (Loftus, 1979). In addition to being beyond the common knowledge of the jury, expert testimony, to be proper, should not invade the province of the jury. Those who have argued that it does invade the jury's province are saying, in essence, that the expert testimony relates to the credibility of another witness, and it is the jury that is to make determinations about witnesses' credibility. However, the proposed expert testimony does not involve an opinion on the credibility of any particular witness's testimony. The psychologist does not say whether he or she believes a particular piece of

testimony is accurate or not. Rather, the task is one of reviewing the relevant psychological findings and enumerating the various factors affecting the reliability of eyewitness accounts. The psychologist is speaking about the powers of observation and recollection of witnesses in general. The jury then decides what weight to give both the eyewitness testimony and the expert testimony. In fact, it is worth noting that typically expert testimony is judged twice. The judge makes the initial determination of whether or not to permit the testimony. If a positive decision is made, the jury is still free to give it little or no weight. One jury instruction, commonly given, reads: "You are not bound by such expert testimony. Give it the weight that you deem it entitled to." The third criterion is that the expert testimony should conform to a generally accepted explanatory theory. This test has been used in determining the admissibility of such techniques and devices as the polygraph, voiceprints, breathalyzers, and sodium amytal ("truth serum"). However, it has not in the past generally been applied to the testimony of medical experts. The requirement of conformity to a generally accepted explanatory theory has been used only in cases in which the jury is likely to mistakenly regard a particular device, such as the polygraph, as being able to magically reveal the truth. This requirement asks whether the scientific community accepts and agrees about the device before allowing the jury to hear about it. Using this standard, one could conclude that there has been reasonable agreement about, say, the breathalyzer, but general controversy surrounding, say, the polygraph. Since the psychologist is not often testifying about "magical devices" but is simply discussing the research that has been conducted on human behavior, his or her statements more properly belong in the same category with medical testimony, to which the criterion for admissibility requiring conformity to a generally accepted explanatory theory has not been applied (Woocher, 1977). Finally, the fourth criterion is that the probative value of the testimony must outweigh its prejudicial effects. Any testimony has probative value if it is important to the determination of guilt or innocence. When eyewitness identification, for example, plays a major role in a case, there is a danger of convicting an innocent person, and thus

•tState v. Galloway.

275 N.W. Rep., 2nd. 736 (1979).

AMERICAN PSYCHOLOGIST • MARCH 1980 • 273

the eyewitness's testimony and the psychologist's testimony are probative. Since jurors regard eyewitness testimony with little skepticism, the expert testimony will increase the likelihood of critical appraisal (Loftus, 1980). This is its value. In terms of its being prejudicial, there is always the possibility that the research evidence will bias or mislead the jury. The judge, however, can take steps to minimize this possibility. A caution to the jurors that the expert testimony forms only one piece of the evidence that they should consider, as well as an insistence that the expert's statements be based on solid scientific data and cover only those studies bearing on the case, will minimize any prejudicial effects that the testimony might have. In sum, an analysis of the four criteria that the Amaral Court felt must be met for any expert testimony to be admissible leads to the conclusion that expert testimony of research psychologists, at least on the issue of the credibility of eyewitness identification, satisfies the law. A similar positive attitude toward this type of testimony can be found in the writings of other legal scholars (e.g., Addison, 1978; Katz & Reid, 1977; Sobel, 1979). Our first example is one in which testimony was offered on this subject. EXAMPLE 1 : EYEWITNESS

IDENTIFICATION

One of us (E. L.) has testified, on about 30 occasions, at criminal trials regarding the ability of people to perceive and recall complex events and to identify persons who they may have seen on a prior occasion. One case arose out of an incident that occurred in April 197'S, in the state of Washington. The defendant was arrested after he left a small grocery store carrying bottles of wine and beer for which he had allegedly not paid. The arrest occurred because a clerk in the grocery store called the police and stated that the man leaving was the same man who had robbed him at knifepoint in February of that year. The police located the defendant a few blocks from the store, took him to police headquarters, and booked him for the armed robbery that had occurred 7 weeks before. Although the defendant admitted to having shoplifted the beer and wine (having been arrested with them in his possession), he denied having committed the more serious offense of armed robbery. He protested his arrest on the robbery charge, arguing that the clerk had made a mistaken 274 • MARCH 1980 • AMERICAN PSYCHOLOGIST

identification. The clerk, on the other hand, maintained that it was the defendant who had robbed him. Thus, the only piece of evidence against the defendant offered at his trial for armed robbery was the word of a single eyewitness. At the trial, the defense counsel attempted to introduce the testimony of the psychological expert for two reasons: (a) to discuss the nature of human memory and the mental processes involved in an eyewitness identification, and (b) to discuss the factors that both are known to influence the accuracy of an identification and were also present in the criminal incident for which the defendant was being tried. The first major factor was the long retention interval, that is, the relatively long period of time between the incident and the subsequent identification (Ebbinghaus, 1885/1964). Approximately 7 weeks had passed. The second major factor concerned the fact that the defendant had allegedly been seen by the identifying witness on several occasions prior to the identification. The defendant claimed that he had been in the grocery store a number of times when the clerk was on duty and that the clerk must have confused him with the robber. Thus, the defense position was that an unconscious transference (Loftus, 1976) had taken place. The expert simply described some of the laboratory research that had been conducted on both the retention interval and the phenomenon of unconscious transference. The jury deliberated only 2 hours before reaching a verdict of guilty. They apparently decided that the eyewitness had not made a mistake. One juror was later interviewed, and when asked "What did you think about the eyewitness testimony?" the juror responded, "We believe that the passage of time can have a serious effect on a person's memory. We believe that an unconscious transference can sometimes take place. And yet . . . we felt that the experience of a subject in a psychologist's experiment and the experience of a person with a knife at his throat . . . they just aren't the same!" A second case concerned the assault of a soldier at the Ft. Steward, Georgia, army base, According to the victim's testimony, the incident began early in the morning at approximately 5:00 a.m. The victim, who had gone to sleep at about 4:30, woke up when he was hit in the mouth. He brushed his hand against his mouth and saw that he was bleeding. "I then looked up and saw this guy holding a pistol pointed at my head. I then told

him, 'Don't shoot, I'll do anything you say!'" familiarity that the victim experienced could have The assailant then tried to shove his pistol into been incorrectly related back to the incident, causthe victim's rectum. A struggle and considerable ing the victim to believe that it was the sergeant yelling followed, and finally the victim managed to who had committed the assault. The defendant, in this case, was acquitted. grab the pistol and run outside of the building. On the night of the incident, the victim signed a statement claiming that "I never saw this guy EXAMPLE 2 : THE PREDICTION OF VIOLENT BEHAVIOR before, but I feel I could identify him if I saw him again." He went on to describe his assailant One of us (J. M.) has testified, on about 10 occaas 5 feet 11 inches or 6 feet tall, with a medium sions, at criminal and civil trials regarding the build, a dark complexion, and a medium-sized ability of psychologists and others to forecast acAfro haircut. He appeared to be intoxicated. curately the occurrence of violent behavior. This "There wasn't anything outstanding I remember issue has arisen in the context of involuntary commitment to mental hospitals (of persons defined except he sure looked ugly to me." as "dangerous to others"), release from prison Five days after the incident, the victim atsentences of indeterminate length (release being tended a lineup. At this time he identified andenied when a prisoner was predicted to be vioother soldier as his assailant. One day later, the lent), and most recently the imposition of the victim was involved in an automobile accident with a sergeant, and at this time he claimed to death penalty (which, according to the U.S. Surecognize the sergeant as his assailant. His pas- preme Court/"' may be made contingent on a presenger at the time of the accident confirmed the diction that certain classes of offenders will confact that the sergeant had been in the barracks tinue to be violent if not executed) (Shah, 1978; on the morning of the assault. A second lineup Underwood, 1979). In each case, the expert testithat took place 4 days after the accident included mony has been to the effect that the available reboth of the previously identified men. The victim search indicates positive predictions of violent behavior to be largely inaccurate. The testimony picked the sergeant. Three months later, the sergeant's court-mar- has been used by the defense in the criminal cases tial was held. Defense counsel introduced the and by the plaintiffs in the civil cases to support testimony of a psychologist to discuss the mental their contention that such predictive decision makprocesses involved in an eyewitness identification ing is inherently arbitrary. The testimony-was based on a group of studies and some of the factors relevant to the case known to influence the accuracy of an identification. The that attempted to validate clinical and actuarial first major factor was the fact that the witness was predictions with respect to persons released from white and the defendant black; that is, it was a prisons or mental hospitals who had been predicted matter of cross-racial identification (cf. Malpass & to be violent in the community. Follow-ups of arKravitz, 1969). The second factor was the stress rest and mental-hospital records for a period of experienced by the victim (cf. Easterbrook, 1959)'. from 1 to 5 years after release found that at least Third, the phenomenon of weapon focus was dis- two out of three, and sometimes over nine out of cussed (cf. Loftus, 1979). Fourth, the victim, ten, predictions were false positives: The people by his own admission, had been drinking prior to predicted to be violent, in fact, were not discovered the incident (Birnbaum, Parker, Hartley, & Noble, to be so (Monahan, 1978). The testimony has been met with mixed legal 1978; Parker, Birnbaum, & Noble, 1976). These reviews. In a civil commitment case of national factors, the expert testified, when taken alone but 0 importance, it was completely ignored by a threeespecially in combination, would tend to reduce a person's perceptual and memorial abilities. Finally, judge panel, which unanimously ruled that committhe phenomenon of unconscious transference was ment could be justified even if the committing described to the jury in the following way. When psychologists or psychiatrists had no overt act of the victim saw the sergeant at the time of the violence on which to base their prediction. The automobile accident, he may have looked familiar. The familiarity could have been due to a chance r 'Jurek v. Texas, 96 S. Ct. 2950 (1976). encounter someplace on the military base, since "United States ex. rel Matthew v. Nelson, No. 72-Cthe two men frequented the same locations. The 2104 (N.J. 111. August 18, 1975). AMERICAN PSYCHOLOGIST • MARCH 1980 • 275

issue of indeterminate prison sentences was mooted when the California legislature abolished the statute before a decision had been reached. In the death penalty cases, however, the testimony appears to have been influential. To date, the death penalty has not been imposed in any case in which this testimony has been introduced. In the most recent case, the jury that had unanimously convicted a defendant of the premeditated murder of five persons was split 11 to 1 in favor of imposing the death penalty. Since unanimity was required, the defendant was not executed but received a life sentence without the possibility of parole. When interviewed subsequent to the trial, the juror who voted in the minority stated that he "held out" because of the question of predictability. The 11 other jurors, however, were not impressed. One juror stated that he found the testimony informative but ultimately discounted it, "You can prove anything with statistics," he said. FURTHER EXAMPLES

The two examples we provided illustrate psychological testimony at work in criminal cases, but the eyewitness expert has also testified in the civil arena (Lower, 1978). Examples in the civil area include testimony on the adequacy of warning labels, or the meaning intended by the use of particular words, in cases where empirical data concerning the public's likely perception of an event are relevant. Another area of relevance for psychological testimony is trademark infringement or fraudulent advertising, again in cases where data concerning human reactions are relevant. Psychological testimony has also been offered in obscenity trials, in libel cases, and in child custody disputes (Woody, 1977). Finally, testimony that is more statistical in nature has been common in jury-discrimination complaints, antitrust cases, and numerous class-action suits (Van Matre & Clark, 1976). These examples of expert testimony indicate the range of issues to which research has been addressed. Those described in detail, Examples 1 and 2, illustrate the varied reception accorded expert testimony: In some cases the testimony appears to have been influential, and in other cases, clearly not so. The fact that juries are willing to discount expert testimony should comfort those who believe it might exert an undue influence 276 • MARCH 1980 • AMERICAN PSYCHOLOGIST

upon their decision making. To our occasional chagrin, this does not appear to be the case.

General Considerations The psychologist who is asked to present expert testimony must face a variety of ethical and other considerations. Many of these concerns have been raised in other contexts in the field of psychology. However, they take on increased importance and novel forms when one attempts to introduce psychological research into the courtroom. These considerations include, but are not limited to, the following: 1. Are the findings being presented trustworthy, that is, do they possess internal validity? 2. Do the findings generalize from the situation in which they were observed to the situation to which the expert is'relating them, that is, do they possess external validity? 3. When there is lack of consensual validity among psychologists regarding the interpretation of data, does the expert have a duty to present both sides? 4. How should the psychologist deal with the fact that any conclusion that might be reached can only be probabilistic in nature? 5. What role, if any, should the psychologist's personal values play in the decision to testify in a particular case or class of cases? THE TRUSTWORTHINESS OF RESEARCH FINDINGS

Psychologists have placed a heavy burden of proof on one another for validating research hypotheses. Any decision about observed data is made on the basis of statistical tests and inferences. By convention, one must be able to reject chance as an explanatory variable in, at least, 19 out of 20 cases (a — .05). Similarly, jurors take in evidence presented in testimony or hearings and make decisions based on this evidence, using "preponderance of evidence" as the standard in a civil suit or "reasonable doubt" in a criminal suit. Although some legal scholars reject any quantification of the burden of proof in law (Tribe, 1971), others allow it. In criminal cases, a figure approximating .95 for the reasonable doubt standard has been put forward by some legal writers (Cocozza & Steadman, 1076; Kahle & Sales, 1980; Stone, 197S).

What may be taken as a "significant" finding etal. (1971) noted that by psychologists, however, may appear otherwise operationally, group counseling means that ten or twelve in the courtroom. When large amounts of money, inmates meet for one or two hours per week under the or the deprivation of human freedom or even hu- guidance of a lay group leader. Some leaders are adpersonnel, caseworkers, teachers, guards or man life, are at stake, one can expect the method- ministrative clerical and technical staff workers, (p. SQ) ological logic or internal validity of a research These "group leaders" were supervised by perstudy to be dissected as meticulously as it would sons who "in most cases" held BA degrees. Stable be by the most anal of journal editors. Neither the design, nor the choice of independent and de- groups were those that had a change of group pendent variables, nor the method of analysis is leader less than once a month. In observing the counseling sessions, Kassebaum et al, noted "a immune from attack. Expert testimony on the prediction of violent tendency for staff members to permit periods of behavior, for example, has been attacked on the silence up to the length of the entire session begrounds that it has relied on studies that used cause of their misinterpretation of nondirective counseling" (p. 123). arrest and mental hospital records as measures of Finally, the researchers interviewed both prisonthe occurrence of violent behavior. In virtually ers and parolees and found that every case, such studies have been criticized by the opposing side for failing to make estimates for the most consistently expressed view of group counseling violent behavior that is not reported to the police was that its value was chiefly in satisfying the Adult Authority at parole hearings. Like class attendance in or, if reported, does not result in an arrest or hos- some universities, inmates felt that a participation in pitalization. It is indeed the case that of every group counseling might hot be a major factor in getting three violent crimes that occur in the United paroled, but a lack of participation was Hkely to be regarded negatively by the Adult Authority. (Kassebaum States, approximately two are reported to the po- etal., 1971, pp. 31-32) lice and, of these, only one is "cleared" by an The point of Quay's (1977) reanalysis is to ask, arrest (Monahan, in press-a). Thus one could What reasonable person could have expected recidmake the argument that predictions are quite ivism, or anything else, to have been reduced as good: Researchers have simply misclassified violent a result of one involuntary hour a week being people as safe. That is, if we could identify those spent in silence in the presence of an untrained persons who are subsequently violent but not secretary? Recall that this study is generally "caught," we might see that our predictions were viewed as among the best research ever done on much better than they have been thought to be. prison treatment and figures heavily in the conTo accept this argument would require the asclusion that "nothing works." While in this case sumption that the one out of three people prethe methodological validity of the research may dicted to be violent who is later arrested for a have been acceptable (e.g., randomization was violent crime is not the same person who also achieved), the weakness of the intervention makes committed the two unreported or unsolved crimes. its overall internal validity suspect. There are data, however, to support the opposite The point here is that the internal validity of conclusion, that the people arrested for a violent psychological research cannot be taken for granted crime are indeed likely to be the people who are _ in the courtroom. There have been enough uninvolved in the unreported and unsolved incidents replicated findings, methodological critiques, and (Shinnar & Shinnar, 1975). impassioned rejoinders published in psychological Psychologists have also testified on research conjournals to make judges and juries justifiably cluding that involuntary psychological treatment skeptical of taking an expert's word for it that the in prisons produces no measurable reduction in received wisdom of researchers is beyond contencriminal recidivism (e.g., Greenberg, 1977; Martion. The would-be expert witness should be well tinson, 1974). Yet in a penetrating critique of prepared to defend the methods by which his or what is generally regarded as the best of the her conclusions were reached. prison-rehabilitation studies, which found group counseling to be utterly devoid of rehabilitative THE GENERALIZABILITY OF RESEARCH FINDINGS merit (Kassebaum, Ward, & Wilner, 1971), Quay (1977) has highlighted frequently overlooked pas- Equally important as the internal validity of the sages from the report. For example, Kassebaum research being presented is its external validity or AMERICAN PSYCHOLOGIST • MARCH 1980 • 277

relevance to the kinds of issues being raised at trial (Campbell & Stanley, 1966). This concern is framed most starkly when the research being described used undergraduates as subjects (as in much of the research cited in the Ballew case) and the defendant or respondent in the case at issue happens not to be a college sophomore. It is also raised whenever the subjects of the research studies, or the situations being studied, differ in substantial ways from the person or persons on trial, or from their situations. For example, as one juror we have cited mentioned, the eyewitness 'in a trial may have been a highly stressed crime victim, whereas the subjects of the research were unstressed observers in a university laboratory. Or the person on trial may be a mental patient, and the subjects of the research may be mental patients but from different kinds of hospitals. Laboratory simulations differ from actual legal situations in many ways. Typically, the experiments are conducted in college classrooms, while the real-world setting is a courthouse. Typically, the experiments use written or tape-recorded trial summaries that last, say, 10 minutes, whereas in the real world, actual legal trials vary in their duration, with the average being 3 days. In experiments, the simulated jurors often do not deliberate to a verdict, and when they do, they spend an average of 20 minutes at it. In the real world, jurors always deliberate to a verdict, spending an average of 120 minutes (Saks & Hastie, 1978). Thus there are many differences between the studies and the settings to which they are intended to generalize. The opposing side in a legal proceeding will surely argue that the studies are irrelevant. Some differences between the studies and the case at bar can always be found. In a trial in the state of Washington, one of us (J. M.) described several studies on the accuracy of clinical predictions of violent behavior, which were done in various parts of the United States. The crossexamination began with this question: "So, Doctor, none of those studies was done on our psychiatrists and psychologists here in Washington, were they?" Unlike the physical sciences, in which, for example, all bodies falling in vacuums accelerate at the rate of 32 feet per second per second, psychological research invariably has limits on its generalizability. All eyewitnesses, or all predictions, or all treatments are not the same. To question the generalizability of a research finding is not only 278 • MARCH 1980 • AMERICAN PSYCHOLOGIST

a legitimate legal maneuver but is commonplace in scientific inquiry in general and psychology in particular. The general body of knowledge is built through the accumulation and synthesis of evidence obtained under particular conditions. Even so, the external validity of laboratory findings, particularly with student subjects, has been a matter of professional, as well as legal, contention for some time (Konecni, Mulcahy, & Ebbeson, 1980; Willems & Rausch, 1969). The crosssituational consistency of psychological research findings gathered by whatever methodology is an open question (Mischel, 1968). To be sure, this admission does not invalidate the introduction of research findings at trial. Even though external validity cannot be established within the confines of a single study, there are steps that can be taken to increase our confidence that an experimental result will generalize to a new, real-world condition. The first is to increase the diversity of experimental populations and contexts; if a particular result holds across different research settings, this is an indication of the likelihood that the result will generalize to a new setting. A second approach to the problem of the inferential leap from experimental to real-world conditions is to extend a well-developed scientific theory to the real-world conditions. If a particular research result is accounted for by a specific psychological theory, then the generality of that theory is an indirect indication of the generalizability of the research result. For example, if an experimental result relating stress and memory agrees closely with the predictions of a general theory on the cognitive effects of stress, and if this theory is useful in predicting performance in a variety of circumstances, then we can have increased confidence in the generality of the results of the experiment. Despite the drawbacks of using experimental simulations as strategy for studying real-world behavior, these simulations have numerous advantages. After a careful consideration of these strengths, Bray and Kerr (Note 1) have enumerated a variety of well-founded reasons for the use of simulations and have concluded that they play a useful role in the study of nonresearch settings because of their many practical and methodological advantages. After weighing the many sides of the generalizability issue, we are led to the resolution that the •relevance oj the general research findings to the

specific case at trial is a separate issue that should be addressed by the expert. Whether the expert raises the issue of generalizability during direct examination or waits to have it raised by the opposition during cross-examination may be considered a matter of trial strategy. If so, it is our experience that the expert's raising and addressing the issue of his or her own accord is the tactic of choice. It "immunizes" the jury to allegations of irrelevance. However, if the expert harbors "reasonable doubt" about the generalizability of the findings presented to the case at bar, it may be an ethical obligation to inform the jury of the question and let them decide the extent to which external validity has been achieved. PRESENTING BOTH SIDES

One objection to social science data is that because social scientists often come to different conclusions about the same questions, social science evidence is unlikely to be of any value to a jury. But,-as Lochner (1973) has pointed out, social scientists investigating the same question make different assumptions and, gather data differently. Inconsistent conclusions may only indicate procedural variations rather than intellectual chaos in the field. Nevertheless, the issue for an expert witness is the extent to which there is a duty to present both sides of a case when the case has two reasonable sides. If, in a trial concerning prison conditions, the expert is testifying that crowding in general produces deleterious effects, is there a duty to report on the specific experimental studies that show crowding does not produce harmful effects? There are two ways to answer this question. One is to take the position that evidence unsupportive of one's general opinion "has no function in the adversary game" (Wolfgang, 1974, p. 246). Proponents of this view believe that unsupportive evidence may be useful in a scientific article but that it has no proper place in argument before a court. In court, there is selectivity unlike that which exists in science. Rivlin (1973) went so far as to suggest that we acknowledge the development of a "forensic social science" rather than pretend "to be part of the tradition of balanced, objective social science in which the scholar hides (or claims to hide) his personal biases, and attempts to present all the evidence on both sides of a set of questions so that the.reader may judge for himself" (p. 61). Rivlin would have the psychol-

ogist take on the task of preparing a position for or against a particular proposition. The position would be clearly stated, and the evidence that supports one side of the argument would be brought together. The job of critiquing the case that has been presented and of detailing the counterevidence would be left to a scholar working for the opposition. There is a second way to deal with this issue, It is to note that the oath taken by all witnesses requires that the witness tell "the truth, the whole truth, and nothing but the truth." By selectively leaving out studies that are crucial to some particular question, the psychologist is failing to perform according to the oath. The "whole truth" is not being told. It is of interest to note that the Federal Rules of Evidence (1975) do not require an expert to disclose facts or data that underlie the opinion or inference being offered; rather, they leave this matter to the discretion of the judge. Rule 70S is specific on this point: The expert may testify in terms of opinion or inference and give his reasons therefore without prior disclosure of underlying facts or data, unless the court requires otherwise. The expert may in any event be required to disclose the underlying facts or data on cross-examination, (p. 84)

This rule allows counsel to make disclosure of the underlying facts or data as a preliminary to the giving of an expert opinion, if he or she chooses, but the instances in which this is required are reduced. Some have argued that leaving it to the cross-examiner to bring out the other side of the coin is essentially unfair; however, legal commentary on Rule 70S disagrees. It is assumed that the cross-examiner has the advance knowledge that is essential for effective cross-examination, usually through the traditional procedures of discovery, This safeguard is reinforced by the discretionary power of the judge to require preliminary disclosure of the findings, the underlying data, and even the identity of the experts. In any event, an opposing attorney is always free to ask this question: "Do you know of any studies that show the opposite result?" Any resolution of this issue will, of course, depend on the quality of the evidence on both sides of the case. If the expert is testifying that overcrowding produces psychological harm and 95% of all investigations support this conclusion, there may be no duty to confuse the jury and prolong the testimony with a discussion of the other 5%. AMERICAN PSYCHOLOGIST • MARCH 1980 • 279

However, if good, solid evidence exists to support the other position, the expert may have an ethical, even if not a legal, obligation to discuss it. In our opinion, the expert who has taken an oath to tell the "whole truth" may have an obligation to bring up research evidence that runs counter to the thrust of his or her testimony. THE PROBABILISTIC NATURE OF RESEARCH EVIDENCE

Suppose one were convinced that the psychological research to be presented in court had both internal and external validity, and that the experts in the field were agreed on the research findings. For example, suppose that a highly competent experiment had been performed on cross-racial eyewitness identification, using actual crime victims as subjects, and that the study clearly showed that people are less good at identifying members of a different race than members of their own race. Suppose further that this experiment conformed to all previous research on the subject, so that the experts in the field agreed that this case had but one side (in other words, no expert would be willing to argue that same-race identifications are harder than, or as difficult as, opposite-race ones). A dilemma would still be present: Because it is highly unusual to conduct a study in which every single participant is identically influenced by the factors of interest, it is likely that fewer than 100% of the participants actually showed the cross-racial identification effect. Suppose a highly significant 92% of the participants showed the cross-racial effect, while 8% showed either no effect or an effect in the opposite direction. Testimony in a criminal case to this effect might prompt a prosecutor (and, in actual cases, has done so) to argue that the psychologist's testimony is irrelevant to the case because although the eyewitness might be a member of the majority class containing 92% of the individuals, he or she might just as well be a member of the minority class containing &%, The witness, in other words, might be one of the rare people who do not suffer from a deficit in ability to make cross-racial identifications. Thus, the argument would go, the testimony may have nothing to do with the witness or defendant at bar. The response, it seems to us, is straightforward. The expert must agree that one cannot be sure whether any particular witness is influenced by this factor or not. The expert can only argue that a certain percentage of people are affected in a 280 • MARCH 1980 • AMERICAN PSYCHOLOGIST

particular way. The jury is then free, as it should be, to use whatever other information it has available to make the final decision about whether the particular witness or defendant is to be classified with the majority or the minority on this particular characteristic. Put another way, probabilistic evidence can be presented as such, with its application to a particular person lejt for the jury to decide. It is of further interest that several legal scholars (e.g., Tribe, 1971) have contended that the explicit quantification of forensic probabilities would be undesirable on the grounds that such a practice would undermine the finality with which legal proceedings are supposed to settle contested issues. A fear is that if triers of fact were encouraged to think in explicit mathematical terms, they would give undue weight to statistical evidence, conveniently quantified for them by experts, rather than attempt the difficult task of integrating such evidence with other relevant considerations to arrive at a fully informed subjective probability. This argument, of course, depends on certain assumptions about human intellect. It may be that certain types of evidence that bring the probabilistic element of inference to explicit attention in a quantified way will usurp the province of the jury. This undoubtedly was so in the case of People v. Collins,7 in which eyewitness testimony claimed that the assailant was a young woman with a blond ponytail who fled from the scene of the crime in a yellow automobile driven by a black male with a moustache and beard. The prosecution called a college mathematics instructor to establish, with probabilities, that if the robbery was indeed committed by a Caucasian woman with a blond ponytail, accompanied by a Negro having a beard and mustache and driving a yellow car, there was an overwhelming probability that the accused couple were guilty because they matched this detailed description. Aside from noting errors in the mathematical analysis, the California Supreme Court perceived a danger in this testimony and warned that mathematics, a veritable sorcerer in our computerized society, while assisting the jury in the search for truth, must not be allowed to cast a spell over it. The expert testimony we have been discussing is different: The expert is presenting an opinion

•People v. Collins, 68 Cal. 2d 319, 438 P. 2d 33, 66 Cal. Rptr. 497 (1968).

based on a commonly observed experimental result; it is up to the trier of fact to accept or reject the opinion, the result, and/or their application to the case at bar. The expert may present some specifics about the data underlying the opinion; to assume that these will overwhelm the trier of fact is to have very little faith in what is perhaps the key element in our legal system. VALUES As social scientists, like all scientists, "we are taught to value unswerving pursuit of truth without regard for the congeniality of that truth and the full disclosure of our findings without regard for the possible ill effects of that disclosure" (Ellsworth & Ross, 1976, p. 164). This attitude often finds itself at odds with the standard adversary methods of legal persuasion (Morse, 1978). Any courtroom testimony that a psychologist might offer must always be presented as part of one side's argument. Occasionally a psychologist may be asked to testify in pursuit of a result about which the psychologist may have serious personal qualms. For example, one of us (E. L.) has often been asked to testify on behalf of the defendant in rape cases. Knowing how difficult it is to obtain convictions in rape cases, and knowing that the expert testimony is likely to help the defendant by reducing his chances of conviction, E, L. is faced with a personal dilemma. In this particular example, the personal dilemma was resolved when she reasoned that although obtaining convictions of actual rapists is of crucial importance, avoiding the mistaken conviction of innocent people is equally important. If the expert testimony serves to make the jurors better able to evaluate the eyewitness testimony so that both of these goals are served, then, in her opinion, justice is improved. In contrast, one of us (J. M.) has had no doubt that the defendant was guilty as charged in each of the cases in which he has testified. This is not unusual. Even experienced defense attorneys estimate that approximately 90% of their clients are factually and legally guilty (Heumann, 1978). The psychologist would be less than candid not to admit that being "on the side of" a person guilty of heinous crimes is personally distasteful. J. M. recalls feeling physically ill upon first meeting his "client" who had brutally stabbed to death a 12year-old baby-sitter after having raped her. He

agreed to testify in the case in an effort to invalidate the death penalty, which, in this state, was predicated on an accurate prediction of violent behavior—a prediction that he believed to be scientifically impossible to make. Since he was morally opposed to the death penalty, the testimony presented no moral problem. If the research findings were otherwise, however—if some new technique allowed violent behavior to be more accurately predicted—he would have refused to testify, believing immoral ends to be no less immoral when pursued with scientific means. This would not, in his opinion, be a personal usurpation of the role more properly played by a judge or jury in a democratic society. Rather, it would be a principled abstention from participating in a morally tainted procedure, analogous to the conscientious objection of Quakers to the military draft or of Catholic physicians to performing abortions. While the issues of personal values in science and in law are exceedingly complex ("Report of the Task Force," 1978), we would offer, at a minimum, that the expert witness should be candid about personal values, so that the jury may evaluate any biasing effects on his or her testimony.

Conclusions At the root of many of the dilemmas of psychologists in the courtroom is a clash of traditions. Law is an adversary process. The truth is believed to emerge from a brawl in which each participant pulls no punches and gives no quarter. The judge is the referee who watches for rabbit punches and keeps things above the belt. The jury does the scoring. The best man or woman—the one with the most truth—wins, it is hoped. Psychology, on the other hand, likes to think it is above all this. The truth is like a mountain, there for all to see. Research is the guide to help us avoid the slippery slopes and the passages that lead nowhere. Benefiting from the work of previous climbers, we reach the summit and all rejoice. Nobody loses when the truth is scaled. It is the clash of these idealized perspectives— mountain climbers in the ring—that gives us difficulty. Yet as long as law is based at least in part on facts and as long as facts are what psychologists deal with, some mixing of metaphors is unavoidable and, indeed, desirable. The only thing worse than the law's uncritical acceptance of psychology would be psychologists' uncritical rejection of the law as an arena worthy of their parAMERICAN PSYCHOLOGIST • MARCH 1980 • 281

ticipation. Horowitz and Katz (1975, p. 168) have put it well: The best argument for the widest possible implementation of social scientists in ... judicial branches of government at national and subnational levels is the break they provide on idiosyncratic decision making that results from too narrow a consideration of evidence and contexts. The worst reason for more widely using social science talent is to avoid or bypass the democratic process—a situation in which the role of expertise comes to displace the will of the people on major issues. Between these two poles the tightrope must be walked. REFERENCE NOTE 1. Bray, R. M., & Kerr, N. L. Use of the simulation method in the study of jury behavior: Some methodological considerations. Unpublished manuscript, University of California, San Diego, 1979. REFERENCES Addison, B. M. Expert testimony on eyewitness perception. Dickinson Law Review, 1978, 82, 465-485. Birnbaum, I. M., Parker, E. S., Hartley, J. T., & Noble, E. P. Alcohol and memory: Retrieval processes. Journal of Verbal Learning and Verbal Behavior, 1978, 17, 325-335. Brodsky, S. The mental health professional on the witness stand: A survival guide. In B. Sales (Ed.), Psychology in the legal process. New York: Spectrum', 1977. Brodsky, S., & Robey, A. On becoming an expert witness: Issues of orientation and effectiveness. Professional Psychology, 1973, 3, 173-176. Campbell, D., & Stanley, J. Experimental and Quasiexperimental design for research. Chicago: Rand McNally, 1966. Clark, K. Effect of prejudice and discrimination on personality development. In Midcentury White House Conference on Children and Youth. Washington, D.C.: U.S. Government Printing Office, 1950. Cocozza, J., & Steadman, H. The failure of psychiatric prediction of dangerousness: Clear and convincing evidence. Rutgers Law Review, 1976, 29, 1048-1101. Easterbrook, J. A. The effect of emotion on cue utilization and the organization of behavior. Psychological Review, 1959, 66, 183-201. Ebbinghaus, H. E. Memory: A contribution to experimental psychology. New York: Dover, 1964. (Originally published 1885; translated 1913.) Ellsworth, P. C., & Ross, L. Public opinion and judicial decision making: An example from research on capital punishment. In H. Bedau & C. M. Pierce (Eds.), Capital punishment in the United States. New York: AMS Press, 1976. Federal rules of evidence for United States courts and magistrates. St. Paul, Minn.: West Publishing Co., 1975. Greenberg, D. The correctional effects of corrections: A survey of evaluations. In D. Greenberg (Ed.), Corrections and punishment. Beverly Hills, Calif.: Sage, 1977. Heumann, M. Plea bargaining: The experience of prosecutors, judges, and defense attorneys. Chicago: University of Chicago Press, 1978. Horowitz, J., & Katz, J. Social science and public policy in the United States. New York: Praeger, 1975.

282 • MARCH 1980 • AMERICAN PSYCHOLOGIST

Kahle, L., & Sales, B. Due process of law and the attitudes of professionals toward involuntary civil commitment. In P. Lipsitt & B. Sales (Eds.), New directions in psychological research. New York: Van Nostrand Reinhold, 1080. Kassebaum, G., Ward, D., & Wilner, D. Prison treatment and parole survival. New York: Wiley, 1971. Katz, L. S., & Reid, J. F. Expert testimony on the fallibility of eyewitness identification. Criminal Justice Journal, 1977, 1, 177-206. Kluger, R. Simple justice. New York: Knopf, 1976. Konecni, V., Mulcahy, E., & Ebbesen, E. Prison or mental hospital: Factors affecting the processing of persons suspected of being "mentally disordered sex offenders." In P. Lipsitt & B. Sales (Eds.), New directions in psycholegal research. New York: Van Nostrand Reinhold, 1980. Lochner, P. R., Jr. Some limits on the application of social science research in the legal process. Law and the Social Order, 1973, 815-848. Loftus, E. F. Unconscious transference in eyewitness identification. Law and Psychology Review, 1976, 2, 93-98. Loftus, E. F. Eyewitness testimony. Cambridge, Mass.: Harvard University Press, 1979. Loftus, E, F. Impact of expert psychological testimony on the unreliability of eyewitness identification. Journal of Applied Psychology, 1980, 65, 9-15. Lower, J. Psychologists as expert witnesses. Law and Psychology Review, 1978, 4, 127-139. Malpass, R. S., & Kravitz, J. Recognition for faces of own and other race. Journal of Personality and Social Psychology, 1969, 13, 330-334. Martinson, R. What works? Question and answers about prison reform. The Public Interest, 1974, 35, 22-54. Mischel, W. Personality and assessment. New York: Wiley, 1968. Monahan, J. The prediction of violent criminal behavior; A methodological critique and prospectus, In P. Blumstein, J. Cohen, & D. Nagin (Eds.), Deterrence and incapacitation. Washington, D.C.: National Academy of Sciences, 1978. Monahan, J. The clinical prediction of violent behavior. Washington, D.C.: U.S. Government Printing Office, in press, (a) Monahan, J. The role of research in changing the legal system. In R. Price & P. Politser (Eds.), Evaluation and action in the social environment. New York: Academic Press, in press, (b) Morse, S. Crazy behavior, morals, and science: An analysis of mental health law. Southern California Law Review, 1978, S I , 527-654. Nagel, S., & Neef, M. Deductive modeling to determine an optimum jury size and fraction • required to convict. Washington University Law Quarterly, 1975, 4, 933-Q78. Parker, E. S., Birnbaum, I. M., & Noble, E. P. Alcohol and memory: Storage and state dependency. Journal of Verbal Learning and Verbal Behavior, 1976, IS, 691702. Quay, H. The three faces of evaluation: What can be expected to work. Criminal Justice and Behavior, 1977, 4, 341-354. Report of the Task Force on the Role of Psychology in the Criminal Justice System. American Psychologist, 1978, 33, 1099-1113.

Rivlin, A. Forensic social science. Harvard Educational Review, 1973, 43, 61-75. Saks, M. J., & Hastie, R. Social psychology in court. New York: Van Nostrand Reinhold, 1978. Shah, S. Dangerousnass: A paradigm for exploring some issues in law and psychology. American Psychologist, 1978, 33, 224-238. Shinnar, S., & Shinnar, R. The effects of the criminal justice system on the control of crime: A quantitative approach. Law and Society Review, 197S, 9, 581-611. Sobel, N. Eyewitness identification. New Yo.rk: Boardman, 1972. (Supplement added, 1979) Stone, A. Mental health and the law: A system in transition. Washington, D.C.: U.S. Government Printing Office, 1975. Tribe, L. H. Trial by mathematics: Precision and ritual in the legal process. Harvard Law Review, 1971, 84, 1328-1393. Underwood, B. Law and the crystal ball: Predicting be-

havior with statistical inference and individualized judgment. Yale Law Journal, 1979, 88, 1408-144S. Van Matre, J. G., & Clark, W. N. The statistician as expert witness. American Statistician, 1976, 30, 2-5. Willems, E., & Rausch, H. (Eds.). Naturalistic viewpoints in psychological research. New York: Holt,' Rinehart & Winston, 1969. Wolfgang, M. E. The social scientist in court. Journal oj Criminal Law and Criminology, 1974, 65, 239-247. Woocher, F. D. Did your eyes deceive you? Expert psychological testimony on the unreliability of eyewitness identification. Stanford Law Review, 1977, 29, 9691030. Woody, R. H. Psychologists in child custody. In B. D. Sales (Ed.), Psychology in the legal process. New York: Spectrum, 1977. Ziskin, J. Coping with psychiatric and psychological testimony (2nd ed.). Beverly Hills, Calif.: Law and Psychology Press, 1975.

AMERICAN PSYCHOLOGIST • MARCH 1980 • 283