## MONDAY, JUNE 28, 2010

### On The Rate of Wrongful Conviction: Chapter 0.027

**we have a lot of people incarcerated, around 2.5 million. That means a wrongful conviction rate of only 1%, if applicable to all those incarcerated, means we have wrongfully imprisoned 25,000 people. A wrongful conviction rate of 10% means we have wrongfully imprisoned a quarter of a million people.**

THE SCALIA NUMBER

In the Winter 2005 Journal of Criminal Law and Criminology, a group led by Samuel Gross, a law professor at the University of Michigan, published an exhaustive study of exonerations around the country from 1989 to 2003 in cases ranging from robbery to capital murder. They were able to document only 340 inmates who were eventually freed. (They counted cases where defendants were retried after an initial conviction and subsequently found not guilty as “exonerations.”) Yet, despite the relatively small number his research came up with, Mr. Gross says he is certain that far more innocents languish undiscovered in prison.

So, let’s give the professor the benefit of the doubt: let’s assume that he understated the number of innocents by roughly a factor of 10, that instead of 340 there were 4,000 people in prison who weren’t involved in the crime in any way. During that same 15 years, there were more than 15 million felony convictions across the country. That would make the error rate .027 percent — or, to put it another way, a success rate of 99.973 percent.

It should be noted at the outset that the dissent does not discuss a single case — not one — in which it is clear that a person was executed for a crime he did not commit. If such an event had occurred in recent years, we would not have to hunt for it; the innocent’s name would be shouted from the rooftops by the abolition lobby.

Of course, even with its distorted concept of what constitutes “exoneration,” the claims of the Gross article are fairly modest: Between 1989 and 2003, the authors identify 340 “exonerations” nationwide — not just for capital cases, mind you, nor even just for murder convictions, but for various felonies. Joshua Marquis, a district attorney in Oregon, recently responded to this article as follows:

“[L]et’s give the professor the benefit of the doubt: let’s assume that he understated the number of innocents by roughly a factor of 10, that instead of 340 there were 4,000 people in prison who weren’t involved in the crime in any way. During that same 15 years, there were more than 15 million felony convictions across the country. That would make the error rate .027 percent—or, to put it another way, a success rate of 99.973 percent.”

The proof of the pudding, of course, is that as far as anyone can determine (and many are looking), none of cases included in the .027% error rate for American verdicts involved a capital defendant erroneously executed.

Of course even in identifying exonerees, the dissent is willing to accept anybody’s say-so. It engages in no critical review, but merely parrots articles or reports that support its attack on the American criminal justice system.

Rape and murder cases, on the other hand, constitute less than two percent of all felony convictions but represent ninety-six percent of all known exonerations. (See Samuel Gross’ rebuttal to Scalia’s opinion “Souter Passant, Scalia Rampant: Combat in the Marsh.”)

If Joshua Marquis were to correct his calculation from

to

as I believe he should, then his estimate would rise to 1.1%.

Marquis’ corrected estimate would still, however, depend entirely on the arbitrary multiplier he selected in the numerator. He simply assumed the Gross study had identified 10% of all the people wrongfully convicted. Had he assumed instead that the Gross study had identified all 100% of those wrongfully convicted, his corrected estimate would be 0.11%.

On the other hand, had Joshua Marquis assumed the Gross study identified only 1% of the factually innocent, a number which seems as reasonable to me as 10%, then his corrected estimate would indicate we wrongfully convict, by trial or plea bargain, 11% of all those people we charge with felonies.

Chapter 1.4: The Poveda Number

Chapter 1.9: The Judge Number

Chapter 2.3: The Gross Number

Chapter 3.3: The Risinger Number

Chapter 5.4: The Defense Number

Chapter 9.5: The Inmate Number

Chapter 10.1: A Skeptical Juror Number

Chapter 11.1: A Skeptical Juror Number

Chapter 11.4: The Common Man Number

The Spencer Number

I was inspired to pursue this quantification effort by a paper entitled “Estimating the Accuracy of Jury Verdicts.” It was written in April of 2006 and modified one year later by Bruce Spencer.

Bruce D. Spencer is a Professor of Statistics at Northwestern University. That’s interesting in that Northwestern University is the home of David Protess and the Medill Innocence Project. Those folks have actually helped free innocent people from wrongful imprisonment, and for that I tip my hat. Their name may ring a bell to those of you who have read my posts on the Hank Skinner case.

1. A person who is considered intellectually gifted in the field of academics. “Egghead” is usually used as college-speak to describe a brainiac.

2. A person’s whose head is shaped like an egg. Most people however, will use this word interchangeably as a pun. It has also been known that people whose heads are shaped like an egg are usually large at the top, which explains the larger brain-size.

Next, we’ll look at just two contiguous sentences from Spencer’s paper:

In Section III, an estimator of jury accuracy is developed that has three components of error, survey error from estimating the agreement rate, specification error arising because differential accuracy between judge and jury is not observed and the dependence between judge and jury verdicts is not known, and identification error arising because we cannot distinguish correct agreement from incorrect agreement. The specification error will be one sided, leading to overestimates of jury accuracy, provided that two conditions hold: (i) errors in the judge’s and jury’s verdicts for a case are either statistically independent or positively dependent, and (ii) the judges’ verdicts are no less accurate on average than the juries’, even though for individual cases the judge’s verdict may be incorrect when the jury’s verdict is correct.

There you go.

The jury verdict was estimated to be accurate in no more than 87% of the NCSC cases (which, however, should not be regarded as a representative sample with respect to jury accuracy).

Caveat: the NCSC cases were not chosen with equal probabilities as a random sample, and the estimates of accuracy should not be generalized to the full caseload in the four jurisdictions let alone to other jurisdictions.

The analysis suggests, subject to limits of sample size and possible modeling error, …

The unequal sampling rates imply that the results for the NCSC sample cases should be weighted if they are to generalize to the full caseload in the four jurisdictions. No such weighting is employed in the present analysis, and the statistical inferences do not extend outside the cases in the NCSC study.

In light of these limitations, the empirical estimates from the data analysis must be interpreted with great caution and in no event should be generalized beyond the NCSC study.

The estimates are no basis for action other than future studies.

Here’s where Spencer started.

Similarly, you can calculate the rate of “wrongful acquittal” from the table. I put the term in parenthesis because it is not necessarily an error to acquit a person who is actually guilty. If the State did not prove its case beyond a reasonable doubt, then the error would be in voting guilty. When I use the term “wrongful acquittal” with the quotes, I am indicating only that the person was acquitted despite being factually guilty, not that the jury necessarily made an error.

The “wrongful acquittal” rate for the jury, based on Spencer’s analysis of the NCSC judge-jury agreement data, is 30.5%. ( .095 / .311 = .305 ) Whereas the judge is almost twice as likely to convict an innocent person, the judge is only one-third as likely to acquit a guilty person. ( .022 / .187 = .118 = 11.8% )

There are so many numbers floating around, and so many numbers that could be made to float, that we need a way to simplify everything. That’s why each chapter in this monograph is defined by a single number, the rate of wrongful conviction for jury and bench trials combined. That rate can then be multiplied by the number of people incarcerated to determine the number of people wrongfully incarcerated.

To arrive at that single number, we need to account for the number of jury trials compared to the number of bench trials. I have summarized those calculations in the three tables below. One table is for the juries, one for the judges, and one for jury and bench trials combined. Each table contains three estimates. Spencer actually provided estimates for a variety calculation assumptions, but recommended only two be considered valid. Those are labeled 3a and 3b. I’ve also included the results from my own judge-jury agreement analysis, which I present in Chapter 11.1.

There’s a whole lot of data there, so you’ll have to click on the image to enlarge it and view it. Don’t be intimidated. I’ve marked up the figure to allow you to quickly home in on what’s important.

The numbers in **bold** are the basic results from the judge-jury analyses.

The numbers * underlined* (near the upper left) are state court conviction data for 2004 from the

*Sourcebook for Criminal Justice Statistics Online*. They are, of course, identical for each of the three analyses within each table. I’m merely seeing what would happen if I applied the results from the judge-jury agreement analyses to real world data.

The other numbers are merely Excel level calculations. The wrongful conviction and wrongful acquittal rates are inside the heavily outlined boxes in the bottom table. Spencer has estimated two different wrongful conviction rates. I took the average of his two results to use as the chapter number.

I’m ecstatic to see that my calculated wrongful conviction rate matches that of Professor Bruce D. Spencer to the first decimal point, assuming I choose to accept his second analysis as correct. The match is interesting since we used two different sets of input data, and two dramatically different approaches for defining and solving our equations. I’m quite frankly stunned.

Spencer and I don’t agree near as well when it comes to the rate of wrongful acquittal. My calculated rate is nearly 50% higher than his. However, since acquittals are far fewer than are convictions, in absolute numbers, both Professor Spencer and I are once again in near agreement. (Notice how I’ve elevated him once again from Spencer to Professor Spencer now that I see he agrees with me.)

The number “n” along the right hand side of the tables is the ratio of guilty men set free to innocent men convicted. In all cases, it’s close to unity. In no case is it close to ten. Ten is the number made famous by the long dead English jurist William Blackstone who proclaimed that it is “better that ten guilty persons escape than that one innocent suffer.”

My choice of using “n” as the symbol for that value comes from a clever and fascinating article by Alexander Volokh entitled “n Guilty Men.” Volokh therein presents an amazing and comprehensive history of various pronouncements of what a proper value of “n” should be, ranging from a high of infinity to a low of 0.1.

Any suggestion, however, that we can better protect the innocent among us (or ourselves for that matter) only by allowing more guilty people to escape is based on a false premise. There is nothing in the mathematics that says it must be so.

We could, if we wished, improve our law enforcement system to identify and convict a higher percentage of those who are in fact guilty and not even bring to trial those who are in fact innocent. An state induced wrongful eyewitness identification, for example, can allow both the escape of a guilty person and the conviction of an innocent. Application of improved arson science could spare thousands of innocents and let not a single guilty person go free, since no crime may have been committed.

We need to learn the lesson so frequently reinforced upon those who attempt to excel at business: poor quality is extremely costly and can be deadly.

[Note to self. The closing paragraph really sucks. Need to fix it. Also, need to discuss actual innocence versus legal innocence.]

## Leave a Reply