In 1999, Sally Clark, a young British lawyer, was convicted of killing her two newborn babies over a period of two years and she received a life sentence. A pediatrician had testified for the prosecution that the probability that the two boys had died from the Sudden Infant Death Syndrome (SIDS) or “crib death” was about 1 over 73,000,000. This was the only real evidence of the crime.
But the probability estimate, which persuaded the jury, was defective. It assumed that the two deaths were statistically independent events, justifying the multiplication of their respective probabilities for both events to happen: 1/8543 × 1/8543 is approximately equal to 1/73,000,000. In reality, however, two SIDS deaths in the same family are not independent events: one such death increases by 10 the medical probability that a second one will happen. Moreover, a professor of mathematics at the University of Salford, Ray Hill, later calculated that the probability of two SIDS in the same family is between 4.5 and 9 times the probability of two infant siblings being murdered. (See Ray Hill, “Multiple Sudden Infant Deaths—Coincidence or Beyond Coincidence?” [Paediatric and Perinatal Epidemiology 2004, 18, 320–326].)
Ms. Clark’s conviction was overturned on appeal after she had spent three years in prison. Statistician David Hand notes that “she never recovered from her ordeal” and, in 2007, was found dead “from acute alcohol poisoning.” As Tim Harford, the Financial Times‘s “undercover economist,” put it, “she drank herself to death at the age of 42.” A very tragic and troubling story.
There have been other documented cases of murder verdicts resulting from similar ignorance of probabilities.
On probabilities and coincidences, I recommend Professor David Hand’s book The Improbability Principle: Why Coincidences, Miracles, and Rare Events Happen Every Day (Scientific American and Farrar, Straus and Giroux, 2014). I think this book is accessible to intelligent lay readers with no prior knowledge of probability theory. (Probability theory is the basis of statistical analysis.)
By discovering the laws of chance—the laws of chance—probability theory is one of the great achievements of the human mind. A testimony to the beauty of probability theory is the name that Mike Lynch had given to his superyacht which sank along the Italian coast last week, claiming his own life and the lives of many of his guests: “the Bayesian.” Thomas Bayes was an 18th-century statistician, who developed an important theorem of probability theory.
Tim Harford notes that “in 2010, the UK Court of Appeal ruled against the use of Bayes’ Theorem as a tool for evaluating how to put together a collage of evidence.” He adds that “a little bit of statistical education for the legal profession would go a long way.”
If you are charged with a crime, your liberty may depend on the understanding of probability theory by lawyers and judges. A necessary condition is that it be familiar to the educated public, the pool from which legal professionals come. But this is not generally the case. If it were, conspiracy theories would have another hurdle to overcome, besides the straight obstacle of rational choice. Politicians don’t know more.
A debate is going on in the United Kingdom on the public’s poor knowledge of science, mathematics, and statistics. The recent trial of British nurse Lucy Letby, who was condemned to several life sentences for allegedly murdering many patients, after a trial with weak factual evidence and again misleading statistical estimates. Many statisticians have voiced strong criticisms. The Economist writes about former prime minister Boris Johnson (“The Trial of Lucy Letby Has Shocked British Statisticians,” August 22, 2024):
Mr Johnson is paradigmatic of what has gone wrong. He is not—despite what his actions often imply—a stupid man and certainly not, after Eton and Oxford, an ill-educated one. His education was etiolated; it was not ineffectual. He could read Archimedes in the original; he could not begin to understand Archimedes’s maths. He is the product of what [the late physicist and novelist C.P.] Snow called Britain’s “fanatical belief in educational specialisation”. And that belief, says David Willetts, a former universities minister, is “as acute as ever”.
I would argue that the problem is as serious regarding people who know only science and are illiterate in the humanities and in economics. This is especially and emphatically true for people who, be it in their official functions or with their votes, intend to coercively intervene in the lives of other people.
I don’t think that a court should ever condemn a person only or mainly on the basis of probabilities: there should be some weighty factual and testimonial evidence. But the lesson of the Sally Clark case and others is that if probabilities are invoked, they should be calculated correctly. The cure for “bad statistics,” Harford argues, “isn’t ‘no statistics’—it’s using statistical tools properly.”
This problem is related to the presumption of innocence and the requirement that guilt be proven by the prosecution “beyond a reasonable doubt.” We largely owe our liberty, however imperfect it is, to these legal principles in the Western tradition. But whatever is the allowed level of reasonable doubt—which, in a free society, must correspond to a tiny probability of error—lawyers and judges need to understand statistical theory enough to get a sense of the probabilities involved.
******************************
READER COMMENTS
Craig
Aug 26 2024 at 12:57pm
Interesting commentary, as usual, Pierre, wanted to expound a bit that the case, while obviously a UK case, shows the importance of the ‘public trial’ element of the VI Amendment. It allows for an insightful retrospective on imperfect justice.
Pierre Lemieux
Aug 26 2024 at 1:58pm
Craig: I agree, of course, that the procedural protections of the 6th Amendment are also very important. However, note Leviathan’s talents in bypassing constitutional restrictions. One thing he (or she) resorted to in America was the use of plea bargaining. More than 90% of the criminal cases don’t come to trial. Throw the book at a defendant, and he will likely plead guilty to some crime. (Mike Lynch was a relatively rare exception, and you better be rich to be an exception.) The basic problem is that the book (the scope of legal prohibitions and obligations) has become much, much too heavy.
Mactoul
Aug 27 2024 at 12:11am
About Mike Lynch who is now dead (and on the same day as his co-defendant who got struck by a car), Mark Steyn comments:
Thomas L Hutcheson
Aug 26 2024 at 1:53pm
My suspicion is that “humanists” are more ignorant of science than scientists are of the humanities. Economist, of course, Radically Centrist economists, to be precise, 🙂 hit the happy medium.
Fun fact. In CP Snow’s day, it was the scientists who were leftist and the humanities conservatives. Today it is more the reverse.
Pierre Lemieux
Aug 27 2024 at 3:00pm
Thomas: Your observation about Snow’s days is interesting. I would argue though that, if we can for a moment ignore the radical centrist’s reference axis, scientists are (in general) as collectivist as humanities people (in general): they just want the collective choices to be made by different people against different victims.
Scott Sumner
Aug 26 2024 at 1:56pm
Excellent post. If Sally Clark had won the lottery, would the UK government have claimed that she must have cheated, as there was only a 1 in a million chance of her winning fairly?
Pierre Lemieux
Aug 26 2024 at 2:10pm
That’s a very good point, Scott. David Hand also raises similar points in The Improbability Principle. One of his best examples is the Bulgarian Lottery when, on September 10, 2009, the winning numbers–4, 15, 23, 24, 35, 42–were exactly the same as those of the draw of four days earlier! Improbable events and coincidences can happen by chance.
steve
Aug 26 2024 at 4:29pm
These kind of stats are how we usually catch bad doctors and how we have caught the “death angel” nurses. We had one in my network. However, we usually have other evidence also, but not always. It’s really difficult to decide what to do when someone consistently has unexpectedly bad outcomes for no apparent reason. My understanding in this case is that there were also behavioral issues and some questionable notes she had written.
Steve
Craig
Aug 26 2024 at 4:36pm
And therein Steve I would suggest lies the difference between ‘reasonable suspicion’ or ‘probable cause’ and ‘beyond a reasonable doubt’
Neven Sesardić
Aug 26 2024 at 5:00pm
The probability argument for Sally Clark’s guilt was much more sophisticated and was not “the only real evidence of the crime”.
https://philpapers.org/archive/SESSID-3.pdf
Pierre Lemieux
Aug 29 2024 at 11:04am
Thanks for your challenging article, Neven. I gave it a quick reading. Has there been any statistician’s rejoinder?
Neven Sesardić
Aug 30 2024 at 10:48am
No.
Jim Glass
Aug 27 2024 at 2:42am
I’ve long believed out school systems should drop requirements to take algebra, geometry, trig and the like, which most human beings will never use, and replace them with basic literacy-level, practical-use statistics and economics, which everybody alive uses many times every day.
Several years ago I took an online algebra course from a Prestigious Name Provider to see both if I remembered any of it and what these online courses are like. Halfway through one of the frustrated students asked, “what will any of this ever be good for?” to which the instructor answered: “Well, if you take advanced chemistry in the xy34tb reaction you’ll need algebra to …” That was the most practical use for algebra the Prestigious instructor could give for it. I swear.
OTOH, imagine if average voting high school grads understood the simple basics of stats and econ — fundamental saving and investing (to face down countless scams on the Internet and everywhere else), where prices come from (so controls are bad), from whence comes the literal money in one’s pocket, why when someone calls just when you think of them it isn’t a miracle, how much money they lose playing the state lottery (so they’d go to a casino instead to lose a lot less and enjoy a show), and when they see Trump stand and proclaim to his fans over and over “When I put a tariff on China, China pays all of it, not Americans. They tell you Americans pay it, but they’re lying!”, think, “Hey, he’s lying!”. (Same for politicians and rent controls and other political scams obvious to the even semi-literate). And they’d all even be better at their jobs as judges and jurors and lawyers — and doctors! for sure, their reported innumeracy is scary! — and all the rest.
The best book I’ve seen on this is Innumeracy: Mathematical Illiteracy and Its Consequences. As the wiki page about it says…
Fun and practical all the way through, the pure opposite of that algebra course. One of the very few books I’ve bought multiple times just for myself.
Richard W Fulmer
Aug 27 2024 at 10:37am
A basic understanding of algebra can make computer programming easier because both involve working with variables, performing operations, and manipulating data. Solving equations, dealing with functions, and working with abstract symbols are all skills directly applicable to both algebra and programming.
Jose Pablo
Aug 27 2024 at 4:47pm
Great comment Jim!
It is useless! We force the young to spend 12 years in school, an average of 7 hours per day, 5 days a week, around 40 weeks a year (around 16,800 man-hours, roughly the equivalent of 10 years of work) learning, for the most part, useless things.
I have lately come to the realization, talking with kids from “difficult” backgrounds, that this mostly protects middle and upper-class kids from the competition of poor kids in the job market.
Kids coming from less privileged neighborhoods stand very little chance of having the discipline, and the commitment and of being able to devote the resources (in the form of foregone wages, for instance) required to overcome the unnecessary barrier to accessing the job market that we have come to call “education”.
Even though some of them are very clever (street smart kind of clever) and will have no problem learning the ropes of any job that gives them the opportunity of “on the job” learning. Money managers, investment advisors, hedge or private equity funds managers, or lawyers, to name but a few of the jobs most easily accessible to these guys, to which the artificial barrier of k-12 education adds nothing of interest.
[But this is totally unrelated to the poor Sally Clark case, sorry]
nobody.really
Aug 27 2024 at 4:36am
The prison’s grainy video shows three prisoners gathering as a prison guard seems to be having some kind of medical problem at the edge of the prison yard. Two of the prisoners grab and bind the third prisoner. The unbound prisoners next attack and kill the guard, and then unbind the prisoner.
Prosecutors bring charges against each of the prisoners for murdering the guard. Each prisoner professes to have been the one bound prisoner. At trial, the prosecutor argues to convict on the basis that there’s a 67% chance that the accused is lying, and that’s sufficient to overcome reasonable doubt. What verdict? What if there had been 10 prisoners, so the odds were 90% that any accused prisoner was lying? What if there had been 100 prisoners, so the odds were 99% that any individual accused prisoner was lying? Can bare probability ever overcome reasonable doubt?
Now imagine that the prosecutor offers to drop charges against one of the prisoners, Joe, if he will testify against his fellow prisoners. Joe takes the deal and testifies to having been the one prisoner who did not take part in the murder–and, thus, all the other prisoners must have participated in the murder. The prosecutor argues to convict each accused prisoner on the basis that there’s only a small probability that the accused prisoner is telling the truth, and Joe has contradicted the accused prisoner’s claims. At trial, what verdict? Would we say that mere probability an insufficient basis to overcome reasonable doubt–but mere probability plus transparently self-serving testimony does provide a sufficient basis?
Jerry Melsky
Aug 27 2024 at 2:56pm
That raises the question of what exactly constitutes “reasonable doubt.” Is a 1 in 73,000,000 probability low enough to be “beyond reasonable doubt?” Is 1 in 7,300,000 low enough? (The latter being the probability of two SIDS deaths in a family where the first SIDS death increases the odds of the second one by a factor of 10.)
Mactoul
Aug 28 2024 at 3:17am
Perhaps the terms like “reasonable doubt” are more intelligible than any numerical quantification thereof.
For instance, when ou say probability is one in 7.3 million, what exactly does it mean?
Knut P. Heen
Aug 27 2024 at 5:59am
I don’t think it would have mattered much if they said the probability was 1 in 7 million instead of 1 in 70 million. Behaviorists like Kahneman say people treat both as zero anyway.
I agree with Hartford about teaching people how to use statistics properly. Unfortunately, when you teach quantitative subjects, most students tend to memorize formulas and refuse to understand the subject.
David Seltzer
Aug 27 2024 at 8:45am
Knut: Yes! Many students who use online equation solvers plug the equation in and get a calculated answer without going through the steps that explain the answer’s logic.
David Seltzer
Aug 27 2024 at 8:33am
Pierre: Really good stuff. When teaching probability theory, I spent much of the semester defining the difference between independent and dependent events using balls in an urn. The concepts of with replacement and without replacement went a long way in helping students understand how probabilities remain constant or change whether or not the first draw was replaced. In my experience, Bayesian statistics were more challenging for some of the students.
Knut P. Heen
Aug 27 2024 at 9:46am
Wouldn’t blackjack and counting cards be more motivating than the traditional urns? I believe the idea is that the cards are not replaced and that certain cards left favors the player vs. the house. That is when you are supposed to up your bet.
David Seltzer
Aug 27 2024 at 12:55pm
Knut: Good point. Blackjack is a nice example.
Comments are closed.