Blunderov
Archon
Gender: 
Posts: 3160
Reputation: 8.69
Rate Blunderov
"We think in generalities, we live in details"
  
|
 |
Prosecutor's fallacy
« on: 2006-10-29 13:25:45 » |
 |
[Blunderov] Statistics alarm me. I know enough to realise that they can be horribly abused - I wonder what the statistics on this phenomenon might be? If such statistics were to reveal that statistics are abused more often than not, would one be entitled to conclude that the study is likely false? Probably.
Eek.
http://www.guardian.co.uk/science/story/0,,1933706,00.html
Bad Science
Prosecuting and defending by numbers
Ben Goldacre
Saturday October 28, 2006
The Guardian
There once was a time when your biggest worry, as a paediatrician, was being lynched by a herd of illiterate tabloid readers; but if you're Sir Roy Meadow you get scapegoated by the innumerate too. First he was struck off by the GMC for giving flawed evidence as an expert witness in the Sally Clark trial. Then he won an appeal. And this week, he's half lost it again.
But what if the flaws in his evidence weren't his fault alone? In the Sally Clark case, where two children in the same family had died - Mrs Clark subsequently had her convictions for murdering her two sons quashed on a second appeal - Meadow quoted "one in 73m" as the chance of two children in the same family dying of Sudden Infant Death Syndrome (Sids). This was a problematic piece of evidence for two very distinct reasons. One is easy to understand, the other is an absolute mindbender - but if you can concentrate, for three minutes on a Saturday morning, I can prove that you are smarter than Professor Sir Roy, the judge in the Sally Clark case, her defence teams, the appeal court judges, and almost all the commentators reporting on the case.
We'll do the easy problem first: the figure itself is iffy, as everyone now accepts. It was calculated as 8543 x 8543, as if the chances of two Sids episodes in this one family were independent. This feels wrong from the outset, and anyone can see why: there might be environmental or genetic factors at play.
But even if we accept that two Sids in one family is much more likely - say, 1 in 10,000 - any such figure is still of dubious relevance. The action is in what we do with this spurious number. Many press reports stated that 1 in 73m was the chance that the deaths of Sally Clark's two children were accidental: that is, the chance she was innocent. Many in the court process seemed to share this view. But this is an example of a well documented piece of flawed reasoning known as "prosecutor's fallacy".
Two babies in one family have died. This in itself is very rare. Once this rare event has occurred, the jury needs to weigh up two competing explanations: double Sids or double murder. Under normal circumstances - before any babies have died - double Sids is very unlikely, and so is double murder. But now that the rare event of two babies dying in one family has occurred, the two explanations are suddenly both very likely.
If we really wanted to play statistics, we would need to know which is relatively more rare, double Sids or double murder. Not only was this crucial nuance missed at the time, it was also clearly missed in the appeal: they suggested that instead of "1 in 73,000,000" Meadow should have said "very rare".
They recognised the flaws in its calculation but still accepted it as establishing "a very broad point, namely the rarity of double Sids". But that rarity is irrelevant, because double murder is rare too. An entire court process failed to spot the nuance of how the figure should be used. Twice. If it is true that Meadow should have spotted the problems in the interpretation of his number, then so should the rest of the people involved in the case. The prosecutor's fallacy is also highly relevant in, for example, DNA evidence. Anyone who is going to trade in numbers also has a responsibility to understand them - and as you've just seen, it's hardly rocket science.
|
Author
Logged
