“Science is prediction.” Not quite true, but still deeply insightful. Especially when you remember that “Science is prediction” doesn’t mean a prediction that you whisper to yourself. “Science is prediction” means a public prediction. You have to shout it from the rooftops, in advance.
When Einstein publicly predicted a specific anomaly in the deflection of light during a solar eclipse – and turned out to be exactly right – it was awesome. Einstein loudly said exactly what was going to happen before it happened. He was right. And the world took notice.
Now ponder this: As a matter of pure logic, the evidence would have been just as strong if Einstein waited for the eclipse measurements to come in, and then showed they were consistent with his theory. But of course the world’s reaction would have been far more tepid in this scenario.
Why? In common-sense terms, the answer is, “It’s easy to ‘explain’ facts you already know. Practically any smart person can do it. As a result, we reasonably discount post hoc explanations.”
If you know a little modern psychology, though, you’ll be sorely tempted to say, “Public prediction greatly reduces the severity of confirmation bias.” Human beings naturally tend to misinterpret the facts in favor of their own views. But such misinterpretation is much more difficult if (a) you say exactly what’s going to happen before you see the facts, and (b) do so publicly to ensure that many of your listeners will refuse to gloss over your failed predictions.
And if you know more modern psychology, you’ll probably pile on. “Public prediction greatly reduces the severity of not only confirmation bias, but also Biases X, Y, and Z.” For example, public prediction also greatly reduces the severity of Social Desirability Bias. Human beings naturally tend to say and believe things that sound good. But sugar-coating reality is much more difficult if (a) you say exactly what’s going to happen before you see the facts, and (b) do so publicly to ensure that many of your listeners will refuse to gloss over your failed predictions.
The upshot: The adage that “Science is prediction” rests not on logic, but on psycho-logic. Logically speaking, explaining after the fact is just as epistemically revealing as predicting before the fact. Psychologically speaking, however, explaining after the fact is far inferior to predicting before the fact. Why? Because explaining after the fact is mired in intellectual corruption. Predicting before the fact is, by comparison, squeaky clean. Predicting before the fact is the best way to signal that you’re overcoming bias.
Otherwise you’re mired in what Tetlock calls “vague verbiage” and “self-scoring.” You say things vague enough to be compatible with a wide range of outcomes. And then you further water-down this forgiving metric by delegating the scoring to yourself. An epistemic kangaroo court.
Notice, by the way, that betting combines the advantages of prediction with a recognition of its flaws. Like predictions, bets are specific. And even if a bet is “private,” at least one person who doubts you – your opponent – knows about it.
At the same time, the bet avoids the unfortunately binary nature of prediction. Suppose Einstein’s light deflection prediction had failed. That doesn’t prove he was wrong. Perhaps something went wrong with the equipment. Or one of the human beings measuring the light could have made a mistake. Or a rival scientist could have deliberately sabotaged the experiment. A bet – rather than a flat prediction – incorporates all of these contingencies. And reminds us to focus not on any particular bet, but on the bettors’ track record. That’s the best guide to whose judgment we should trust.
Given my still-perfect betting record, I’ll admit this is a suspicious conclusion. And if that’s what you’re thinking, good for you. You’re using psycho-logic to ferret out the truth, as every thoughtful person must.
READER COMMENTS
Todd Ramsey
Jan 3 2022 at 10:10am
“Given my still-perfect betting record…”
Your March 2020 exit from the stock market, costing you a significant portion of your wealth, apparently notwithstanding.
It might be fair to include that mistake in a post about the importance of public proclamation.
However, you deserve much credit for publicly acknowledging your investment mistake. You could easily have kept that information to yourself.
zeke5123
Jan 3 2022 at 10:24am
Do you think your track record is actually an indication of not betting enough? That is, aren’t you leaving money on the table by preserving a pristine record?
DeservingPorcupine
Jan 3 2022 at 10:43am
I think that would be true if his purpose in making public bets was to generate income. But since I’m pretty sure it’s simply to hold other predictors accountable and to help normalize the practice of public bets, having a good record might well be the superior choice.
Ankur Aggarwal
Jan 4 2022 at 12:03pm
Compared to whom, though?
How many other pundit/prognosticator types even think like this?
Do even NN Taleb and Robin Hanson, other proponents of SITG-type credibility tests, have such “extensive” records?
I think your question is very intelligent and well-motivated, but it’s the kind you’re probably better off asking once publicly betting on predictive questions becomes the standard norm in academic settings, rather than the rare exception.
Daniel Klein
Jan 3 2022 at 11:10am
Interesting post! Some random remarks:
Yes, bold specific predictions are noteworthy, and indeed your betting track-record is impressive. Hats off.
The final words of volume 1 of Democracy in America contain an amazing prediction. We’re talking 120 years before the fact!
That wasn’t Tocqueville’s only sound prediction—or warning—alas.
In Adam Smith’s History of Astronomy (p. 103), he speaks of how Newton’s followers—that would be Edmund Halley, who died in 1742—put their cred on the line by predicting the next coming of what came to called Halley’s comet. A footnote added subsequently, probably by executors Black and Hutton, says: “the return of the comet happened agreeably to the prediction.”
The title of your post “Overcoming Bias in the Mother of Science” is perhaps true by definitions, since “bias” has bad built into it, and “science” as you use it here has good built into it.
Peter Gerdes
Jan 3 2022 at 1:07pm
Yes, very good.
But, I’d add that this issue isn’t psychological in the sense in which an paredolia is psychological and we could easily program machines to do better. The fundamental problem is that your priors shouldn’t be dependent on your observations (only updated by them) but we never really explicitly set out all our priors nor could we really hope to. So in order to give the post-diction equal weight we would need to be able to recover our earlier mental state and check how plausible we would have found the claim before seeing the evidence. Not only would that have risks of manipulation if it was possible it likely would also require saving/copying something like the agents complete mental state frequently (don’t know what facts will later be used as post-diction evidence). This might be prohibitive even for AI
Ron Browning
Jan 5 2022 at 7:51am
My memory of Tocqueville is that he worried and envisioned a US federal government withering away as to become non-relevant.
Jose Pablo
Jan 5 2022 at 11:26am
“explaining after the fact is just as epistemically revealing as predicting before the fact.”
Maybe, but “predicting” is far more useful. You can “act” upon predictions but not on “after the fact explanations”
Afterall “explaining after the fact is totally useless”. Particularly so on the Economics realm, where elaborate “explanations after the facts” frequently lead us to horrible predictions.
Comments are closed.