Categories:
Economic Methods
Interesting email from Todd Proebsting, reprinted with his permission.
Dear Prof. Caplan,
Your recent attempt at a wager on Iran’s nuclear progress, and the subsequent exchange with the author about why you wanted 10:1 odds reminded me that the CIA tried to quantify what certain claims meant:
| 100% Certainty | ||
| The General Area of Possibility | ||
| 93% | give or take about 6% | Almost certain |
| 75% | give or take about 12% | Probable |
| 50% | give or take about 10% | Chances about even |
| 30% | give or take about 10% | Probably not |
| 7% | give or take about 5% | Almost certainly not |
| 0% Impossibility | ||
It’s from a 50-year old CIA proposal.
It’s interesting to me that “Almost certain” was only 93%. I think the CIA would have argued that you should have been asking for even longer odds given the original claim.
Cheers,
Todd Proebsting
READER COMMENTS
Sam
Aug 3 2015 at 11:12am
Bizarre that the language is asymmetric about 50%. Is there some prospect-theory justification for that?
Adam
Aug 3 2015 at 11:13am
Or there’s the IPCC (TS, box TS1) typology:
Virtually certain: 99–100% probability
Very likely: 90–100% probability
Likely: 66–100% probability
About as likely as not: 33–66% probability
Unlikely: 0–33% probability
Very unlikely: 0–10% probability
Exceptionally unlikely: 0–1% probability
Tom West
Aug 3 2015 at 11:24am
I remember having a long debate with my wife about this early in our marriage.
It took a while to understand that we weren’t using the terms the same way:
“Absolutely certain” me: 90% her: 100%
“Very certain” me: 80% her: 100%
“Pretty certain” me: 70% her: 98%
“Fairly certain” me: 60% her: 95%
“I think so” me: 25% her: 90%
Eventually, I just started using actually percentages. As for her, if she’s willing to venture an opinion, then she’s right (with exceptions countable on one hand in 25 years).
Al
Aug 3 2015 at 11:31am
It’s interesting to me that the CIA define “Probable” as approx. 75%, but “Probably Not” as approx. 30%.
The definitions are not balanced between events which are likely and other events which are equally unlikely.
There are also a few grey areas, most notably events with probability of approx. 12% to 20% have no suitable description under this proposal.
There appears to be a considerable amount of discretion involved.
Examples:
1. Using these definitions an event with probability 39% would be described as “Probably Not”, but an event with probability 61% wouldn’t be described as “Probable” (it’s in a grey area between “Chances about even” and “Probable”).
2. An event with 85% probability would be described as “Probable”, but one with a 15% probability would fall in the large grey area between “Almost Certainly Not” and “Probably Not”.
Al
Aug 3 2015 at 11:38am
Adam: At least the IPCC definitions are symmetrical about 50%. I don’t much care for their incredibly wide “About as likely as not band” which ranges from 33% to 66% though. 66% is about twice as likely as not.
Richard
Aug 3 2015 at 3:10pm
I think that it depends on what you’re talking about. If I say “I’m absolutely sure that she has a crush on you,” it probably means about 90%, since I think that’s about the maximum level of certainty you can have about something like that. If I say “I’m absolutely certain that the sun will come up tomorrow,” it’s more like 99.999999999%, since we can be that certain about physical phenomena.
Ken P
Aug 3 2015 at 10:57pm
Like Al, I have a problem with the lack of symmetry between probably and probably not. It’s like nails on a chalkboard.
It sounds like Watson would be answering only when almost certain.
Dick White
Aug 4 2015 at 7:43am
Can someone refer me to a pithy essay on how one concludes, for example, that a given outcome is, say, 63%-87% likely (Probable)?
D. F. Linton
Aug 4 2015 at 11:59am
Pretending that you can estimate a numerical probability for future events that will either occur or not occur once is at best a rhetorical device and at worst gross self-delusionary scientism. You can estimate odds for a bet or weigh whether given odds are favorable or unfavorable given your present knowledge and preferences. But such polls will vary from person to person; otherwise you would take the other side of bets? Frequentist considerations aside, how can the truth or falsity of such statements even be proven?
If I say there is a 83% chance that some event will occur tomorrow and it happens I say see! If it doesn’t happen then we just experienced a 17% probability event.
_NL
Aug 4 2015 at 1:24pm
Tax opinions have similar methods of converting confidence levels expressed as phrases into numerical percentages, and here is one common interpretation:
These aren’t exact, and the percentage level assigned will vary across situations and has varied over time. Some practitioner interpretations treat “reasonable basis” as 1 in 3 while reducing “should” to 60% chance.
Joe Topp
Aug 4 2015 at 4:32pm
The CIA, where even the error bars have error bars.
(e.g. give or take about 10%)
Comments are closed.