Where Superforecasters Start
How can you improve your forecasting? One of Superforecasting‘s most useful principles is “outside first.” Suppose you have a lot of details about the Renzetti family, and you’re trying to guess whether it owns a pet. It’s tempting to dwell on the details, but:
The first thing [superforecasters] would do is find out what percentage of American households own a pet.
Statisticians call that the base rate – how common something is within a broader class. Daniel Kahneman has a much more evocative visual term for it. He calls it the “outside view” – in contrast to the “inside view” which is the specifics of a particular case. A few minutes with Google tells me about 62% of American households own pets. That’s the outside view here… Then I will turn to the inside view – all those details about the Renzettis – and use them to adjust that initial 62% up or down.
It’s natural to be drawn to the inside view. It’s usually concrete and filled with engaging detail we can use to craft a story about what’s going on. The outside view is typically abstract, bare, and doesn’t lend itself readily to storytelling.
As far as I recall, I’ve yet to lose a bet I publicly blogged. What’s my secret? I doggedly take the outside view. When long-run trends say X, and the “latest news” says Y, I go with X. When Democrats won big in 2008, I saw good luck, not a new political regime. That’s why, in 2009, I bet my former co-blogger Arnold Kling that the Republicans would regain control of one branch of the federal government by 2017. I won in 2010. During the Great Recession, I saw a bad business cycle, not a “new normal.” That why, in 2013, I bet Tyler Cowen that unemployment will fall below 5% by 2033. I am very close to winning already. When anyone forecasts any specific major disaster, I scoff. Disasters do happen, but almost all specific predictions of major disasters do not come to pass. And that’s why I am eager to bet against such predictions.
I sometimes joke that “news melts your brain.” What I really mean is that news amplifies our misguided impulse to take the inside view of things. To be a superforecaster, of course, you have to use the news to tweak the outside view. But most people would be better forecasters if they ditched the news and relied on base rates alone.
Oct 19 2015 at 12:31am
I can’t believe Tyler made that bet with you, especially at those odds, in late 2013. If nothing else, demographics are highly in your favor. Workers at baby boomer ages have low unemployment rates.
Hilariously, the commenters on that post seem unanimously and strongly in favor of Tyler, except for the ones with some conspiracy theory about the data.
Oct 19 2015 at 6:31am
In the tournament, I was able to score lots of big gains in areas where I knew others would suffer from prejudice or lack of knowledge (and having more personal knowledge of a given thing.)
For example, relating to China, Westerners very often assume the very worst of the government, and I was able to secure large gains from making informed bets on this (e.g., probability of violent military confrontation in the South China Sea in the next month or two – not 20%, more like 2% – it does happen from time to time). In another case, a lot of people attributed very high probability (30%) that the Shanghai Free Trade Zone would allow Facebook, but I know that Chinese officials regard Facebook as a potential locus of organizing a revolution, and hate the idea of US officials having access to extensive data on the social networks and blogging habits of Chinese citizens, and the chance they would do so is essentially 0%.
With respect to questions on the Middle East, I was able to score big gains against assumptions that Iran is always lying and can never be trusted and will never live up to their promises (granted, I don’t trust them all the much) or about the inherent stupidity or tyranny of several regimes (on the matter of Syrian refugees, I knew it would be quite high because I’d met a very terrified man in Istanbul who had fled Aleppo in the first week of the conflict). In several other cases, the questions related to hypothetical specific loci of conflicts which had 0-1% probability because the hypothetical aggressor had essentially 0% chance to play a winning hand in that conflict (e.g., Syria attacks Israel, Iran blocks the Straight of Hormuz).
About Africa questions, people always seemed to bet heavily on the worst possible outcome, so lots of gains could be secured there.
A lot of other gains were easy to come by in cases of very low probability events, which people ascribed higher probability. E.g., probability of a war starting with the very short time period of just a few months (often pegged in the 10-20%, which is ridiculously high for such a short timeframe, so lots of gains in betting at 1-3% odds) or of a long term civil war coming to an end despite no credible peace talks (often pegged at 10-20%, whereas without credible peace talks this is more like 0-1% in the space of the following months). Or, the probability that some pandemic would set in (often betting at 10% within just a month or two, whereas historically this is less than 1%).
Other easy gains could be made on the backs of climate skeptics, who bet heavily against higher temperatures or lower arctic ice coverage (admitted, given natural variation, it’s not sensible to bet strongly on these).
In questions about currency and stock markets, the easiest gains were not so much about knowing anything … just check a day or two before closing and often people had not updated their expectations, so you could make gains without much long-term liquidity loss.
I don’t really like the new format (I think the 3rd year was the best format), and also now it’s a for-profit venture not an academic project, so I rarely participate anymore.
Oct 19 2015 at 12:19pm
I think watching the news is better than you realize. Sure, if you don’t pay much attention to the world and then one day see something about the stock market crashing you might naively assume that a recession is imminent. But if you frequently watch the news you realize that people make these kind of predictions all the time and you learn to ignore the doomsayers.
And it can also help you resist opportunistic political narratives. Right now, people on the left are outright ignoring the fact that the budget cuts in 2013 did not lead to recession(even though they claimed it would). And yet, many also claim that austerity is this great evil and needs to be stopped and they point to Europe as an example supporting their claims. History is a collection of selective facts that people use to support their story. The news has this problem too but paying attention allows you to more thoroughly see the details behind the narratives.
Oct 19 2015 at 1:44pm
“Did Tyler Cowen ever write out a rational for why he made that bet?
I get that they’re generally in line with the premise of his book
but the whole thing seems absurd, the odds and timeline seems ridiculous
as do the stakes, which seem to override everything else, seems like $10 is a pretty light tax”
Oct 19 2015 at 5:00pm
Since often both sides predict doom if their policy is not enacted I often find myself thinking both sides are wrong.
For example I doubt that atmospheric CO2 will be anywhere near as harmful most people concerned about it think, but I also think it would cheaper to get to zero net CO2 going into the air than most skeptics think (Assuming a CO2 tax with most of the proceeds paid out to people who remove CO2 from the atmosphere.) Pessimistic bias?
Pessimistic bias is very interesting to me because in their personal lives people often seem overly optimistic.
Oct 19 2015 at 8:06pm
Floccina, can you show your work on getting to zero net CO2 being inexpensive? As one point that is influential on me, the Kyoto Protocol is widely assumed to be economically devastating to the U.S. had it been adopted, and yet is far short of zero net CO2 even for the U.S. alone.
Kudos for bringing the issue up and all. To my view it should be front and center.
Oct 20 2015 at 12:12am
If we look at the IPCC’s actual estimates for the Social Cost of Carbon, we see it’s “31–121 US$/tCO2” at the high end. That means a tax of $1 per gallon at most. This is a livable (but painful) amount.
It will also be temporary because a carbon tax that high will make various forms of nuclear energy both politically and economically unbeatable.
The real problem is that the problem will be used as an excuse for other regulations.
Kyoto is, of course, irrelevant because nobody pays any attention to it.
Oct 20 2015 at 7:44am
“can you show your work on getting to zero net CO2 being inexpensive”
Energy intensity in the economy is fair low compared to historical standards, and getting lower every year. Now, energy represents just a few percent of production costs in most businesses.
Achieving a carbon-free economy in 10 years time might double the cost of energy (sorry, too lazy to show my work), but that represents only a few percent change in GDP. Moreover, that’s assuming that the policies required to engineer this change to a carbon neutral economy did not result in significant technological gains in energy efficiency and production.
Would it have been a disaster for 2014 if the economy in 2014 were the same real output as 2013? There would be costs, to be sure, but hardly disastrous.
I suggest that the alarmists on the one side of the story are nearly as unbelievable as the alarmists on the other side of the story.
I recall being at an energy conference, and listening to the representative of the International Energy Agency discuss the costs of change. He presented all figures in nominal terms. But some quick mental calculations reveal that this represented a level of annual investment equivalent to 1% of global GDP.
With global GDP growth in the 3-5% range, would 1% of global GDP dedicated to carbon free energy production be disastrous?
Here’s the real problem. Energy expenditures are relatively inelastic. So a carbon tax will almost entirely come out of the pockets of fossil fuel industry shareholders. This creates a very obvious constituency to resist change.
Oct 20 2015 at 11:54am
I would have no objection to a carbon tax combined with a policy of getting out of the way of nuclear energy and repealing laws mandating CFLs, low-flow toilets, etc.
Oct 23 2015 at 3:52pm
You say, “Energy expenditures are relatively inelastic. So a carbon tax will almost entirely come out of the pockets of fossil fuel industry shareholders.”
I thought energy demand was relatively price inelastic (e.g., Bernstein & Griffin, 2006). Your statement that energy expenditures are inelastic implies to me that, as prices rise, quantities demanded must fall correspondingly to keep total expenditures constant in the face of a price increase. Additionally, I should think that price inelasticity would imply that suppliers have pricing power; that is, they need not fear much loss in volume if they raise prices. Consequently, I should think fossil fuel suppliers would have little inhibition about passing a carbon tax on to consumers — the opposite of your conclusion. (Consumers might try to shift to nuclear & hydro, now tax-advantaged forms of energy. But neither of these can be expanded enough to replace the approx. two-thirds of our energy supplies that are carbon based. So consumers must either drastically lower their consumption, or they get forced back to newly taxed fossil fuels.)
I am not trained as an economist. So what am I getting wrong?
Comments are closed.