Social scientists use both controlled and natural experiments to estimate the impact of various sorts of public policies. One problem with this approach is that it is very difficult to ascertain the long run effects of policy, which might differ dramatically from the short run effects.
Consider how rates of alcoholism vary in different parts of the world:
1. Alcoholism tends to be more of a problem in northern Europe than in the Mediterranean countries of southern Europe.
2. Alcoholism rates are especially high among indigenous peoples in the Americas, Australia and the Pacific islands.
How can we make sense of this pattern?
One possibility is that alcoholism is least severe in areas with the longest history of drinking alcohol. Wine has been a popular beverage in the Mediterranean area for millennia. In contrast, indigenous peoples in the New World had little exposure to alcoholic beverages until recently.
This might seem an odd example to use when thinking about the effects of public policy, as these differences were due to cultural factors, not differing legal systems. But this example does illustrate that cultures evolve gradually over very long periods of time, in such a way that one group of people may react quite differently to the availability of alcohol than another group, even if the two groups currently face the same legal and economic constraints.
This fact is important when considering experiments on public policies such as “Universal Basic Income”. You can certainly imagine an experiment where you give a group of people a lump sum of money. But what does that tell us about the long run impact of such a program? Suppose the program continued over many generations, until there was no longer a stigma associated with living your entire life on UBI and putting no effort into work?
Any time we institute a permanent new public policy such as Social Security or federal deposit insurance, we are running a very long run experiment with our society. We don’t know how these programs will shape attitudes and behavior over the ultra-long run, as we don’t have the patience to run such experiments before the program is instituted.
In these cases, I rely more on my intuition about how society is likely to be affected in the very long run, and put little weight on social science experiments (many of which fail to replicate, even in the short run.)
READER COMMENTS
Nicholas Decker
Feb 8 2024 at 1:59pm
While not undermining the main thrust of the article, I would note that the prevalence of alcoholism in a culture is greatly affected by actual evolution — over time, we evolve protections against serious alcoholism. East Asians, for example, who have been exposed to alcohol for thousands of years, often become physically ill when they drink alcohol — an excellent guard against excess consumption!
steve
Feb 8 2024 at 2:44pm
You are making the argument for maintaining the status quo. We dont have alternate realities to test whether a novel idea will work so we cant really know the future for certain. For example, do we really know the long term effects of having free markets? Maybe it just takes a long time and they will result in something awful. I dont think so, but I cant know for sure.
“In these cases, I rely more on my intuition about how society is likely to be affected in the very long run, and put little weight on social science experiments”
Do you? Essentially every paper written is either a model or an experiment or both. You can reject empiricism and just embrace economic thinking, some economists seem to believe this, but I think that largely devolves into deciding stuff based upon your feelings/biases. I think it’s better to actually use what info we have available in experiments/literature and use our best judgment to evaluate them in the context of other literature, our experience and best knowledge.
Steve
vince
Feb 8 2024 at 5:41pm
Scott did say he limits his use of intuition for special cases.
How much weight should empirical information be given in social sciences anyway? There’s a serious replication problem. Austrian economists recognize it. The replication problem is why some regard social science as pseudo science.
I’ve come to the conclusion that economics falls under complexity theory. Good luck expecting certainty.
Scott Sumner
Feb 8 2024 at 11:48pm
“You are making the argument for maintaining the status quo.”
Not really. I can think of lots of changes I would like to make, such as abolishing rent control. But that preference is not based on a controlled experiment.
I’m not even saying that UBI is a bad idea. I think it’s quite plausible that it’s better than the current welfare system. It’s just that I don’t base that view on any particular experiments.
Junio
Feb 8 2024 at 5:31pm
Steve,
I wouldn’t interpret it too strongly as an argument for the status quo, more as to be cautious. We need that dose of caution.
Junio
Feb 8 2024 at 5:50pm
Good post. I’m curious: what’s your view on seeing Pareto efficiency as a starting point for analysis rather than, let’s say, a welfare criterion?
I feel like this would be an extension of your point, if I interpreted it correctly, to be skeptical and not overly confident that a particular policy proposal will work/improve conditions.
My question comes from this post:
https://open.substack.com/pub/pricetheory/p/is-everything-efficient?utm_source=share&utm_medium=android&r=1iy56g
Scott Sumner
Feb 9 2024 at 7:26am
I prefer a welfare criterion.
Jose Pablo
Feb 8 2024 at 7:02pm
In the ultra long run we are all dead
Bob
Feb 8 2024 at 11:50pm
The Soviet union looked pretty competent for a bit to… and the long run showed us its failures after decades of big mistakes. So yes, we definitely can’t go by early results.
The American suburban experiment is doing crazy things to prices, and it provides a built in lobby to slow down development. It wasn’t great, but people still think that car centered development is something to keep.
We also have the student loans experiment. It’s not unlike a rent co tool experiment: Looks great for a while, and then the market adapts in ways that give us bad outcomes. If those loans could be discharged by bankruptcy… how many people would be unable to get loans due to high risk, either due to the student’s underlying risk, or a loan too big to justify a low viability degree?
So more than worth about the experiments that haven’t been tried yet, I worry more about those that have already good evidence of failure, but that we refuse to abandon.
Scott Sumner
Feb 9 2024 at 7:28am
“The American suburban experiment”
Do you mean the original suburban experiment, which allowed for the construction of lots of apartment buildings? Or the modern version, that mostly bans apartments?
Arqiduka
Feb 9 2024 at 4:24am
Indeed, that’s the function of ideology, pooh-poohed by many on account of the allegedly superior common sense: common sense and d ideology apply over the short and ultra-long term respectively: since there’s no way to test these things in advance, societies with better ideologies outcompete others
Jose Pablo
Feb 9 2024 at 3:01pm
Given the actual (over)size of the State any public intervention will have negative consequences. “Public interventions” are sure subject to the law diminishing returns (at least if we rule out the sudden appearance of “public intervention design genius”, which looks like a reasonable thing to do). So, if there was any public intervention that could cause any good (and this is a big “if”) it has been adopted a long time ago.
It was the case even as far back as the 1970’s when Ronald Coase was editor of the Journal of Law and Economics and performed a positive analysis of the effects of regulations. The conclusions were pretty clear:
https://reason.com/1997/01/01/looking-for-results/
Coase: This is a very interesting question because one can’t give an answer to it. When I was editor of The Journal of Law and Economics, we published a whole series of studies of regulation and its effects. Almost all the studies–perhaps all the studies–suggested that the results of regulation had been bad, that the prices were higher, that the product was worse adapted to the needs of consumers, than it otherwise would have been. I was not willing to accept the view that all regulation was bound to produce these results. Therefore, what was my explanation for the results we had? I argued that the most probable explanation was that the government now operates on such a massive scale that it had reached the stage of what economists call negative marginal returns. Anything additional it does, it messes up.
John Hall
Feb 12 2024 at 7:43am
I meant to respond this earlier since I thought it was interesting, but got caught up.
My 2 cents:
As Tyler Cowen likes to say, solve for equilibrium
I saw something about the percent of Americans living in multi-generational households after reading your post. I suspect that Social Security played a significant role in the prevalence of the elderly living on their own instead of with their adult children. Perhaps that is one factor driving fertility lower. Elderly parents are less likely to live nearby and be available to babysit than they were pre-Social Security.
John Hall
Feb 12 2024 at 7:44am
Looks like my formatting (1., 2.) disappeared…
Comments are closed.