Critique My First Serious Survey
By Bryan Caplan
I’ve never written a serious survey before. Sure, I wrote the Libertarian Purity Test twelve years ago, but that was just for fun. But now I’ve written a first draft of a survey on which my co-author Ilya Somin and I plan to base a series of academic articles.
I’ve already used the Survey of Americans and Economists on the Economy to test for systematically biased beliefs about economics. Now Somin and I are designing a survey to test for systematically biased beliefs about political responsibility. The idea: Ask both the general public and Ph.D. political scientists about who in government has power over what, and see if their average beliefs are the same.
Who cares? Well, one big idea in economics and political science is that voters don’t need to understand policy; they can just re-elect incumbents when there is prosperity and peace, and throw the bums out if there is depression and war. Sounds good, but what if an incumbent has nothing to do with a particular problem? As Achen and Bartels put it:
If jobs have been lost in a recession, something is wrong, but is that the president’s fault? If it is not, then voting on the basis of economic results may be no more rational than killing the pharaoh when the Nile does not flood, as some scholars believe occurred in ancient Egypt, or voting against Woodrow Wilson when sharks attack the Jersey shore, as we believe happened in 1916. (citations omitted)
In any case, I’m attaching a draft of the survey. Please tell me what you think. Specifically:
1. Any suggestions for how to improve the wording of the questions?
2. Any political actors I should add or remove?
3. Any outcome variables I should add or remove?Who in Government Has Power Over What?: Draft Survey
In the United States, there are several different branches of the federal government, including the President, Congress, courts, and government agencies. Some of these branches may have more power than others to get the outcomes they want. Others may have a lot of power over some outcomes, but little power over others. I am going to ask you some questions about how powerful you think the different branches of the government are.
1. The President
Tell me if you think that the President has a lot of power, some power, a little power, or no power over:
a. How well the economy does during the next two years
b. How well the economy will be doing twenty years from now
c. The size of government
d. How long people in the United States live
e. Whether we are at peace or war
f. Whether we win the War on Terrorism
g. Crime rates
h. The environment
[Repeat same questions for…]
3. The courts
4. The Federal Reserve
5. The Department of Defense
6. The Environmental Protection Agency
7. The Food and Drug Administration