How the Blinder-Zandi Study Was Done
Alan Blinder and Mark Zandi used Zandi’s econometric model as the basis for a claim that the stimulus and the TARP worked. Thirty-five years ago, I was Blinder’s research assistant, doing these sorts of simulations on the Fed-MIT-Penn model for the Congressional Budget Office. I think they are still done the same way. See lecture 13. Here are some of the things that Blinder had to tell his new research assistant to do.
1. Make sure that there were channels in the model for credit market conditions to affect consumption and investment.
2. Correct the model’s past forecast errors, so that it would track the actual behavior of the economy over the past two years exactly. With the appropriate “add factors” or correction factors, the model then produces a “baseline scenario” that matches history and then projects out to the future. For the future, a judgment call has to be made as to how rapidly the add factors should decay. That is mostly a matter of aesthetics.
3. Simulate the model without the fiscal stimulus. This will result in the model’s standard multiplier analysis.
4. Make up an alternative path for what you think would have happened in credit markets without TARP and other extraordinary measures. For example, you might assume that mortgage interest rates would have been one percentage point higher than they actually were.
5. Simulate the model with this alternative scenario for credit market conditions.
6. (4) and (5) together create a fictional scenario of how the economy would have performed had the government not taken steps to fight the crisis. According to the model, this fictional scenario would have been horrid, with unemployment around 15 percent.
i) Blinder and Zandi do not spell out the details of step 1 or step 4. Thus, I have no idea how to evaluate their approach to estimating the impact of financial measures.
ii) Other than the add factors, and any ad hoc adjustments that were made in step 1, every result in the paper would have been found by simulating the model three years ago. There is no new evidence being brought to bear. What Blinder and Zandi are reporting is the Keynesian theory that was built into the model.
iii) They report model multipliers to two decimal places, e.g. 1.61 for extending unemployment insurance benefits. They do not provide confidence intervals or any other estimate of reliability.
iv) the paper has not been published in a peer-reviewed journal. The theoretical and statistical properties of the model probably would not be considered acceptable in modern practice. Even if those issues were overlooked, the intensity of the political rhetoric combined with the opacity of the exercise would cause difficulties for most editors of academic journals.
I do not think we will ever know what would have happened to the economy without the fiscal stimulus and the large monetary interventions. My guess is that the overwhelming majority of economists would agree that we will never know the answers to those questions. However, in the competition for public attention, Blinder and Zandi have two advantages. First, they support a narrative in which government experts did the right thing, which is comforting to government experts and all who believe in them. Second, at a tactical level, their use of an esoteric computer model along with those two decimals of precision, they intimidate journalists and other laymen.
I know that they think this is for a good cause. They really believe that the stimulus and TARP were good policies that got a bad rap. But in my view that does not justify this unseemly exercise in propaganda dressed up as research.
Jul 28 2010 at 1:25pm
It is nice having someone who knows where the bodies are buried from a modeling perspective to give guidance on this.
This reminds me of a task I was charged with as a young researcher to estimate the impact of electricty deregulation in the state I was living in.
We essentially assumed prices that a bunch of things would happen at the begining and then used an untested model to predict what would happen ceteris paribas.
In other words it was totally BS. The only thing I think these type of models are good for is to say.
If you think X will happen
If you think the world works like your model say it does
Here are the relative magnitudes of what you might expect.
But both of those first two statements are totally unproven, so any results assume you believe them to be true on faith. Essentially it is an UPPER bound on the impact as it puts the story in its most charitable light.
J. Daniel Wright
Jul 28 2010 at 1:43pm
Dr. Kling, with all due respect, I believe you just snatched the pebble from Dr. Blinder’s hand.
David R. Henderson
Jul 28 2010 at 2:09pm
Home run, Arnold.
Jul 28 2010 at 2:10pm
But how bad are these problems? What would you discount their findings by?
Just as important: what does it mean that “we’ll never know”? That economists won’t debate niggling details until everyone has tenure? That every calculation will suffer fatal methodological defects? That every calculation will be methodologically valid but in stark disagreement?
Jul 28 2010 at 2:21pm
[Comment removed for rudeness. –Econlib Ed.]
Jul 28 2010 at 2:44pm
There seems to be a lot of assumptions in the economic models about stimulus spending, but I never hear any discussion of what happens when the money is taken out of the economy in order to fund the stimulus. After all, the money does not just appear out of nowhere. If we borrow $500 billion from other nations to fund the stimulus, does that cut into our ability to sell our exports? If it does, how much and what multiplier effect does this have on the suppliers to export companies and, in turn, their suppliers? If we borrow domestically, what other investments do the lenders forgo in order to make the loan to the government? If we just print the money, what happens to the economy? If a spending multiplier is created by spending the same money over and over through the economic food chain, why would there not be a similar multiplier when money that would have otherwise been spent is diverted to the stimulus? Do the economic models include any of this?
Jul 28 2010 at 3:05pm
If they wanted a serious test of the model, could they input some clearly disastrous policy & see what happened? For instance, if they input a stimulus of $20 trillion, then if the model said we’d be living the high life, wouldn’t everyone agree that it was missing something important?
Jul 28 2010 at 3:34pm
This convinced me but I was already convinced. What do Drs. Blinder and Zandi say? Other Keynesians and proponents of TARP/Stimulus? I can guess what they might say, but if they can agree that this ‘paper’ is bologna then at least they are trustworthy.
Although that makes some intuitive sense, I doubt that the models work that way. I can’t speak for economics, but in physics, our model are usually relatively fined tuned to give good results for a small range of inputs. Consider one of the simplest approximations in physics made when discussing pendulums. Sin(X) = X. This approximation will be very good for a displacement from equilibrium of 2 degrees but if we multiply the input by a factor of 20 as you suggest (1 trillion dollar stimulus becomes a 20 trillion dollar stimulus) and displace it by 40 degrees, our results might be nonsense. Models are always approximations and are not guaranteed to give good results for all inputs. GIGO, as they say.
Jul 28 2010 at 4:28pm
Arnold, you’re right to point out that Keynesian assumptions were put into their model, but of course any model needs assumptions in order to produce anything. Blinder and Zandi may not have done a top-level, academic, peer-reviewed job, but calling it propaganda goes too far. Both of them command a little more respect than that.
I’d be interested in reading your own econometric study about how TARP and ARRA affected the economy, but since you say you “do not think we will ever know what would have happened” it sounds like you’re discrediting econometrics altogether.
I wish all the facts agreed with my philosophy, but it doesn’t happen that way.
Jul 28 2010 at 4:49pm
It seems like $1 trillion could already be at about 30 degrees. But really that’s the problem – there’s no control measurement to calibrated to.
Jul 28 2010 at 4:54pm
Arnold refuses to play the game of ‘eCONometrics’ and wisely in my opinion. He’s written a lot about macro-econometrics and the con game that it is. You may want to do a google search on this blog for the topic if you are new.
Jul 28 2010 at 5:14pm
Great to see this analysis, Arnold. It was confirmation to me how I was feeling about the Blinder-Zandi Study.
Jul 28 2010 at 5:37pm
That the stimulus helped was evidently a foregone conclusion, given the Keynesian assumptions built into the model. To offer the results up to the public as “proving” anything seems to me to be fundamentally dishonest. The results are a false quantification, which tend to lend a spurious verisimilitude to the assertion that the stimulus helped. That is why it is fair to describe it as an exercise in propaganda.
Probably it would also be fair to describe the recent CBO exercise the same way.
Jul 28 2010 at 6:12pm
I really liked the conclusion that we would all be eating mud and twigs without TARP and the stimulus. Case closed.
Jul 28 2010 at 6:53pm
Thanks for shedding some light for us non-specialists on how this kind of modeling is done.
Question: Can you confirm or reject my suspicion that when the politicos quote some number of jobs created by the stimulus, they’re giving us the predictions of a model like this one rather than empirical data?
Jul 28 2010 at 6:53pm
Thank you for the inside view. I knew all this kind of thing form grad school but these guys release their results with such confidence that I don’t analyze it as I should.
“What Blinder and Zandi are reporting is the Keynesian theory that was built into the model.”
david (not henderson)
Jul 29 2010 at 6:24pm
Sweet. Nice post, Arnold.
Jul 29 2010 at 10:19pm
Are the claims about 3 MM jobs saved or created based ENTIRELY on the model? i.e., could they have been made at the time ARRA was passed?
Comments are closed.