The previous installment was here.
The installment below discusses macroeconometrics. It is very brief, because I think that the issue is better discussed in the context of the historical perspective that I envision providing in the main part of the book. We are still on the introduction.Econometrics
What can we learn about macroeconomics by studying the history of macroeconomic data? Much less than one might hope.
One approach to using historical data is called calibration. This involves writing down a set of generic equations that determine the evolution of macroeconomic variables, and then by a process of trial and error arriving at values for parameters that allow the equations to reproduce historical performance. As a very simple example, one could take all components of spending other than consumption as exogenous, and then write down a simple consumption function as C = a + bY, along with the identity Y = C + I + G + NX. Then, try out various values for a and b, and, using historical values for investment, government spending, and next exports, find the values that allow you to best track the history of C and Y.
This process of trial and error can be assisted by the use of computational techniques borrowed from statistics. The most basic technique is linear regression, which can be modified and enhanced in a variety of ways. However, the statistical properties of these techniques rarely carry over to the practice of macroeconometrics.
In statistics, each observation is supposed to be independent of the other observations. With macroeconomic data, this independence condition is violated. If you pick two periods at random, they are going to be much more similar if you happen to pick adjacent quarters (say, the second and third quarter of 1965) than if you pick two quarters from different decades.
In statistics, parameter estimates are unbiased under the assumption that the investigator makes only one attempt to use the data. In practice, economists tend to engage in what Edward Leamer calls specification searches. They obtain parameter values from statistical computations, then tweak the model, obtain new values, tweak again, and so on. This process vitiates the statistical properties of the resulting estimates.
Regardless of methodological problems, macroeconometrics would be useful if the data spoke with a strong voice. That is, if it were the case that certain model specifications and parameter values were clearly needed in order to explain and predict the evolution of macroeconomic data, then we could use historical evidence to limit the range of theories worthy of consideration. Unfortunately, that is not the case. Often, a simple one-variable random walk with drift can explain and predict a macroeconomic series of interest as well as or better than a sophisticated multivariate model, even at long time horizons. There simply is not enough information in the data to make a convincing case for any particular theory of macroeconomics.
READER COMMENTS
Mike Rulle
Aug 13 2010 at 12:56pm
Given what you call “specification searches” (other terms/concepts include multiple comparison procedures—like flipping a 100 different fair coins 10 times each and declaring the one that came up with 9 heads “unfair”) is covered in almost every introduction to statistics texts, it is mystifying this kind of error appears to be ubiquitous, particularly in the social sciences.
This conceptual error (assuming it is conceptual) is most common when the primary source of data is a given and not experimentally derived—(like historical data series in Economics and Finance)—although the latter does not prevent it, as many in pharmaceutical research can attest.
Jagdish Handa has written about the history of the academic study of monetary policy and its inability to predict income. Models were built, they always failed to predict, data was then refit, and the process continued uninterrupted for 40 failed years.
My favorite article on this topic is still the one by Caltech and Berkley professor David Leinweber where his famous tagline was coined to characterize data fitting tricks: “Butter in Bangladesh”.
“STUPID DATA MINER TRICKS: OVERFITTING THE S&P 500”
fundamentalist
Aug 13 2010 at 3:21pm
Part of the problem is that the data we need doesn’t exist. Hayek mentioned this in his Nobel Prize speech. All macro models have fixated on aggregate demand because the data is available. But that is like the drunk looking under the street lamp for his keys because “that’s where the light is.”
I think a good Austrian model would perform well, but Austrians tend to not be interested. A few have done econometric work and it looks good, but their work is limited to proving a few aspects of the Austrian model. I think better theory will produce better models.
Also, I think a technique called structural equation modeling would do a lot to help weed out bad theory. SEM is designed to compare various models. SEM is used a lot in psych, sociology and marketing, but it seems few economists are aware of it.
Comments are closed.