In the comment section of my previous post, a number of people were dismissive of the model because it failed to do well in out-of-sample tests. Here are some examples:

It’s literally fit to the data through 2017. There’s nothing to explain.


The model quickly fails as soon as it encounters out of sample data, wouldn’t read into it too much.

I don’t agree.  To illustrate my objection, I’ll review an example from previous research I did on the Great Depression.  

Irving Fisher first developed the idea of the Phillips Curve in a 1923 paper that contained this graph (from a later 1925 version):

The variable P is a distributed lag of inflation, whereas T is a crude measure of real output.  I then applied Fisher’s model to the period from 1923 through 1935:

Fisher’s model worked very well during 1915 to 1922, and continued to do OK from 1923-33. Then the model completely broke down after July 1933. It’s important to know why.

Instead of assuming “there’s nothing to see here”, I decided to investigate why the Phillips Curve model failed after July 1933.  It turns out that in July 1933 the federal government mandated a 20% across-the-board nominal wage increase, which caused prices and output to move in the opposite direction.

The failure of the Phillips Curve model after July 1933 suggests a need to replace this simple model with a more fundamental model, such as the AS/AD framework, which allows for both positive and negative correlations between prices and output. 

I am reminded of this exchange in a recent David Beckworth podcast with George Selgin:

Beckworth: But don’t we, as people who like to look through maybe an aggregate demand or nominal GDP perspective, aren’t we also invoking something similar to the Phillips curve? We invoke a short run aggregate supply curve that gets really steep going up. How different is that?

Selgin: I think it’s very different. I think that the Phillips curve takes too many shortcuts or tries to do too many things at once. That’s what a reduced form relationship basically is. I think that thinking in terms of aggregate supply and aggregate demand shifting around is closer to grasping the true structural relationships. Even though it’s not particularly complicated, it isn’t much more complicated, but it is somewhat more complicated. You have two schedules instead of one, for starters, and I think that it matters.

The model of economic sentiment discussed in my previous post continued to do well between 2016 and 2020, and then failed spectacularly.  It’s interesting to think about why this happened.  The Economist attributed the failure to Covid, whereas I cited increased political polarization.

Out-of-sample failures do point to a flaw in the model.  But they can also provide hints as to how the model can be improved.