The Methodology is Flawed
By Arnold Kling
My father always said that there were three iron laws of social science.
1. Sometimes it’s this way, and sometimes it’s that way.
2. The data are insufficient.
3. The methodology is flawed.
On politically sensitive topics, the tendency is to apply (3) to the other guy’s methodology, never to your own. But if my comments here were not sufficiently clear, let me say that I do not believe that the Conley-Dupor paper is reliable. If I thought that they were, I would not have said that this is an important research question going forward. Having said that, I give them credit for doing an actual study, as opposed to cranking out a pre-existing computer model.
In the stimulus-debunking category, I tend to find the work of John Taylor and John Cogan more convincing than Conley-Dupor. Taylor and Cogan claim that the states used stimulus money mostly to reduce borrowing. That would imply that the stimulus affected neither public sector nor private sector jobs, in any direction.
On another issue, Gifted and Talented programs, people are attacking the method of looking at students who were borderline. But the whole point is that it is not helpful to make an observational study comparing the performance of very bright students in GT and very weak students not in GT. The best way to measure the impact of the GT program itself is to look at students who were on the borderline, and compare what happens to those in GT and those not in GT.
If that methodology does not satisfy you, then run a controlled experiment, in which you randomly assign some very bright students into GT classes and others into non-GT classes. My guess is that the long-term educational outcomes will not differ. But don’t just tell me stories that bright students are bored if they are not in GT as if that invalidates the methodology of looking at students on the borderline.