Grist for Arnold's Mill
By Bryan Caplan
From Kahneman et al’s “Would You Be Happier If You Were Richer?”:
Strack and colleagues reported an experiment in which students were asked: (i) “How happy are you with your life in general?” and (ii) “How many dates did you have last month?” The correlation between the answers to these questions was -.012 (not statistically different from 0) when they were asked in the specified order, but the correlation rose to 0.66 when the order was reversed with another sample of students. The dating question evidently caused that aspect of life to become salient and its importance to be exaggerated when the respondents encountered the more general question about their happiness. Similar focusing effects were observed when attention was first called to respondents’ marriage or health. One conclusion from this research is that people do not know how happy or satisfied they are with their life in the way they know their height or telephone number.
My response, naturally, is that question-wording effects are an argument for improving surveys, not getting rid of them.