# Statistical vs. Material Significance

## By:

I appreciate Bryan’s pointer to an article by Alan Gerber and others on personality and ideology. However, the article illustrates what I consider to be a methodological error. That error is to use statistical significance as a metric.

Statistical significance is *not* a measure of the importance of a relationship. Statistical significance is a measure of the unlikelihood that you got your results solely due to chance sampling error.

In measuring the importance of personality traits in predicting political ideology, I would ask how much of political ideology can be explained by personality. The answer, if you look at the R^{2} in table 6 is, “not much.” For economic conservatism, in fact, the R^{2} ranges from .07 to .15. Because the sample sizes are large, this is statistically significant. But from a practical point of view, one would have to describe variation in economic ideology as almost entirely unexplained by these variables.

In statistical research, I believe strongly that you should focus on the *practical* significance of results. Statistical significance is not practical significance.

For example, suppose that you found that the elasticity of demand for good X was -3.0, but the t-ratio was 1.8, while the elasticity of demand for good Y was -.06 with a t-ratio of 8. The practical significance is that X is elastic while Y is inelastic. But if you were to use the t-ratio as a metric, you would conclude that the demand for Y is more elastic than the demand for X. That would be incorrect.