Basic Decision Theory
By Arnold Kling
From my latest essay:
If we give more people MRI’s, we reduce type I errors but increase type II errors. If we give fewer people MRI’s, we reduce type II errors but increase type I errors.
The Maggie Mahars of the world want to blame such errors on the fact that we have private-sector medicine. However, errors are inherent in medicine, because knowledge is imperfect and decisions must be made under uncertainty. Given the uncertainty, one cannot reduce errors of one type without increasing errors of another type. Most importantly, the existence of errors does not prove that the system is flawed.
UPDATE: Bruce Schneier says,
The data mining success story is in credit cards. The credit card companies use data mining to constantly look at the stream of credit card transactions, and find credit cards that have been stolen. It works because credit card thieves are relatively numerous.
…It doesn’t work well looking for terrorists. The number of terrorists, with respect to the general population, is infinitesimally smaller than the number of credit card fraudsters to the number of credit cards. Also, there is no well-defined profile.
Schneier’s view is a counter to my argument for more data mining with regard to terrorism. But we agree that our current approach on airline security makes too many unnecessary type II errors.