An AP Stats Lecture
By Arnold Kling
Because yesterday’s hearing ran so late, I missed my AP stats class. Here is the way I would explain what went on to my students.At Freddie Mac, we used the concept of Type I and Type II errors. If a $100,000 loan defaults, you lose about $50,000. If the borrower pays on time, your profit is about $100. So approving a bad loan is a much costlier error than rejecting a good loan. In terms of type I and type II errors, we have:
|Decision||Loan Repays||Loan Defaults|
|Approve Loan||correct||Type I error|
|Deny Loan||Type II error||correct|
Given the costs involved, you want to set alpha (the probability of type I error) to a low number, say .0001, or 0.01 percent. That is, you want to experience very few defaults.
Every applicant has a credit score, called a FICO score (FICO is an acronym for Fair, Isaac Company, the name of the leading credit scoring firm). Oversimplifying, let us say that a credit score of 660 is the cutoff that gives a loan a default probability of .0001.
In some sense, the null hypothesis is that the loan should be denied. You reject the null hypothesis (approve the loan) if the FICO score is above 660. You fail to reject the null hypothesis (that is, you deny the loan) if the FICO score is below 660.
Using this rule, If the FICO score is above 660 but the loan later defaults, you will commit a Type I error. If the FICO score is below 660 but the loan later gets made by another company and the borrower repays on time, you will have committed a Type II error.
Suppose that a loan with a 630 FICO score has a probability of default of .06. Since you deny all of these loans, you have a probability of a Type II error on these loans of (1-..06) = .94. We say that the probability of Type II error is beta. We say that (1 – beta) is the Power of the test. We calculate Power based on a specific alternative, namely the alternative that the FICO score is 630. In this example, the power is pretty low–we have a pretty high probability of making a Type II error.
Other things equal, if you try to make fewer Type II errors, you will make more Type I errors. For example, you could reduce Type II errors by approving loans with FICO scores of 630, but that means you will experience more Type I errors.
In 2004-2007, Freddie Mac and Fannie Mae lowered their lending standards. This meant approving loans with lower FICO scores, along with other methods (mortgage underwriting is based on a number of factors). At the hearing, Congressmen were asking (not in these words), why did you succumb to pressure to reduce Type II errors and increase Type I errors?
The CEO’s denied that they were under pressure from Congress to approve more loans to meet “affordable housing” goals. Assuming that they were not lying under oath, that would say that the CEO’s never received a phone call from Congressmen asking them to approve more loans. However, one suspects that they felt pressure indirectly, because they know that they need friends in Congress, and the friendship of some Congressmen, particularly from urban districts, depends on how well Freddie and Fannie serve those markets.
The CEO’s said that the market was changing. In the past, when they made Type II errors, nobody saw them. If Freddie and Fannie denied a loan, then the loan was not made. Wall Street was now making loans to borrowers with lower FICO scores, and these borrowers were not defaulting. The Type II errors that Freddie and Fannie were making were more visible than they had ever been before. This made the CEO’s question their own credit policies, and they decided to loosen up.
As it turned out, the success of Wall Street’s looser lending policies had been due mostly to luck–rapidly rising house prices. Once house prices stopped rising, Wall Street’s loans started defaulting. The large number of Type I errors had been exposed. Meanwhile, Freddie and Fannie had loosened up at just about the worst possible time–just as house prices were reaching their peak. They made a lot of Type I errors, for which we as taxpayers are going to pay a steep cost.