Why does the National Security Agency (NSA) spy on Americans? In short, it is attempting to reduce even further the small probability of terrorist attacks on Americans. That reduction in probability, times the value of the damages averted, is the expected benefit of spying. However, spying is costly in a number of ways. A numerate analysis shows that the cost of NSA spying is substantially higher than the expected benefits. NSA spying on Americans should be ended.
This is the opening paragraph of one of the two Featured Articles for January’s Econlib. It’s “NSA Surveillance: A Cost/Benefit Analysis,” by Charles L. Hooper.
Starting with some reasonable estimates of various costs and various probabilities, Hooper finds that the costs of NSA spying greatly exceed expected benefits. So, to bias the results against this initial finding, he inflates his estimate of benefits and understates his estimates of costs. Bottom line: it’s still not worth it. And he does this without even considering some costs of NSA spying, costs that he lists but does not delve into further.
READER COMMENTS
Finch
Jan 6 2014 at 1:53pm
Surely $1 billion per terrorist attack is far too low? Isn’t the risk completely dominated by the tail events? September 11th cost near on a trillion dollars, even if you only count the direct costs of the Iraq war. And we haven’t ever had a really big terror attack, like a nuclear or biological attack, but I’d expect that to dominate the calculation.
Further, there’s no need to treat people equally – you can lessen your focus on huge swaths of the population.
I’m not saying the cost-benefit calculation obviously works out, I’m just saying this one was rigged not to.
David R. Henderson
Jan 6 2014 at 2:08pm
@Finch,
The Iraq war was not a cost of the 9/11 attacks. The Iraq war was chosen by Bush and Congress.
And yes, you’re right that the risk is completely dominated by tail events and that the 9/11 attacks were such an event. It’s also true, I think, that the terrorists on 9/11 got very lucky. Recall that the architect (IIRC) of the Twin Towers was in a depressing funk for weeks after the attack because he had thought the towers would not fall. I bet one of the most pleasantly surprised people on 9/11, out of the small number who planned it and knew it was coming, was Osama bin Laden.
Finch
Jan 6 2014 at 2:29pm
I wrote an email to a friend on 9/13 predicting it. And it would have been easier to convince Congress to go along had Gore been president. It was an obvious consequence.
Leaving 9/11 aside, surely there’s a greater than 0.1% chance of a significant bio attack in 2014 killing, say, 30,000,000 people at roughly $10,000,000 per life? Obviously this is harder to estimate than it is to add up the number of days Boston was closed multiplied by the number of people not working, but it’s the necessary first step in this calculation.
I’m reminded of meteor impacts, where the risk math is entirely driven by low-probability but very-bad events.
I could believe $1,000,000,000 is the modal event, but not the mean event, though it still seems low.
David R. Henderson
Jan 6 2014 at 2:42pm
@Finch,
I wrote an email to a friend on 9/13 predicting it. And it would have been easier to convince Congress to go along had Gore been president. It was an obvious consequence.
I’m not challenging your predictive ability. I’m saying that the Iraq war should not be counted as a cost of the 9/11 attack.
surely there’s a greater than 0.1% chance of a significant bio attack in 2014 killing, say, 30,000,000 people at roughly $10,000,000 per life?
From my reading of the literature, it’s hard to conceive of a bio attack that would kill even 1,000 people. Do you have a cite for your claim?
Finch
Jan 6 2014 at 2:55pm
> I’m saying that the Iraq war should not be counted
> as a cost of the 9/11 attack.
I don’t understand this. It should be counted just as, say, the TSA should be counted. Is there some world in which we could have chosen not to do it? When A is an inevitable consequence of B, I think it’s fair to count A as a cost of B. I could imagine a world in which we had attacked Iran and not Iraq, but I don’t think that was the way to bet contemporaneously.
> Do you have a cite for your claim?
No, I completely made up those numbers. I have no cite. And the prosecution of the Iraq war certainly greatly reduced our estimate of the probability of such an event. I remember reading Judith Miller at the time and being honestly worried about things like pre-positioned smallpox stocks cited around the US. We now have more information, so we can rule that particular scenario out. It would probably take a nation state to do such a thing.
If you want something easier to estimate, imagine an Iranian nuclear weapon smuggled into New York, which is likely higher probability today.
Finch
Jan 6 2014 at 2:56pm
Irk. For “cited” please read “sited.”
Kushana
Jan 6 2014 at 3:00pm
The NSA’s job is not only to find potential terrorists. If you’re going to compare against the full NSA budget, you need to tally all of the NSA’s benefits.
Chris
Jan 6 2014 at 3:07pm
There are economic costs and political costs. Which one do you think the NSA cares most about?
David R Henderson
Jan 6 2014 at 3:11pm
@kushana,
Did you read the article?
Bostonian
Jan 6 2014 at 4:35pm
Whether Mr. Henderson thinks the U.S. should have invaded Iraq after 9/11 is not relevant. What matters is whether terrorist attacks increase the chance of U.S. military action, rightly or wrongly, and they almost certainly do.
The open borders advocates should realize that Muslim immigration to the U.S. increases the chance of a terrorist attack by Muslims in the U.S. and thus U.S. reprisals against Muslim countries — “invite the world, invade the world” as Steve Sailer would put it. Maybe would should not invite the world.
David R. Henderson
Jan 6 2014 at 4:36pm
@Finch,
Is there some world in which we could have chosen not to do it? When A is an inevitable consequence of B, I think it’s fair to count A as a cost of B.
As you probably know, if you read this blog frequently, I try to avoid talking about “we” when it doesn’t apply. “We” didn’t choose. Congress and the President did.
But now to your substantive point. Yes, there is such a world. The government could easily have chosen not to attack Iraq. Look at the lengths Bush went to to make the case for about a year and a half. He could have easily chosen not to do that. So it follows that A was not the inevitable consequence of B.
James G
Jan 6 2014 at 5:05pm
I think it’s incorrect to focus on the actual costs and benefits of trying to prevent terrorist attacks. I think we can all agree that: 1) there are non-pecuniary costs to a terrorist attacks. 2) No reason the government should be risk neutral in choosing the optimal amount to spend.
The real question, I think, is why do we focus on this one specific low probability event? There are countless others that we are unconcerned with (e.g. a large asteroid smashing into the earth, dramatic climate shift). Is it because we have a direct reference event for terrorist attacks? It would be interesting to hear some feedback on this.
MingoV
Jan 6 2014 at 5:47pm
The NSA response to the presence of terrorists among us reminds me of a governmental response to AIDS in the 1980s.
AIDS panic was rampant. Tests were developed to detect HIV. Idiot politicians in Chicago, against the advice of medical experts and epidemiologists, voted to require HIV testing to get a marriage license. The prevalence of HIV in heterosexual couples was extremely low. The program was expensive (over $100 per couple). In 18 months they had tested over 100,000 people and found six positives. Four of the six were false positives.
Given the extremely low prevalence of terrorists in the USA, I expect the false positive to true positive ratio of NSA screening to be more than 1000:1.
@Kushana: “The NSA’s job is not only to find potential terrorists….”
The NSA’s charter is to use electronic surveillance to detect and track foreign terrorists and spies. The only spying the NSA is authorized to do within the US is of people in contact with suspected foreign spies and terrorists. And the NSA is supposed to get warrants to do that. It’s the FBI’s job to find and monitor potential ‘home-grown’ terrorists.
Finch
Jan 6 2014 at 5:48pm
“The government could easily have chosen not to attack Iraq. Look at the lengths Bush went to to make the case for about a year and a half. He could have easily chosen not to do that. So it follows that A was not the inevitable consequence of B.”
Easily? I think we have to agree to disagree.
Do you really think the probability of invading Iraq was not materially increased by September 11th? That’s just not credible to me. I think it went from about 0% to about 100% in the space of one morning. As I said, I think there’s some chance you could have substituted Iran for Iraq, had there been some smoking gun of state involvement.
I’m sorry about the use of “we.” “I” could not have chosen not to do it. From my perspective, it was an inevitable consequence. Maybe the president could have chosen not to do it, but it’s hard to see how someone could be elected president who could have chosen not to do it.
Charley Hooper
Jan 6 2014 at 10:35pm
@Finch,
Pardon my ignorance, but what did Iraq have to do with the 9/11 attacks? Iraq didn’t attack this country. Also, from what I can tell, Saddam Hussein had no connection to al Qaeda, had no weapons of mass destruction, and was not a threat.
@James G,
I think it’s incorrect to focus on the actual costs and benefits of trying to prevent terrorist attacks.
That’s what economists do; they conduct cost/benefit analyses, even for events that involve disease, destruction, and death.
No reason the government should be risk neutral in choosing the optimal amount to spend.
Risk tolerance is usually employed for decisions that involve a large downside, relative to the decision-maker’s wealth and attitude about risk. If the average terror attack has a cost of $1 billion, that $1 billion is a relatively small amount for the U.S. government or the country as a whole.
Charley Hooper
Jan 6 2014 at 11:05pm
@MingoV,
Given the extremely low prevalence of terrorists in the USA, I expect the false positive to true positive ratio of NSA screening to be more than 1000:1.
If there are 100 terrorists, the NSA’s tests would need to have 99.99% sensitivity and 99.969% specificity to get 1,000:1.
If there are 1,000 terrorists, the NSA’s tests would need to have 99.99% sensitivity and 99.685% specificity to get 1,000:1.
If there are 10,000 terrorists, the NSA’s tests would need to have 99.99% sensitivity and 96.845% specificity to get 1,000:1.
Dan S
Jan 6 2014 at 11:29pm
David and Finch,
I think you two may be talking past each other. There is an endogeneity/exogeneity problem here. Whether we choose to treat the US gov’t’s response to an attack as endogenous or exogenous matters in our calculation of the benefits of NSA surveillance. Finch, if you don’t mind me putting words in your mouth, you seem to be suggesting that we should treat the response as endogenous, and I think that is largely correct. If a terrorist group were to successfully carry out a WMD attack in the US, Congress would pass a law that makes the Patriot Act look like the Freedom of Information Act, and with the full support of the American public would authorize the president to turn large portions of the Middle East into a glowing parking lot. I’d say this response is endogenous because, sad to say, it is really not something that is going to be changed by the three of us debating it, or even a thousand or ten thousand people heartfeltly petitioning the gov’t about it. So we really ought to consider that reaction as a cost of a possible terrorist attack.
Andrew
Jan 6 2014 at 11:48pm
I agree with most the piece and the general point the author is trying to make, but the assumptions about the specificity/sensitivity are a bit arbitrary and silly. Garbage in garbage out…
why not try a sensitivity of 10% and a specificity of 99.9%. Then, assuming 1000 terrorists as the author suggests, I get 3000 innocents arrested per terrorists, it actually narrowly passes the authors criterion costing 90 million/terrorist. This to me seems far more reasonable (though probably an underestimate of the program)
Even if we can’t catch 10% of terrorists, we can certainly make it more difficult to plan an attack, and I doubt that 300,000 Americans have really been arrested under this program, so I doubt that the specificity is any lower than that.
Not to mention that even a small risk that a nuclear weapon kills 1 million people in new york city with a cost in the trillions implies that a mean of 1 billion/ terrorist attack is a hopeless underestimate, (a previous commenter mentioned this already)
It still says something that the cost of the program is potentially so high on such a narrow view of the costs, but the author should be more careful about his assumptions.
Charley Hooper
Jan 7 2014 at 12:24am
@Andrew,
I used your numbers and got $100 million per terrorist, so the direct costs would equal the direct benefits.
You make a good point about garbage in/garbage out, but none of these sensitivity and specificity assumptions are garbage. Rather, they are thresholds that need to be met to get X result. If the NSA proceeded for 30 years and gave us complete access to its data, we could compute the sensitivity and specificity. In the absence of that, we need to make reasonable assumptions and look at thresholds.
So your 10% and 99.9% are what the NSA would need to break-even.
I question whether the NSA could ever achieve a specificity anywhere close to 99.9% (when has that ever been done?) and I question whether the American people would be satisfied with a program that flags one out of ten terrorists.
Shane L
Jan 7 2014 at 6:55am
Remember that terrorist groups are fully expecting the US and its allies to embark on expensive anti-terrorist campaigns. Indeed they are relying on it:
“For example, the latest issue of the Islamist magazine Inspire celebrated the negative economic effects of the Boston bombing, and the “lockdown” response of law-enforcement. Similarly, it previously proclaimed the Christmas Day 2009 “underwear” bombing attempt a success even though the bomber was stopped because the attempt was part of “Operation Hemorrhage,” an effort focused on instigating fear and causing economic damage. The initial disaster is the terror attack. But a subsequent crisis occurs if we self-impose further stresses on the nation by our response.”
http://nationalinterest.org/commentary/handling-the-unstoppable-terror-threat-9061?page=1
Organisations like Al Qaeda deliberately seek to provoke expensive campaigns by the US to prevent Al Qaeda attacks. Even when security is successful at preventing an attack, they celebrate at the self-imposed cost.
I’m not familiar enough with NSA’s work to know if it is worthwhile or not. But a system that prevents any terrorist attack from happening at all could still be one that terrorists are very pleased with, curiously, since it imposes economic and social costs.
Finch
Jan 7 2014 at 9:24am
Thank you Dan S. You may have better articulated my point. Without 9/11, we would not have invaded Iraq, at least for many years. With it, we were near certain to invade Iraq.
Were there another major successful terror attack on the US, depending on the nature of the attack, it would not be surprising in the least to find us invading Yemen or using WMD in the middle east. That reasonably has to be counted as a cost of a successful terror attack.
Kushana
Jan 7 2014 at 10:45am
@David R Henderson
Of course I read the article. The author never mentions any benefit to come out of the NSA other than terrorism prevention, and his only attempt to dissect the NSA budget is a guess that half of it goes to surveillance.
The NSA uses surveillance for more than finding terrorists, and if he is going to use simplistic cost/benefit analyses, he had better account for those benefits.
Charley Hooper
Jan 7 2014 at 11:29am
Finch and Dan S,
If a man burns his house down after his wife ruins dinner, should we blame the woman for the damage to the house? Or, more to the point, should we plan on a house fire for each ruined dinner? Another man wouldn’t have reacted that way and, further, that same man might have a different reaction–we hope–the next time.
Shane L,
Remember that terrorist groups are fully expecting the US and its allies to embark on expensive anti-terrorist campaigns. Indeed they are relying on it.
Exactly. Which is why the government shouldn’t react that way. We thwart them precisely by not overreacting.
It’s as if, in my example above, the woman wants the house burned, so she ruins dinner hoping the man will start a fire. The man should realize that his burning down the house is exactly what the woman wanted, is unnecessary, and just hurts himself. He’s being duped.
Dan S
Jan 7 2014 at 12:04pm
Charlie Hooper,
I think the two situations are not comparable here.
Speaking at a very high level, criminal law does not exist to punish murderers because murder is bad, therefore we should punish them. At least I think that is a very naive moralistic interpretation. Laws against murder (and their enforcement) exist so as to solve a basic collective action problem. Alice is willing to give up her right to murder Bob in exchange for Bob giving up his right to murder Alice. Police, judges, and courts exist to enforce this basic pact. That is why a breakdown in law and order is generally seen as bad, because although gov’ts can often be abusive, it’s really really useful to have a big strong guy with a gun tell Alice and Bob to stop fighting, go home, and start cooperating and trading with each other, and whoever starts trouble will regret it.
In other words, your example with the man and his wife is flawed in my opinion, because you’re assuming we’re talking about morality when we really need to be talking about incentives.
Note that these considerations don’t apply at all in the world of Congressional actions and international relations, because nations are not themselves subject to a larger world government, and similarly Congress can pretty much do what it wants. So if we can pretty much guarantee that Congress will do all this stuff in the event of a terrorist attack, we ought to consider it a real cost. And believe me, I wish it weren’t so! I hate the NSA spying, the Patriot Act, the Iraq War, and all that!
Charley Hooper
Jan 7 2014 at 1:56pm
Dan S,
I think the real distinction is between normative and positive (or descriptive) thinking. Or, between what the government will likely do (overreact and start a war) and what it should do (react appropriately).
You will note that there have been at least three terrorist attacks since 9/11 that did not cause the government to invade another country: the underwear bombing attempt , the shoe bombing attempt , and the Boston marathon bombing.
Dan S
Jan 7 2014 at 2:31pm
Charlie Hooper,
Compared to a WMD attack, those were nothing. They shut down Boston, and that was for a domestic terror attack. Now imagine a foreign WMD attack. I think the responses would be totally different.
You are right though, in that there is a positive/normative bridge here. I guess I should have handed out my philosophical business card first: not only do I not believe in free will (in the sense that a person is incapable of willfully undertaking an action that is contrary to what their brain state would otherwise demand), but I also don’t believe in objective morality (in the sense that there is no rational reason for an agent to not do X when he otherwise would merely because he just “shouldn’t”)!
Andrew
Jan 7 2014 at 2:33pm
@Charley Hooper
There is always a trade off between specificity and sensitivity in any test that you are free to adjust by changing the test threshold. ie, its always possible to achieve a very high specificity, but the sensitivity may suffer dramatically. The author fails to recognize this aspect, and implicitly assumes that specificity and sensitivity are correlated.
It depends also on what you define to be a false positive. The author assumes that positives are people arrested under the program, in which case I would say that the specificity is more than 99.9% for the program as it exists now. 99.9% would already imply that more than 300,000 Americans arrested over some time period. 80% would mean that arresting 20% of our population was being arrested, it doesn’t vibe with my personal observations.
Now, there may be many more people who get flagged for closer attention, but are cleared or marked for observation in the future without their knowledge. But if you want to count those people as positives then the cost per person is certainly not $30,000.
Charley Hooper
Jan 7 2014 at 3:56pm
@Andrew,
There are an infinite number of combinations of sensitivity and specificity. The article, for the sake of brevity, lists just a few.
The author assumes that positives are people arrested under the program, in which case I would say that the specificity is more than 99.9% for the program as it exists now.
You are assuming that the NSA spying has actually done something. From what I can tell, it is essentially just getting started and there have been no false positives or true positives to-date. That in no way obviates our scrutiny of the NSA’s programs; we can question whether something will be bad before it is proven bad.
David R. Henderson
Jan 7 2014 at 4:28pm
@Kushana,
This is what I was responding to when I asked if you had read the article. You had written:
The NSA’s job is not only to find potential terrorists. If you’re going to compare against the full NSA budget, you need to tally all of the NSA’s benefits.
And you may be right. But that’s why, in his sensitivity analysis, Charles Hooper did not compare against the full NSA budget, as he made clear. Instead, he went from assuming that half of NSA’s budget is devoted to spying on Americans to the extreme assumption that it spends none of its budget on spying on Americans. That way, he can look just at the benefits of spying on Americans, such benefits being the reduction in the probability of a terrorist attack. He doesn’t need to look at the benefits of other things the NSA does with possibly large parts of its budget.
Roman Lombardi
Jan 9 2014 at 11:25am
The thing I’ve noticed about these massive government programs is that the publicly stated goal is rarely the actual goal. Unfortunately, I can’t help but think the NSA has more sinister motivations…
Kushana
Jan 13 2014 at 12:54pm
@David R. Henderson
I concede the point. The author’s estimate of NSA’s budget is secondary both to the point he is trying to make and to mine, and I spent too much effort belaboring a broken point.
The real problem with the author’s analysis is single-use probabilities. In trying to draw an analogy with medicine, he assumes that the NSA only gets one shot at declaring yea or nay on a potential US terrorist. And that’s not the way it works.
Assuming the NSA does such things, it would be functioning more like a police department. If we assume the TSA is roughly like the police, how do the police secure *any* convictions with a 40% detection rate? Answer: they rely on more investigation prior to hitting the courtroom.
Medicine and the TSA are similar in the data source you have is what you get. You can run more tests, or interrogate longer, but you get one shot at diagnosis. The police and NSA’s terror detection are ongoing investigations, in which multiplicative probabilities mean your accuracy rate can approach 1. If they’ve bought a lot of fertilizer AND they’ve talked to a AQ recruiter AND they visited an extremist preacher AND they’ve talked about jihad with their pals, … you get the picture.
This is what I meant when I called his analysis simplistic: it’s a terrible simplification for the complex task of finding and identifying terrorists.
Comments are closed.