It’s generally recognized that our (American) response to the Covid-19 pandemic was disastrous. But I think far fewer appreciate the full scale of the disaster, or the most significant causal levers by which the worst effects could have been avoided. (Yes, Trump was bad. But his public health disinformation and politicization of masking—while obviously bad—may prove relatively trivial compared to the mammoth failings of our public health institutions and medical establishment.) Much of the pandemic’s harm could have been mitigated had our institutions been properly guided by the most basic norms of cost-benefit analysis.
This is the opening paragraph of Richard Yetter Chappell, “Lessons from the Pandemic,” DailyNous, January 19, 2021.
The whole thing is excellent. Chappell is a philosopher but the piece reads like a well-written analysis by a policy economist.
Another excerpt:
In ordinary circumstances, the status quo is relatively safe and so untested medical innovations present asymmetric risks. That is, until they are proven safe and effective, it may be reasonable to assume that the potential risks of an untested product outweigh its potential benefits, and so block public access to such products until they pass stringent testing requirements. (There are arguments to be made that FDA regulations are excessively onerous even in ordinary circumstances, but I remain neutral on that question here. I take it that there is at least a reasonable case to be made in the FDA’s defense ordinarily. No such case for the FDA’s stringency seems possible in a pandemic.)
A pandemic reverses the asymmetry of risk. Now it is the status quo that is immensely dangerous, and a typical sort of medical intervention (an experimental drug or vaccine, say) is comparatively less so. The potential benefits of innovation likely outweigh the potential risks for many individuals, and vastly so on a societal scale, where the value of information is immense. So the FDA’s usual regulations should have been streamlined or suspended for potential pandemic solutions (in the same way that any ethics barriers beyond the minimum baseline of informed consent should have been suspended for pandemic research). This should be the first thing the government does in the face of a new pandemic. By blocking access to experimental vaccines at the start of the pandemic, the FDA should be regarded as causally responsible for every Covid death that is occurring now (and many that occurred previously).
This last sentence is almost correct and similar to what Charley Hooper and I argued last month in “The FDA’s Deadly Caution,” AIER, December 16, 2020. Surely there are some deaths that would be occurring now, even without FDA intervention, due to people not taking the vaccine or due to the vaccines’ not being 100% effective
Yet another excerpt:
Closely related to the above mistake is the implicit assumption that it’s somehow better to do (or allow) nothing than to do (or allow) something imperfect. Letting the perfect be the enemy of the good in a pandemic is disastrous. Blocking quick Covid tests for having lower accuracy than slow ones is an obvious example of this form of stupidity. Deciding in advance that a vaccine must prove at least 50% effective in trials to receive FDA approval is another. (Obviously a 40% effective vaccine would be better than nothing! Fortunately it didn’t come to that in the end, but this policy introduced extra risk of disastrous outcomes for no gain whatsoever.)
Compare Dr. Ladapo’s argument in the WSJ that “Doctors should follow the evidence for promising therapies. Instead they demand certainty.” (Steve Kirsch expands on the complaint.) Again, this is a very basic form of irrationality that we’re seeing from the medical establishment.
Misguided perfectionism has also damaged the vaccine rollout due to prioritizing complex allocation schemes over ensuring that as many people are vaccinated as quickly as possible. (Some are letting doses spoil rather than “risk” vaccinating anyone “out of turn”!)
More examples are discussed here.
Do read the whole thing.
HT2 Daniel Shapiro.
READER COMMENTS
Thomas Hutcheson
Jan 25 2021 at 10:36am
“Closely related to the above mistake is the implicit assumption that it’s somehow better to do (or allow) nothing than to do (or allow) something imperfect.”
This implicit assumption seems to me to be the central fault of much of Libertarian critique of, minimum wages, carbon taxation, even “lockdowns.”
Jon Murphy
Jan 25 2021 at 1:27pm
Given that the author is using that critique against lockdowns, I do not see how you arrived at the conclusion that it is a critique in favor of lockdowns.
Thomas Hutcheson
Jan 25 2021 at 10:43am
On the substance of Yetter Chapell:
I find this very persuasive. But lots of things seem persuasive until one has heard the counter-arguments. I feel this way about this kind of criticism of the conventional public health mainstream.
Why haven’t we heard (I, at least, have not heard) an explanation in the MSM of what establishment public health officials say about why it was NOT a good idea to accelerate vaccine development with human challenge trials? Why was it NOT a good idea to massively test and retest asymptomatic people using cheap and quick, if not highly precise, screening tests so that those who test positive could self-isolate? Why de we NOT use the One Dose First system to speed up vaccinations as in the UK? Why do public health officials NOT discuss how the models they use to make recommendations and short-term predictions and how they have changed with the experience since February?
And although I wish public health officials to take the initiative to address these questions, if they have not, since they have not, why has the MSM not posed the questions?
Take for example an interview I overheard with Dr. Fauci. He gave very good answers to why we should believe that vaccine development was not accelerated and in particular why we should believe that serious side effects will be rare and minimal. But the interviewer did not ask him why vaccine development was NOT accelerated. Was the time used to allow him to be sure that side effects were rare and minimal worth the number of cases and deaths during that time?
Cost benefit analysis along with statistics oug to be part of the curriculum of schools of journalism.
Jon Murphy
Jan 25 2021 at 1:29pm
Eh, I disagree. I think an argument ought to stand on its own merits rather than on the strength of counter-arguments. To wit, if the house is on fire and someone suggests pouring gasoline on the fire, the mere fact there are no counter-arguments offered does not diminish the fact that the idea will not solve the problem.
Jon Murphy
Jan 25 2021 at 1:55pm
Let me clarify my comment a little bit:
When I review a paper for a journal, grade student assignments, or read a new book/article, my first question I ask myself is: “does this person’s argument make sense? Is it coherent? Can I see how they got to their conclusion?” If the answer is affirmative, then I move on (if it is negative, then I either recommend rejection, fail the student, or stop reading the paper. At this point, it is not persuasive).
So, if the argument is coherent, then and only then do I consider counter-arguments. What are reasonable objections one can make? What are factors the individual may not have considered? That sort of thing. If there are significant counter-arguments that the author omitted, I recommend a revise and resubmit (or, in the case of the student, revise the draft). At this point, the argument is ready to be considered whether or not it is persuasive.
At this point, I am done with the review of the argument. Note that at no point thus far have I considered whether or not the argument is persuasive. My evaluation of an argument does not depend on whether or not I find it persuasive. I recently recommended publication of an article where I did not find the author’s argument persuasive, but their reasoning was sound.
It is only at this stage do I consider whether or not an argument is persuasive. I am done with the positive analysis and now doing a normative analysis. I am Bayesian in my outlook: given the strength of my priors, I may require more or less evidence. For example, it would take *a lot* of evidence to convince me of the existence of a Giffen good. But whether or not I find an argument persuasive is different from whether or not there exist counter-arguments.
David Henderson
Jan 25 2021 at 3:35pm
Jon,
I like the order you lay out in referring a paper.
I disagree, though, with your claim that whether something is persuasive is a normative issue. It can be a normative issue, but it’s typically not.
Imagine someone argues that murder is wrong and you are persuaded. This is normative because the issue of right or wrong enters.
But imagine that someone writes a paper saying that a 10% increase in the minimum wage reduces teen employment by 1%. Imagine that you are persuaded. Where did the normative issue come in? I don’t see it.
Jon Murphy
Jan 25 2021 at 5:45pm
What I mean to say is there is nothing inherent to the argument itself that makes it persuasive or not. Rather, it depends on the individual reading (or hearing) the argument. There is no objective persuasiveness of the argument. but rather only the subjective persuasiveness.
I am using “normative” here to mean its a judgement call.
Floccina
Jan 25 2021 at 3:17pm
An interesting thing about that is, you do have to test in humans to get through stage 3 trials anyway. So even that method has the risk of serious side effects just with a smaller group of people.
Floccina
Jan 25 2021 at 3:12pm
When the logic and the theoretical science are there, we have many vaccines that work, allowed until proven damaging seems better than banned until proven safe and effective.
Todd Lorenz
Jan 25 2021 at 5:55pm
I wrote the letter to the editor describing how Dr. Ladapo’s editorial was misinformed. THERE IS NO SUCH THING AS PERFECT CERTAINTY when it comes to clinical research. However, adopting every new idea that may make it into print would sow chaos. There are just too many examples of preliminary reports that are shown to be wrong – in fact they are more often wrong than right.
Here is the text of my letter:
“Physicians do want to see at least reasonable evidence that drugs help patients with the diseases for which they are prescribed, but it is incorrect to state that “certainty” is always required. It is true that the FDA requires “the totality of scientific evidence available” to grant full marketing approval for new drugs, but it also has the ability during the pandemic to grant emergency-use authorization to drugs for which it is “reasonable to believe” may be effective. That status has been granted (and in the case of hydroxychloroquine, withdrawn) for several drugs during the Covid-19 pandemic.
The article cites several examples of preliminary studies that show what appears to be the benefit of drugs in treating Covid-19. Perhaps they will be proven to provide benefits on some level, but those of us in the clinical investigation community have learned not to jump on every early clinical trial with a sea change in medical practice. The folly of that approach has been well documented in the medical literature. As of Nov. 29, the website http://www.clinicaltrials.gov listed 2,295 clinical trials of treatments of Covid-19 currently under way. With this amount of investigation, there will be many false positives that are reported. Accepting every new claim of benefit that comes along as fact would betray the public trust.”
Jon Murphy
Jan 27 2021 at 11:52am
Todd-
It is true there is no such thing as perfect certainty in clinical research. Indeed, there is no such thing as perfect certainty in anything. As my friend and co-author Dr. Schuler likes to quip: “Statistics means never having to say you’re certain.”
The question on the table is why does the FDA get to decide what is “reasonable to believe”? As has been discussed here and elsewhere, the incentive would not be for firms to ” jump on every early clinical trial” for exactly the reason you state: they’ve been burned from such practices in the past. Firms have a strong incentive to not run with false positives. Comparatively, governments have a strong incentive to run with false negatives (if you read the FDA’s reports over the course of 2020, they are extraordinarily conservative even with their EUA authority).
Todd Jeffrey Lorenz
Jan 27 2021 at 4:51pm
Hello Jon
Well, who does get to decide what is “reasonable to believe”.
I disagree that firms have an incentive not to jump on every early trial – I’m sure many would, particularly if it is likely to be the only clinical trial conducted using a particular drug. Once a drug is selling for a particular indication, a company has little incentive to conduct further testing that might compromise the market.
The major point I make is that MOST early small trials that suggest a drug works fail in larger studies. If we changed medical practice based solely on such studies, the results would be confusion and chaos. The public would become even more suspicious of clinical science than they already are.
BTW, I am a physician who has worked in clinical research for many different biopharma companies for over 25 years. My opinion is based on my observations over these many years.
Dylan
Jan 28 2021 at 12:16pm
Jon,
My best guess is that, in the absence of some other entity taking up the mantle of proving drug efficacy (i.e. insurance companies or Medicare), we would see clinical trials be run mostly from the advantage they can give in marketing. That would likely mean underpowered trials that can suggest directionally some benefit, but that are unlikely to fail outright. There seems to be incredibly low likelihood that companies would have an incentive to spend billions of dollars running clinical trials that will likely fail.
We also have an example of a market where safe compounds with supposedly healthy benefits can be sold and marketed without any clinical evidence, the nutraceutical and additive markets. Some of those substances probably do have health benefits, but we don’t see any company running rigorous trials to find out. Why? In what ways would you want the drug industry to differ from the nutraceutical market?
KevinDC
Jan 25 2021 at 8:40pm
Credit to Chappell for argumentative boldness. This line in particular stood out to me:
I was immediately reminded of the economist David Colander giving kudos to “communicators, such as Charles Murray or Robert Samuelson, neither of whom is formally trained as an economist, who are applying economic analysis to real-world events with reasonable objectivity. They upset people, but that’s precisely what good economics analysis does. It forces people to look at reality from a different perspective than most use.”
The costs of delay can be massive. Every now and then I see announcements of some new drug or treatment with accompanying predictions that it can prevent up to 15,000 deaths per year nationwide. But since it takes around 10 years to get a drug approved, that means 150,000 people died waiting, in the name of safety. If ten such drugs or innovations are released any given year (a very conservative estimate) then the cumulative cost of safety for those ten drugs, and the decade it took to approve them, is one and a half million lives.
Chappell talks about the asymmetry of risk, but there’s also an asymmetry of outrage at play here too. If Bob Generic dies of heart failure, that’s sad for Bob’s family and loved ones. But people die of heart failure – it’s natural. So while people are sad, nobody is outraged about Bob’s death. And nobody thinks “Curse the fates – Bob wouldn’t have died if not for the fact that the medication that would have helped his heart condition is still four years away from being approved!” But people harmed by an unsafe drug are easily identifiable and seeing the identified harm promotes extreme outrage, so we will allow a hundred statistical lives to be lost to prevent one identified life from being endangered. This, I think, is a major failure of compassion. “Statistical lives” might seem like faceless abstracts, but they are real people with real lives that deserve just as much concern as anyone else’s.
To close out with a snark from Doctor Gregory House:
Dylan
Jan 25 2021 at 9:35pm
This seems to presuppose that the reason the drugs are being held off the market are for safety reasons, when that’s always been true only for a small fraction of drugs. Most clinical trials fail because the drugs simply don’t work (or they don’t work and have safety issues).
So, if you want to do the counterfactual, you’d have to assume that those 150,000 people would have gotten a drug that was 70% likely not to have worked, and 20% likely to have safety issues that (if it were to have gone through clinical trials) would have been severe enough to stop it from reaching the market. On top of that, you should add in the additional time it will take us to figure out which drugs do actually work. That’s not a trivial matter, as you can tell when drugs that have been on the market for decades go back and do testing, only to find out that they don’t actually work at all. See the Makena story for one notable recent example.
Comments are closed.