Scott Alexander has a new post on our highly dysfunctional FDA. He starts with the debate over the new Alzheimers drug, and then explains how FDA errors led to needless delays of Covid testing and vaccines, which caused thousands of unnecessary deaths.
While that discussion is well worth reading, Alexander was just warming up:
I worry that people are going to come away from this with some conclusion like “wow, the FDA seemed really unprepared to handle COVID.” No. It’s not that specific. Every single thing the FDA does is like this. Every single hour of every single day the FDA does things exactly this stupid and destructive, and the only reason you never hear about the others is because they’re about some disease with a name like Schmoe’s Syndrome and a few hundred cases nationwide instead of something big and media-worthy like coronavirus. I am a doctor and sometimes I have to deal with the Schmoe’s Syndromes of the world and every f@$king time there is some story about the FDA doing something exactly this awful and counterproductive. For example, there are only a few hundred cases nationwide of Infant Short Bowel Syndrome, a condition where some babies cannot digest food effectively. You can save their lives by using an IV line to direct nutrients directly into their veins, but you need to use the right nutrient fluid. The FDA approved an early draft of the nutrient fluid, but it didn’t have enough fish oil, which is necessary for development, so a lot of the babies still died or ended up with permanent neurological damage. In the 1990s, researchers figured out what was going on and recommended adding fish oil to the IV fluid. The FDA responded that they had only approved the non-fish-oil version, it would take them a while to approve the new version, and until they did that adding fish oil was illegal. A bunch of babies kept dying and getting permanent neurological damage, and everyone knew exactly how to stop it, but if anyone did the FDA would take away their licenses and shut them down. Around 2010, Boston Children’s Hospital found some loophole that let them add fish oil to their nutrient fluid on site, and infants with short bowel syndrome at that one hospital stopped dying or ending up permanently disabled, and the FDA grudgingly agreed to permit it but banned them from distributing their formulation or letting it cross state lines, so for a while if you wanted your baby not to die you had to have them spend their infancy in one specific hospital in Massachusetts. Around 2015 the FDA said that if your doctor applied for a special exemption, they would let you import the correct nutritional fluid from Europe (where, lacking the FDA, they had just added fish oil to the fluid as soon as researchers discovered it was necessary), but you were only able to apply after your baby had already sustained serious damage, and the FDA might just say no. Finally in 2018 the FDA got around to approving the corrected nutritional fluid and now babies with short bowel syndrome do fine, after twenty years of easily preventable state-mandated deaths. I CANNOT STRESS ENOUGH HOW EVERY SINGLE THING THE FDA DOES IS LIKE THIS ALL THE TIME.
Obviously there’s a bit of hyperbole there at the end, but later on he has a nice explanation of why the problem is not the specific people that work at the FDA, rather it’s the incentive structure:
I want to stress that, despite my feelings about the FDA, I don’t think individual FDA bureaucrats, or even necessarily the FDA director, consistently make stupid mistakes. I think that given their mandate – approve drugs that definitely work, reject ones that are unsafe/ineffective, expect people to freak out and demand your head if any unsafe/ineffective drug gets through, nobody will at all no matter how many lifesaving treatments you delay or stifle outright – they’re doing the best they can. . . .
And it’s hard to even blame the people who set the FDA’s mandate. They’re also doing the best they can given what kind of country / what kind of people we are. If some politician ever stopped fighting the Global War On Terror, then eventually some Saudi with a fertilizer bomb would slip through and kill ~5 people. And then everyone would tar and feather the politician who dared relax our vigilance, and we would all restart the Global War On Terror twice as hard, and drone strike twice as many weddings. This is true even if the War on Terror itself has an arbitrary cost in people killed / money spent / freedoms lost. The FDA mandate is set the same way – we’re open to paying limitless costs, as long as it lets us avoid a very specific kind of scandal which the media will turn into 24-7 humiliation of whoever let it happen. If I were a politician operating under these constraints, I’m not sure I could do any better.
If you are confused by Alexander’s reference to media irresponsibility, look at how the media is hyping a few cases of vaccinated people getting Covid.
As with Matt Yglesias’s posts on passenger rail and Razib Khan’s posts on woke excesses, one ends up feeling a sense of almost complete hopelessness after reading Alexander. And that’s why Alexander, Yglesias and Khan are the three Substacks that I read—I don’t want my information sugar coated.
READER COMMENTS
Thomas Lee Hutcheson
Aug 5 2021 at 6:54am
You should feel hopeless only if believe that any market intervention is bad, so of course the FDA kills babies with SBS.
If you thought that, although difficult given cognitive biases, in some cases approving/disapproving drugs might be done in ways that would pass a cost benefit test and that it makes sense to try to get those kinds decisions made and the ones that don’t pass changed, you could be conditionally hopeful. One would think that Public Choice theory could help in designing better public policies, but I don’t see t used that way very often.
TGGP
Aug 5 2021 at 12:31pm
What is the market failure that justifies the FDA banning a vaccine or test rather than granting or withholding a certification (and holding people legally liable if they fraudulently claim such a certification)? We could then sell those unapproved things in would-have-been-banned stores.
https://www.overcomingbias.com/2007/03/paternalism_is_.html
Dylan
Aug 5 2021 at 7:50am
I like Scott Alexander, he’s a great writer and he tends to think logically through things. However, in my reading of this post, I feel like he misses some important and relevant facts that should influence the discussion.
One, The default bias of the FDA and just about everyone who works in pharma should be that nothing works. That’s hyperbole of course, but it isn’t that far from the truth. Many, many drug development people can work an entire career and not touch anything that will ever reach the market. You can argue this is because the FDA is too strict (and many do). I see it though as most drugs don’t do what we hope they will, but they do do a bunch of things that are dangerous.
Two, once something makes it to market, it’s kind of impossible to figure out if it works or not. This seems counterintuitive to most people, with more people being able to use a drug, surely it becomes much easier to figure out if it works or not? But, at least for the drugs we’re discovering these days, the effect size tends to be small and the variance is large. Without a proper control arm, there are just too many factors at play to really tell what is going on. Someone takes a pill and the next day their cancer suddenly goes into remission? More likely to be chance than the pill.
We see this with drugs approved before the efficacy requirement. The FDA put some incentives in place to get drug manufacturers to run clinical trials on old drugs without patent protection that “everyone knows” work. Yet, a lot of the time when the trial is run, we find out that no, no they don’t
Three, it is hard to recruit patients into trials for a drug that is already on the market. Why take a 50/50 chance you’ll get a placebo, if you can just get the drug directly? This slows recruiting, and sometimes makes doing a trial impossible.
From a utilitarian point of view, it isn’t clear that in the long run this would have the best outcomes. I think we end up in a world with less knowledge about what works and less ability to iterate on it. It also isn’t clear that in the short run we’d be that much better off either. People love to bring up the vaccines but, in the very early days, which horse would you have put your money on? The mRNA vaccines that had a twenty year history of failure? Or one of the other 50 or so candidates?
I’m not sure how rare these breakthrough cases are, and it isn’t clear to me that they are being comprehensively tracked. I know of several people that have confirmed cases of Covid post vaccination, many that got it in just the last couple of weeks. I just spoke with one young woman yesterday who had a pretty bad case that knocked her out for a few days, she said she was surprised that her test (which was by mail) didn’t ask anything about her vaccination status. Another friend had her 2nd case of Covid, where she got symptoms exactly two weeks after her second dose. Luckily for her, the second time wasn’t nearly as bad, the first time she was very sick for a full month.
I’ve talked with others that had symptoms of Covid, including loss of smell and taste, but didn’t get tested. That’s mostly because of how hard it is to still get a test. When my wife and I had Covid a couple of months ago (more than a week after we got our first shot) the only way to get an in person test at our local clinics was to go out in freezing weather at 7am and wait two hours for the clinic to open to get your test appointment for later in the day. Just the kind of thing you want to do when you have a high fever and can barely get out of bed.
Alan Goldhammer
Aug 5 2021 at 11:47am
I pretty much agree with Dylan here. I was frustrated last year when the New England Journal of Medicine rejected a Letter to the Editor regarding a national vaccine database for COVID-19
Alan Goldhammer
Aug 5 2021 at 11:52am
Sorry, I hit submit comment by accident!!
I proposed the database for tracking adverse vaccine events, vaccine efficacy, and breakthrough infections. It would have made it easy to track this stuff and figure out which vaccine was performing best. Too bad it didn’t get published.
With the exception of a few years working at NIH, I spent my whole career in the biopharma industry doing regulatory affairs and drug safety. Whether this makes me a candidate for confirmation bias, judge for your self. I’ve known many past and present senior officials in both the vaccine and drug centers and believe them to be hard working and doing the right thing. It is a very difficult job because they get hammered from all sides regardless of the decision they make. The libertarian alternative of let people decide for themselves maybe works of 0.001% of the US population who have enough intelligence to make those decisions. The rest would be at serious risk were the FDA not around.
My view on Scott Alexander is that he is a good polemicist but a poor critical thinker.
Max More
Aug 5 2021 at 1:13pm
Scott Alexander is one of the best, most critical thinkers I’ve come across. There is also a lot of strong work demonstrating the terribly poor incentives for decision making in the FDA. The fact that you think Scott is a bad critical thinker says to me that you are too fixed to your favorable view of the FDA and/or you haven’t read anything else by Scott.
robc
Aug 5 2021 at 6:56pm
3000 people? Yeah, you greatly underestimate the American people.
Scott Sumner
Aug 5 2021 at 12:06pm
I think Scott would agree with you that most drugs don’t work, and indeed he starts off being skeptical of whether the new Alzheimers drug actually works. So I don’t think you disagree on that point.
The basic problem with the new Alzheimers drug is its cost to the taxpayer, otherwise I wouldn’t even care if they sold an ineffective product. Scott recognized the same problem.
Nothing that has happened with “breakthrough cases” of Covid is in any way inconsistent with what we expected based on the studies done in 2020. We knew these vaccines were roughly 90% effective, not 100%. And we still believe that to be true. Many news stories have created confusion among the public about whether vaccines slow the transmission of Covid. It still appears that they do.
Todd Kreider
Aug 5 2021 at 11:54pm
July 23, 2021: “Israel says Pfizer Covid vaccine is just 39% effective as delta spreads, but still prevents severe illness” The study showed 88% effectiveness against hospitalization and 91% against severe illness.
Yesterday: “People who are fully vaccinated with a two-dose coronavirus vaccine have a 50% to 60% reduced risk of being infected with the delta variant, even asymptomatically, compared with unvaccinated people, according to a new study conducted in England.”
Dylan
Aug 7 2021 at 10:41am
After reading Scott’s follow-up post that Peter links to below, I think our disagreement is slight. Scott characterizes the FDA as being the name for the giant hurdle that stands between patients and effective and safe drugs. I’d characterize it as our limited knowledge of biology and the fact that we don’t know how to figure out what works without doing really long and expensive RCTs. That‘s the hurdle that I want to figure out how to make a lot smaller. If we do that, reforming the FDA will be higher on my agenda. At the moment, I tend to think it does more good than harm.
Also, after reading more about the fish oil case, one that I was not familiar with before, if a well-powered RCT came along and I had to bet one way or the other on the outcome, I’d bet on it not being effective. I hope I’m wrong, but too much experience has told me that drugs that “everyone knows” work, don’t seem to when you look at them more rigorously.
Christophe Biocca
Aug 5 2021 at 6:05pm
I’m not sure how to parse this. My understanding is that the mRNA approach (which can be used for vaccination but also for just about any process where we get the body’s own cells to produce specific proteins) has had a bumpy road, but the issues we still don’t know how to fix (like the risk of triggering an immune response with frequent use), didn’t apply to vaccines, which is precisely why companies trying to commercialize the technology focused on that application starting around 2018. Moderna’s got Flu and Zika vaccine trials underway now. I have not seen any failed vaccine trials using mRNA, but perhaps you can enlighten me.
Unproven would be the more defensible statement. But IIRC most of the vaccines turned out to have some protective effect, though not always enough to hit the 50% threshold to qualify for guaranteed dose orders. So even a monkey with a dartboard could reliably have made a better-than-no-vaccination decision if all candidates had been allowed onto the market.
Dylan
Aug 5 2021 at 7:21pm
Failure is probably too strong a word, however, mRNA vaccines had been under investigation since the 90s, and the few, small clinical trials that had been done had not shown much efficacy. Here’s a review from 2019 that goes over the state of the field. Things were very encouraging by that point, as several challenges appeared to be solved, yet there still wasn’t clinical support to go with it.
Phil H
Aug 6 2021 at 1:43am
Yeah… I like Scott Alexander a lot, and this essay of his would persuade me if anything would. But I’m still stuck on facts of global and American drug development: more new (and effective) drugs get invented in the USA than anywhere else. Every proposal to radically change the way the FDA or any other part of the ecosystem represents implicitly a claim of this nature: “I see the greatest production system for drugs the world has ever seen, and I know how to do better.”
In other contexts we know that imposing strict rules and forcing people to play a somewhat arbitrary game is often a spur to creativity, because it enables direct comparison of things. In medicine, those arbitrary rules look horrifying because they are arbitrarily obstructing people’s access to healthcare. But the gamification seems to be working. Tweak the game, sure, but removing a major plank of it seems reckless.
robc
Aug 6 2021 at 9:30am
Small tweak.
Eliminate the FDA and replace it with a Kosher style system.
Some of the organizations might be even tighter than the FDA. Some would be about the same, some much looser. Insurance companies would choose which ones they consider acceptable, and would only cover drugs approved by those orgs. Same for hospitals and pharmacies and etc.
If you didn’t care about any of them, you could sign a waiver and buy direct with cash from the drug company.
Phil H
Aug 6 2021 at 9:46am
My position is always this: You have to make a complete argument. Not just “The FDA has a bunch of problems” but “The FDA has a bunch of problems my proposed solution would work better because X.”
A “kosher style system” has no such complete argument because no one knows how often the kosher system has failed, because when the kosher system fails there are zero consequences. No consequences, no records, no idea.
I’m making a consequentialist argument, an argument from efficacy. And there is no reason to think that kind of system would be more effective. (At least, I can’t think of one, and you didn’t give one.)
robc
Aug 6 2021 at 11:02am
I am making a deontological argument, but even from a consequentialist POV, how can you say that there are no consequences to failure from the kosher system?
There obviously must be some effect — people wouldn’t keep kosher if they didn’t find some benefit from it, possibly it is improvement to the soul. Hard to measure, but it is a consequence.
But if Kosher doesn’t work for you, how about Underwriters Lab?
Phil H
Aug 7 2021 at 4:32am
Sure, and the deontological argument is considerable and relevant; I just wasn’t addressing it with that comment.
I looked up Underwriters Lab, and they seem to be safety consultants. I don’t understand the argument: doesn’t everyone know that government health & safety regs have gone mad? I thought that passed for common knowledge these days. I don’t doubt that UL offer good safety. But the private sector can’t claim credit for the widespread improvements in industrial safety that have occurred, not if government safety rules are terrible and onerous, as the writers of this website would frequently have me believe.
I see few examples of widespead big changes in industrial practice without government intervention. Medicine doesn’t get from snake oil to effective without the law; industrial safety doesn’t go from satanic mills to decent workplaces without unions and the law; vehicle emissions don’t improve without the law; etc., etc.
robc
Aug 7 2021 at 11:06pm
Phil H,
In all of those cases, government lagged the private sector.
And unions are a tool of the private sector, so count towards my argument.
Matthias
Aug 6 2021 at 10:15am
You can replace the current FDA with the pre-1960s FDA (the one that didn’t check for efficacy), and your critique doesn’t apply: it’s a known system that worked better.
Phil H
Aug 7 2021 at 4:41am
I don’t know enough about the history to know if that’s right, but it’s not obvious to me. Was the USA the world’s leading developer of new drugs back then as well? I’m not sure the same set of conditions held at all.
Peter McCluskey
Aug 6 2021 at 4:59pm
>I see the greatest production system for drugs the world has ever seen
Eroom’s law (https://en.wikipedia.org/wiki/Eroom%27s_law) is evidence that it’s not as good as it used to be.
You’re correct that Scott Alexander mistakenly implies that the FDA is worse than the equivalents that exist in other major countries. He partly retracts that in a follow-up post, suggesting the difference between countries is pretty random for each treatment: https://astralcodexten.substack.com/p/details-of-the-infant-fish-oil-story
KevinDC
Aug 6 2021 at 9:25pm
True, but I don’t see any reason to connect that with the workings of the FDA in particular, and you don’t seem to offer any arguments for why anyone else should either. The US is where the most medical innovation occurs, and the US has the FDA, but that by itself doesn’t say much of anything, or give any reason to give credit to the FDA in particular. For all that shows, the innovation levels we see are in spite of the FDA, rather than because of it. As it happens, healthcare economics is an area where I know a thing or two (it is my job, after all!), and the research doesn’t support the connection you seen to be implying.
This is a topic that’s had some pretty extensive scholarship put into it, and the consensus for why the US is where more medical innovation occurs is because, compared to the rest of the world, the US has fewer price regulations on drugs and procedures, especially newly developed ones, so it’s basically the place where you’re actually most likely to be able to recoup the investment costs that went into developing those drugs and procedures. You can find studies to that effect here, or here, or here, or here, or here, or here, or here, as a small slice. I’m not aware of any scholarship which gives credit to the (allegedly) unusually high competence of the FDA relative to equivalent agencies elsewhere in the world.
Greg Jaxon
Aug 14 2021 at 12:47pm
The medium term world may hold less of the binary (low entropy) knowledge you’d like to acquire clinically, but it is chock full of higher entropy information that will in the long term form into an objective set of facts. Markets are a distributed learning technology. Whether their apparent inefficiency matters or not will depend on how one assesses the costs of the different kinds of experiments each approach runs.
Doing proper science might seem to cost more in an unregulated market, but to think so, you must ignore the hidden costs of regulation needed to perform the pristine experiments of a rational central planner.
MarkLouis
Aug 5 2021 at 8:11am
Media strikes me as a weakness of capitalism. People demand sensationalized, partisan, overly-simplistic news. And the private sector is more than happy to provide it. The problem is getting worse, not better. Not sure what the fix is because state intervention surely wouldn’t help.
Jon Murphy
Aug 5 2021 at 8:26am
You get the same thing in state-run media as well. The BBC and Pravda are just as sensationalist. I think it’s a human thing rather than an economic thing. The question becomes how to minimize it.
Lizard Man
Aug 5 2021 at 10:39am
Is that true of PRC media? My proficiency in Mandarin is very limited, but my impression is that generally the PRC wants news to be boring, because it doesn’t want its citizens to get emotionally stirred up, as emotionally stirred up people are harder to manage. Maybe it is also the case that outraged people tend to think of the outrages that they themselves have suffered, and that isn’t good for social stability either.
Scott Sumner
Aug 5 2021 at 12:09pm
PRC news is far, far worse. Far more inaccurate than the NYT or WSJ.
There are English language versions available online if you don’t believe me.
The only exception is the South China Morning Post, which is still semi-respectable at times (but for how long?)
Jon Murphy
Aug 5 2021 at 1:00pm
This post reflects what I think is one of the key insights of Public Choice economics and Expert Failure that is often lost in discussions of public policy: both benefits and costs are subjective. We need to consider the incentive structures to get an understanding of the actual costs and benefits the public choosers are facing. Discussions of “NPV” and this-or-that type of analysis are of limited use without considering the political economy of the choosers.
Thomas Lee Hutcheson
Aug 5 2021 at 9:49pm
Agreed. How do use Public Choice Theory (or some other theory) to adjust the incentives of the decision makers so that they design NPV maximizing rules or at least marginal improvements in that direction?
Max More
Aug 5 2021 at 1:21pm
It’s easy to become deeply discouraged and gloomy about the state of politics, of stupid thinking, of institutional biases and the apparently impossibility of changing them for the better. But there are way to improve things, although none are easy.
I definitely favor abolishing the FDA and replacing it with individual and institutional choice under sensible liability law. But there are less drastic steps. One would be to repeal the 1962 laws that greatly expanded the FDA’s power. Another would be to require the FDA to accept drug and device approval from similar agencies in certain other countries (pretty much anywhere in Europe). (Reciprocity.) 100,000 people died in the US because the FDA stopped beta blockers that were approved in Europe.
Also: Legally require the FDA to use cost-benefit analysis. Ban the use of the precautionary principle. (I would suggest the proactionary principle instead.) Annual audit of FDA decision processes.
The results must be published. Auditors to include people from organizations of varying views. expansion of right-to-try. Rolling approvals.
David Seltzer
Aug 6 2021 at 11:17am
Sorry, but all the hand wringing over a problem so endemic may require another way to solve this. This is Kafka’s The Trial writ large. Think about the non-violent resistance used by Gandhi And MLK. The cost is high, the risks are great but I suspect the benefits are greater.
Comments are closed.