Bryan Caplan makes a number of good points about straw manning and steel manning in his critique of philosopher Mike Huemer and in his 2015 post on the same issue. One of the major, and obvious points he makes is that you’re not arguing against a straw man if an actual person you’re arguing with is making the argument you’re arguing against.
I would put his overall point even more strongly. I’ll do it with an example.
Imagine that I’m arguing with someone on the minimum wage. The person claims that increases in the minimum wage will benefit people who are working at below the proposed minimum and will have no downsides for them. I point out that some of the least skilled workers will lose their jobs and even some or many of those who keep their jobs will lose other non-pecuniary benefits of the job.
I could go on to then make a more sophisticated argument, for example the monopsony argument, that the person “should have made.”
I don’t. Here’s why.
First, I’m trying to deal with people as they are. The person I’m arguing with has made presumably a good-faith argument for his position. If I then substitute another argument that he hasn’t made, it is, to some extent, a way of dismissing the idea that his argument is the one he really believes. It borders on bad faith on my part.
Second, arguments are ideally conversations. If I substitute a stronger argument for his bottom line–the minimum wage should be raised–I’m jumping ahead in the conversation without giving him a chance. I’ve short-circuited a conversation. I should instead let my argument sink in and see where he goes from there.
Third, by steel manning, I’m implicitly attributing bad faith to him. In raising the stronger argument for his bottom line, I’m implicitly assuming that he wanted an increase in the minimum wage regardless of the argument for it. I’m essentially saying, “Here’s a stronger argument for increasing the minimum wage because I know you want the government to do so and you are looking for arguments to justify its doing so.”
Fourth, people have a great ability to rationalize. It’s possible that the person I’m arguing with really does want the increase in the minimum wage and is trying out his best argument for it. If I give him a better argument for it, he might glom on to it. But I’m not inside his brain. I don’t know why he glommed on to it. Maybe the new argument makes sense to him or maybe he’s saying to himself, “Ah, good, I now have a better argument for the position I always wanted to believe in.” It’s better to let him sit with my original argument against his policy proposal and let him come up with a better argument. This reduces the likelihood of quick rationalization and helps him build his intellectual firepower. This actually dovetails with my second point above.
I vaguely recall a talk I gave about 20 years ago in which someone in the audience made an easy-to-refute argument against a deregulation proposal I had made. Unfortunately I don’t remember the venue, the topic, or the person’s argument. I started pointing out what was wrong with the argument. I saw the person obviously listening to me, even nodding his head as he was grokking what I was saying. It looked to be the start of a good conversation.
Unfortunately, a friend of mine was in the audience and he knew almost as well as I did what other arguments the audience member could have made. I say “unfortunately” because this friend was a strong believer in steel manning. He interrupted my conversation with the audience member and said words to the effect, “No, David, that guy’s argument is a weak one but there are better arguments. Why not deal with the strong ones?”
I usually read audiences well. I saw the original questioner kind of turn off and exit the conversation. My friend had interrupted what could have been a good conversation between the audience member and me. I can’t prove it, but I think that hurt the questioner’s intellectual growth.
READER COMMENTS
Jon Murphy
Jun 7 2021 at 3:52pm
Does your argument apply for teacher-student conversations as well? When students discuss research project or term paper ideas with me, I like to poke holes in it, so they know where to shore up their argument. But I will also suggest ways to strengthen their argument. For example, I recently discussed a research project with a student. After pointing out some (easily fixable) flaws with their argument, I suggested using a Tullockian Transitional Gains Trap argument to make their paper stronger.
Do you think that I may be doing my students a disservice?
David Henderson
Jun 7 2021 at 4:11pm
Two good questions.
I’m not sure of the answer. On your second question, I don’t think you’re doing your students a disservice. So somehow I need to resolve that contradiction.
KevinDC
Jun 7 2021 at 5:59pm
I think part of the issue here is there’s two possible meanings behind the concept of “steel-manning” and it’s important not to equivocate between them. You can steel-man a specific argument for a position, and you can steel-man a position itself. In the former case, if someone is making a particular argument, you should try to interpret every step of that argument as fairly and generously as possible. Paul Krugman gave a good description of this approach in his essay Ricardo’s Difficult Idea when he said:
This version of steel-manning is simply finding the strongest and most charitable interpretation of an argument which is still consistent with what your interlocuter actually said, rather than replacing their actual argument with a different, “better” argument. This seems appropriate when engaged in the kind of conversations David highlights. Call this “conversational steel-manning.”
Jon, by contrast, seems to be talking about steel-manning of the second sort, which is less about a conversation a specific person to understand the reason for their views on a specific issue, and more about outlining the strongest possible arguments for an issue in general. Call this “subject steel-manning.”
I think the student-instructor dynamic is more akin to encouraging subject steel-manning, rather than conversational steel-manning. An instructor isn’t simply there to steel man a student’s conversation – they are also there to help their students better understand the steel-manned arguments for a position.
Jon Murphy
Jun 8 2021 at 7:56am
I think that’s right. The student-professor relationship is different from the debater relationship or the conversation relationship. In the debate/conversation relationship, the two parties meet as equals. In the student-professor relationship, the professor is the comparative superior to the student. That is to say, the professor is the expert and the student is the leaner. The job of a professor is both to converse but also to teach, so it would make sense to nudge or steelman.
Philo
Jun 8 2021 at 10:42am
“An instructor isn’t simply there to steel man a student’s conversation – they are also there to help their students better understand the steel-manned arguments for a position.” Yes, both activities are appropriate, and not just in the professor-student context.
Evan Sherman
Jun 7 2021 at 4:52pm
On the one hand, I do agree that any specific claim or argument has no duty to be comprehensive. Too many critiques basically boil down to ‘You wrote about X, and you didn’t write about Y, and I wish you had written about Y. Also Y is related to X, so you’re arguing in bad faith.’ This sets an absurd requirement that literally no one can fufill, especially in blog posts.
On the other hand, I wonder if you have considered the distinction between the straw-man and the weak-man – often articulated by the more recently coined term “nutpicking”. In this fallacy/ugly rhetorical tactic, one does not construct a fictitious straw-man but rather knowingly selects only weak arguments for a position one opposes – and thus avoids accepting any fights with strong-men, or the strong arguments for a position one opposes. (Insert PC joke about strong-women-not-only-strong-men here.)
It’s always hard to say where responding organically stops and weak-manning, or nutpicking, begins, but the spectacle of someone constantly dunking on weak-men does eventually justify at least a suspicion that one is avoiding confrontations with strong-men.
Not to accuse you of weak-manning necessarily. Just worth pointing out the disctintion.
Evan Sherman
Jun 7 2021 at 4:56pm
Apologies – I read through the listed rationale against steel-manning but failed to read through the example, in which you do specifially mention weak-man vs. strong-man dichotomies.
Fair enough. I would propose that the internet is different, and you can very easily choose weak-men vs. strong-men, so the organic meat-space-style encounter with weak-men arguments (and showing respect for real people) is less relevant. But I at least see what you’re saying.
Dylan
Jun 7 2021 at 6:01pm
David,
Picking up on Evan’s comment, I’d like to respectfully suggest that you’ve “weak-manned” here as well, by which I mean I see you respond to comments where the arguments are fairly elementary, yet sometimes ignore the (seemingly) more sophisticated argument.
Not a knock, I know I’ve done that here as well, because the more sophisticated response usually takes more time to digest and understand, might even require me to go and read a paper and synthesize my thoughts. By the time I do all of that the conversation has likely moved on and it no longer feels worth replying to an old comment on a blog post.
However, a lot of times you will write on the same topics over and over, and it would be nice to get the idea that some of those were genuinely trying to steelman the opposing position a little better instead of knocking down the same weak(er) arguments.
One example is drug regulation and the FDA. You’ve written many posts on this over the couple of years I’ve been reading EconLog, yet none of the posts give the impression (to me at least) that you’re really steelmanning the other side, or that you’ve even talked to any experts in the field outside of your co-author, Charley Hooper.
David Henderson
Jun 8 2021 at 5:06pm
You wrote:
Your impression is incorrect.
But maybe we can advance the ball this way. Since I know this FDA issue interests you, tell me the 2 or 3 strong arguments for FDA requirements on safety and efficacy that you find most persuasive.
Dylan
Jun 8 2021 at 5:44pm
Hi David,
I appreciate the response and the offer. I think the recent approval of aducanumab gives a timely example, this is the Biogen drug for Alzheimer’s that didn’t show efficacy in their clinical trials and was approved despite the FDA advisory board recommending against it. Biogen stock was up something like 30% on the news, which means a lot of people think that Biogen will be able to sell a lot of a drug that doesn’t work, which gets at part of my objection to no efficacy requirement from the FDA (or comparable private organization)
The link goes to Derek Lowe’s post on the topic, and since he is far more knowledgeable on this topic than I am, I’m going to paraphrase his objections and I’d love your response.
1) Most drugs don’t work
2) Clinical trials are currently the only way we have to show that a drug is effective
3) Without a gatekeeper like the FDA – clinical trials (at least well designed and properly powered trials are unlikely to be run)
4) It is very easy to convince even experts that some drug might do something and propose a plausible mechanism of action for why it would be effective. So the market would likely be flooded with relatively safe, but ineffective drugs.
5) Going back to 2, doctors won’t be prescribing in anything like the uniform way they do in a clinical trial, so even if you have millions of doses of an unproven therapy given, it will be incredibly difficult or impossible to figure out what works and what doesn’t. Particularly when we’re talking about small effect sizes, like we are for most modern therapies which are evolutionary and not revolutionary.
Dylan
Jun 8 2021 at 6:22pm
Adding on a conclusion, since I was interrupted with a work call. I believe in previous exchanges either you or I have suggested that something outside of the FDA could step in and fill the role of gatekeeper, perhaps the insurance companies won’t reimburse unless we get proper proof of efficacy?
In theory, I don’t have a problem with that, but I do have a couple of objections:
1) We don’t have a lot of evidence of insurance companies paying all that much attention to efficacy right now in their reimbursement decisions. We have to wait and see on aducanumab, but there are other ineffective drugs that have been approved and that insurance companies have been pressured into covering.
2) If the insurance companies do step in and act as a gatekeeper, I don’t know that it is practically all that different from what we have now.
3) We already have a mechanism where substances that are generally regarded as safe but haven’t shown efficacy can be sold, that’s the dietary supplement market. $140B-worth of supplements were sold in 2020. Would the incentive structures for drugs in your world look markedly different than that for nutritional supplements?
David Henderson
Jun 8 2021 at 7:01pm
Thanks. Both of those help.
Charley Hooper
Jun 8 2021 at 8:22pm
Let me paraphrase your concern. If there are ineffective drugs on the market, patients will pay for the ineffective drugs, many will have false hopes, and all will suffer an opportunity cost because while the ineffective drug was being used, a better one wasn’t.
Now consider a specific, real example. A clinical trial shows that a drug for cancer moderately helps 23% of patients. If you were the FDA, would you consider this drug “efficacious” and approve it?
If your answer is yes, then aren’t 77% of patients who try it paying money for false hopes and missing out on better drugs?
My point is that efficacy isn’t black and white—it’s gray. You might consider a drug efficacious while I don’t. Plus, a large percentage of patients won’t be helped by the typical drug. For those patients, the drug isn’t efficacious for them.
Charley Hooper
Jun 8 2021 at 9:46pm
The pro-FDA thesis basically assumes that with the benefit of clinical trials, it is clear which drugs are safe or not and which drugs are efficacious or not. But it isn’t clear. Some patients will benefit. Some patients will have adverse events.
Whether we have the FDA or not, patients, in conjunction with their doctors, must try the drugs themselves to see if they benefit or have an adverse event.
It is as a result of this trial and error phase that patients and their doctors decide whether to keep taking a drug. The FDA and clinical trials are generally unhelpful at that point.
Dylan
Jun 8 2021 at 10:34pm
Charley,
Thanks for the reply. I think your paraphrase misses my key worry (my fault, I didn’t come out and say it). I’m less concerned with what an individual patient does and more concerned about advancing the state of knowledge so that we can make better and better drugs in the future and help more people. Given how little we understand of human biology, I don’t see how we can get there without running well-designed and controlled clinical trials.
I’m also not sure I get your example. Yes, human variability is a big issue, yet this seems to argue for clinical trials rather than against them. If we find that 23% of people appear to be helped by the drug, the natural next step is to try and see what those people have in common and then run a trial just in that sub-population and see if you can use that to predetermine who should get the drug.
Even if we can’t do that, isn’t it better to have a few options and know the results on average, than to not know anything? To have my doctor be able to tell me, 23% of patients had tumor size reduced by 30% with drug A, and with drug B we saw no response in most patients, but 10% had tumors go down by 90%?
The other option is having literally millions of options and only anecdotal evidence of any safety or efficacy. Look at what we saw with Covid and hydroxychloroquine. So many different dosing strategies and combinations and if you looked at anecdotal cases it seemed like it worked like gangbusters. Yet, when the clinical trials were run we didn’t see any of that. How do we not get a repeat of that on a massive scale if we move in your direction?
Dylan
Jun 9 2021 at 7:50am
My reply from last night seems to have got caught in moderation. So, I’ll just say here thanks for the response and note that my biggest concern isn’t about opportunity costs in terms of an individual patient it is more about making sure the incentive structures are right to advance scientific knowledge so that we can continue to make better and better drugs. At the moment, I don’t know how we do that without well-designed clinical trials.
Charley Hooper
Jun 9 2021 at 12:12pm
Dylan,
That’s what researchers try to do. The problem is, right now we don’t know why only 23% benefit. We often don’t have any hypotheses. What clinical trial would you run if you don’t even have a hypothesis why only some patients benefit?
You say “we don’t know anything” without clinical trials.
You are discounting the information that doctors get from their patients’ use of drugs. If a patient tries Drug A and her tumor shrinks by 80%, that’s good. If another patient takes Drug A and his tumor doesn’t shrink at all, that’s bad. That’s better information than the clinical trial data showing that 23% benefit—it is specific and actionable. It’s not all anecdotal.
Not all clinical trials are run by drug companies. At least one of the key clinical trials testing hydroxychloroquine for COVID-19 was funded by the National Heart, Lung, and Blood Institute, part of the NIH.
Even without the FDA around, drug companies would still run clinical trials. Clinical trial results are the bread and butter of pharmaceutical marketing. It’s how you convince a hospital, physician, or P&T committee that your drug has the potential to treat patients.
Everything has a cost and a benefit. “Advancing the state of knowledge” may be a benefit of the FDA approach, but what is the cost? What if the cost of that approach is half as many drugs that are themselves ten times as expensive?
Dylan
Jun 10 2021 at 8:05am
My reply to Charley is below, forgot to hit the reply button.
John Alcorn
Jun 7 2021 at 6:10pm
A note to clarify terms of art:
Michael Huemer draws a distinction between straw manning and weak manning.
Straw Man
Weak Man
If I understand correctly, David Henderson distinguishes and defends a 3rd stance in argument. Let me rephrase it to augment Prof. Huemer’s glossary:
Point Man. A position that your interlocutor states and that you rebut specifically. This good-faith practice shall be known as ‘point manning’.
Evan Sherman
Jun 7 2021 at 8:43pm
Great point about the point man.
For what it’s worth, though, I would propose that the conversation – and especially refernces to the point-man construct – thus far inadequately accounts for the mechanics of arguing on the internet vs. arguing IRL / in meat-space, if you will. In real life – for example, in a classroom with a finite and known number of participants, with all participants experiencing time together and at a normal pace – you can enounter claims organically and can simply respond to what you hear. There is even a procedural formality to IRL converstations that further governs the pace. On the internet, though – and especially for a D-level minor celebrity (i.e. A-level rock star celebrity in online libertarian circles! 🙂 ) – one encounters way too many arguments to which to respond. Also, on the internet, time is compressed and distorts the natural cadence of a conversation, so you don’t get to just respond to things organically. Someone like David might log in after a meal and see tons of comments. And remember, he’s a niche rock star, so he’s probably a little busy.
In that context, I would argue that it is not only possible but also necessary to pick and choose what and to whom one will respond. If one is not careful, then, it is natural to choose the path of least resistance: dunking on weak-men and side-steping strong-men. Not an accusation, but something about which we should all be careful. Especially minor celebrities.
Phil H
Jun 7 2021 at 7:48pm
I agree with DH that it’s better to address the arguments as made (though in public debate, worries about weak-manning are reasonable). Both straw manning and steel-manning are failures to take your interlocutor’s arguments seriously, and involve substituting your own judgement for theirs. The problem with that is that you assume you know all the arguments and reasons for disagreement in advance; but you probably don’t.
Mark Z
Jun 7 2021 at 9:56pm
I think steelmanning is still useful toward identifying the correct position, just not necessarily useful toward convincing a specific person; it’s also not terribly unlikely that one might find a better argument for a position than someone who actually endorses that position. If you’re looking for the right answer to a question, you shouldn’t limit yourself to arguments other people actually make, as there’s no guarantee that someone you know (or know of) has made the best argument for the position even if it’s correct.
Phil H
Jun 8 2021 at 5:25am
Yeah, I guess that’s fair. I just mean that it’s important to pay very careful attention to the arguments of people who disagree with you, rather than allowing yourself to focus exclusively on what you *think* the “best” argument might be. I mean, think about what the implicit positions are:
I believe in position X.
I believe that the best possible argument for some position not-X is A.
That means that I’ve already considered A, and I maintain my belief in X. So I know that A is not a strong enough argument to change my mind.
If I “steelman” my interlocutor and round his arguments up to A, I’m still not really entertaining the possibility that my belief in X is wrong. That is, by definition, you can’t steelman well enough to change your own mind (if you coulda, you woulda, and you wouldn’t be arguing X any more). So… what’s the point in it? It would be better, I think, to spend your time listening really closely to what other people are saying, because they might have a new and interesting take on the matter.
In a truly open, exploratory debate, of course there’s space for helping each other work out the best possible versions of each argument. But the listening has got to come first.
Philo
Jun 8 2021 at 12:24am
I do not agree that “by steel manning, [one is] implicitly attributing bad faith to him [the person whose argument is being steel-manned].” It may well be that your opponent did a poor job of expounding his own thinking, and really had the steel-manned argument vaguely in mind. He may *feel* that his conclusion is right without having correctly identified why he felt that way: hence he gave an argument that does not fully represent his (inarticulate) thinking on the matter.
If your opponent does not understand your refutation of his given argument, you might allow the discussion to linger over it (though some people are too thick-headed to make this worthwhile). But if your refutation is grasped, it is time to move the discussion forward–perhaps onto the more interesting and challenging argument for your opponent’s conclusion. And there will often be an audience to your debate–other people, to whom you want to convey your thinking. Having appreciated your refutation, they may be interested in further, profounder discussion of the issue.
So, after having briskly refuted the given argument, it is often appropriate to launch into steel-manning. This is not to accuse your opponent (implicitly) of anything more than having offered an easily refuted argument–something (alas!) we all do quite regularly.
Peter Gerdes
Jun 8 2021 at 5:53am
I tend to see steel-manning (in one on one convos…it has an additional role if you mean to persuade observers) as a rule of charity. It’s hard to understand each other and we are crazy biased to assume opponents’ args are wrong so if you at all can interpret them as saying something stronger do so as there is a good chance they meant that and just didn’t formulate it clearly
Yes, I agree you shouldn’t do it when you can’t even sorta interpret them as saying the stronger case but I don’t think most of the other critics about this issue would disagree.
TMC
Jun 8 2021 at 8:26am
“I can’t prove it, but I think that hurt the questioner’s intellectual growth.”
That’s the educator in you. Going right to the steelman is a way win the argument, but your way is more effective long term, and certainly more gentlemanly.
David Henderson
Jun 8 2021 at 5:07pm
Thanks.
zeke5123
Jun 8 2021 at 4:46pm
Seems to me I want to steelman not a particular person’s argument (i.e., when dealing with an interlocutor address his or her argument).
But if I am making a claim in eg a written article (i.e., X position is right or Y position is wrong), then I should steelman the best arguments against X or for Y. In this way, I am making sure that my position is more likely to be correct.
Stated differently, context matters.
David Henderson
Jun 8 2021 at 5:07pm
Well put. Context does matter.
MarkW
Jun 9 2021 at 12:55pm
One of the reasons to address the ‘weak’ arguments for high minimum wages is that it’s probably true that those are the arguments that are most persuasive to the general public. Most voters (even educated ones) would not recognize the word ‘monopsony’ and only a fraction of those would be able to explain it and how it relates to minimum wages. Depending on the audience, it seems to me that refuting the ‘weak’ (straightforward, appealing, but wrong) arguments is what’s needed.
Bill Drissel
Jun 9 2021 at 5:16pm
Dear Dr H:
I don’t think that Steelmanning is intended to furnish a better argument for opponents. I think that constructing a Steelman is intended to state the strongest possible set of arguments for all of your opponents so you won’t be unexpectedly overwhelmed by an argument you aren’t prepared to deal with.
Salesmen frequently study lists of “objections” that usually have tested answers intended to mitigate the objections and close the sale.
Dylan
Jun 10 2021 at 8:02am
Charley,
Appreciate you taking the time to reply, particularly in this space that is pretty off topic from what David was writing about. Your response I think helps me understand better where some of our differences in opinion lie, which is helpful.
It’s true, we don’t know why in a bunch of cases. We’re getting better at it all the time (in fact just last week data was presented by a company at ASCO that is promising of a biomarker that will help the clinical development of a company I’m advising). But, when we run a clinical trial and find out 23% of people benefit, we have useful data even if we don’t know why those particular people benefit. We can say you’ve got a 1 in 5 chance of seeing improvement if you take this drug. If you haven’t done those trials the doctor and patient are instead faced with a million+ treatments, and they have no idea on how to evaluate which one they should try.
But it is anecdotal. If a patient tries Drug A and their tumor shrinks by 80% that’s great, but we still have no idea if it was the drug that caused that. People go into spontaneous remission all the time for all sorts of reasons we don’t understand. That’s why we run clinical trials. I have trouble how you can think it is better information when we have no idea what would have happened if they hadn’t taken the drug?
Of course, and I expect those kinds of trials would continue. Drug companies don’t have an incentive to run trials for drugs that are no longer patent protected (at least generally, there are some FDA programs to give marketing exclusivity if you run trials on some older drugs). Other organizations and non-profits step in and fill the void to some extent.
I’m doubtful of this, and I’m even more doubtful that the trials would be run in as rigorous of a way. It’s pretty easy to run an underpowered trial that can hint at efficacy, while running less risk of outright failure. We don’t have to guess though. Biogen has been given 10 years to complete the post-marketing aducanumab trial that could show efficacy. That’s a lot longer than was needed for the Ph III trials. If you’re right, Biogen should rush to complete this trial to encourage doctors to prescribe it and payors to pay for it. Want to wager that this will be completed in 7 years instead of 10?
I think you’re underestimating how many drugs there would be without the gatekeeper (FDA or otherwise*). I think it would be far, far more, and they would be much less expensive. The patient experience would be worse though, because there wouldn’t be anything to guide them on which drug might have a benefit for them.
*If we were to banish the FDA role in efficacy monitoring here, but other countries kept their rules in place, I would imagine we would freeload to large extent on their efforts. That would probably still lead to fewer and smaller trials, just given how large the U.S. market is in pharma.
Charley Hooper
Jun 10 2021 at 8:06pm
Dylan,
You want empirical evidence that drugs work. I want empirical evidence, too. You want to avoid anecdotal evidence. I want to avoid anecdotal evidence, too.
Your hypothesis is something like: Without the FDA requiring that companies perform proper clinical trials, companies would skimp on clinical trials, launch ineffective products, and swamp doctors with therapy choices that were supported not by hard evidence but only by anecdotal evidence.
Presumably this hypothesis could be measured by the percentage of drugs on the market that are considered “efficacious,” however we would define that. With the FDA in place, the percentage of ineffective drugs would be much lower than in a situation without the FDA in place.
I’m genuinely curious. Do you have empirical evidence to support your hypothesis? Or is it based on anecdotal evidence?
Dylan
Jun 10 2021 at 9:37pm
Charley,
It’s a good question that I don’t have a great answer to. After spending 15 years in the industry, I’ve got plenty of anecdotal experience of people that should know better, still managing to fool themselves that drugs that don’t work will work, and they’ve thrown a whole lot of good money after bad to prove the fact. When it is that easy to fool yourself, I think fooling other people (sometimes people whose very life depends on it) that your drug works isn’t that difficult. And I think combining that with an analysis of the incentives gives a pretty good idea of how things would play out. But, that’s not empirical evidence.
On the empirical front, I think there are two possible sources of data, but I don’t know if they exist.
The Nutritional Supplement market. Clinical trials seem to be pretty rare here, and the ones that I’ve seen that exist are usually of very poor quality whose results can’t really be trusted. My impression is that the vast majority of these supplements don’t do what they purport to do, and may be harmful (I know of a few studies on daily multivitamins for example that point in this direction, but are not conclusive).
Evidence of drugs that have been in use prior to FDA efficacy requirements that are later found to not be effective. after a company sponsors clinical trials. I know this has happened, but I don’t know how frequently or what percentage of drugs. I’d be interested to see if there is data on this since the FDA started the program to incentivize companies to test older drugs for efficacy.
Charley Hooper
Jun 11 2021 at 12:41am
Dylan,
There is some empirical evidence suggesting that the “no FDA” case is pretty good.
The 1962 Kefauver-Harris Amendments add a proof-of-efficacy requirement to the already existing safety requirement. Economist Sam Peltzman, in 1973, analyzed drug approvals before the amendments took place and after. His finding? The number of new chemical entities approved dropped 61 percent from his pre-1962 projections. There should have been about 41.8 new approvals each year, but, instead, there were just 16.3. Further, Peltzman and other researchers concluded that it wasn’t just inferior drugs that the FDA’s process weeded out. Peltzman said it was “as…if an arbitrary marketing quota…had been placed on new drugs after 1962.”
According to this study, we ended up with roughly half as many new drug approvals and no, or only a very slight, reduction in the percentage of ineffective drugs.
Dylan
Jun 11 2021 at 8:39am
The Peltzman paper is good, I’ve read it before when David linked to it, and does provide some evidence to support your position. Do you know if there have been any follow-ups looking at the data or doing a cross-country comparison, since countries added in efficacy requirements at different times? I haven’t been able to find any, and surprisingly few cites for the Pelzman paper using Google Scholar (assuming I was doing my search correctly).
I would like to get your thoughts on the supplements market and in what ways you’d expect the pharma market to behave differently if we move to a world where the FDa is only responsible for basic safety (i.e. a Ph I trial). Is it the presence of patents? I don’t recall if you favor drug patents in your preferred regulatory regime?
Charley Hooper
Jun 11 2021 at 12:46pm
Dylan,
Using the fact that the 1962 amendments affected only American drug reviews, researchers were able to contrast the results in this country with those in other countries. Notably, researchers did not see a similar decline in new drug launches in the United Kingdom, suggesting that the root cause here was the new legislation. William Wardell, who coined the term “drug lag,” found that four times as many new drugs were launched in the U.K. as in the U.S. during the 1962-1971 period.
The supplements market is quite different from the Rx market. There’s no IP, no efficacy requirement, no pre-marketing approval, to prescriptions, and no claims made. I think everything starts with IP. What company would spend, say, $50 million over ten years to test resveratrol when, if the results are positive, the company would have a hard time charging more than the going rate for resveratrol capsules?
Dylan
Jun 11 2021 at 1:28pm
Thanks Charley. Appreciate the conversation. I’ll have to do some more reading and thinking.
Charley Hooper
Jun 11 2021 at 1:31pm
Thanks, Dylan. I appreciated the conversation. I’m going to do some more thinking on the topic.
Comments are closed.