Recently, David Friedman posted a response to an argument from Michael Huemer about when one should defer to experts or attempt to figure out the truth of some issue directly. David Friedman argued more in the direction of working out the truth directly, while Huemer seemed to argue more in favor of deferring to experts. There may be less disagreement between the two of them in principle than it seemed at first – in the comments, they both make some quick caveats and clarifications that seem to narrow the apparent gap in their views quite a bit.
Still, there’s one heuristic I think is worthwhile to add to this conversation. Sometimes, experts will disagree with each other, and we might ourselves not have the knowledge needed to properly evaluate which one is more likely to be correct. In those cases, what should we do?
For example, let’s say you wanted to know as much as possible about how to mitigate the effects of aging and to live longer. Right now, two of the biggest names in longevity research are Dr. David Sinclair, author of Lifespan, and Dr. Peter Attia, the author of Outlive. Let’s say I want to know how to best live a longer, healthier life. Both of these men are about as well-educated on this topic as anyone can be at this point, and their level of relevant knowledge vastly exceeds my own, so I read their books looking for advice. On the topic of how to eat, David Sinclair argues that it’s very important to limit the amount of protein in your diet. Meanwhile, Peter Attia argues that it’s very important to have a high protein diet – eating far more protein that the standard recommended daily allowance guidelines show.
Okay, so now we have two experts who offer contradictory advice. I’m in no way an expert in nutrition science, and I’m not likely to become one either. In this case, is there some heuristic I can use to decide which of them is more likely to be correct?
I believe there is, and in this case, it points me in favor of Peter Attia. When this kind of situation arises, my usual response is to lean towards the person who is making the more modest claim. David Sinclair’s claims are quite extravagant – the subtitle of his book is “Why We Age – And Why We Don’t Have To.” He argues that aging is optional and can be halted or even reversed – which is a very, very strong claim. Peter Attia, by contrast, makes the much more modest claim that we can slow the effects of aging, modestly increasing our lifespan and spend our last years healthier and with greater control of our faculties than we otherwise would. For example, in his own case, he doesn’t think it’s in the cards for him to live to 100, but he thinks that the dietary and lifestyle choices he recommends might help him live 8 to 10 years longer than he otherwise would have and will make his quality of life during his final decade much higher than it otherwise would be. This makes me far more inclined to assume that Peter Attia’s advice is correct.
This is basically operating in the spirit of Bayes Theorem about prior probabilities, and Carl Sagan’s dictum that extraordinary claims require extraordinary evidence. Almost by definition, an extraordinary claim just is a claim with a low prior probability. If two experts with vastly greater knowledge than me are arguing for opposing positions, and if the arguments and evidence they offer seem equally strong to me, then I rule in favor of the one that started with the more modest claim – that is, the claim that started out with a higher prior probability.
Is this a guarantee of accuracy? No, of course not – that’s why it’s just a heuristic. But I still think it’s a good tool, one that will point you in the right direction more often than not.
READER COMMENTS
steve
Feb 25 2024 at 1:07pm
I dont think that’s a bad approach. At the very least it helps if you eliminate people making hard to believe claims unless they have very solid evidence. Neither of them have especially good evidence, just good arguments and partial evidence. However, more broadly I think you should be pretty skeptical about all diet studies. They mostly fail as fad after fad has been proven wrong. Eat in moderation, do your own cooking with real ingredients when you can, exercise, sleep well, spend time with family and friends. Maybe at the margins specific stuff might make healthier.
More broadly, I am bit negative now. It’s not that hard to find people with supposedly good credentials making polar opposite claims. Both sides of an issue will have YouTubes up with high production values, often with statistics that seem convincing. If you have an in depth understanding of an issue and a good background in statistics you can dispute one side but then it looks like you are making an appeal to authority. I think that if you are truly open minded you can discern, but only with effort, what is correct, but at least in the US we have become so tribal that not many people can do that.
Steve
Kevin Corcoran
Feb 26 2024 at 10:28am
Yes, I think this is basically correct, and in fairness I should point out that Attia says much the same in his book. He goes out of his way to stress that nutrition science is a giant muddle right now. He apparently (only half jokingly) tried to convince his publisher that the chapter for nutrition should be basically one page with about a half-dozen, one sentence bullet points, because that’s all it takes to accurately sum up as much as information we can hold with high confidence. He of course hits those basics, but he also goes out of his way to stress that everything else in the chapter is and should be treated as extremely tentative and open to update, refinement, and revision, or even reversal.
Jon Murphy
Feb 25 2024 at 1:26pm
How effective experts are in their role as advisors depends on their institutional constraints. In a forthcoming working paper, my coauthors and I show that when the advisee (non-expert) is actively involved in the process, the expert’s advice becomes considerably better. Indeed, one thing I show in a seperate (also forthcoming) working paper is that part of the reason there was so much misinformation during COVID is because of the institutional structure of experts during the response.
In short, the discussion about deference or not toward experts is irrelevant without a conversation about how experts are organized.
Thomas L Hutcheson
Feb 27 2024 at 3:00pm
Maybe there is some overlap, but I’d stress experts speaking on the area of their expertise or not.
PH experts on how disease spreads and benefits of reducing the spread is fine. PH experts on which measures with which costs should be put in place to reduce spread is not fine.
Climate scientists on how CO2 emissions cause weather events, sea level rise, etc. is fine
Climate scientists on the cost of weather events, sea level rise, etc. or the least cost ways of reducing CO2 emissions is not fine.
Mutatis mutandis.
Andre
Feb 25 2024 at 1:53pm
False dilemma fallacy.
Attia has a quack track record. Along with Gary Taubes, he set up NuSI initiative to prove that it is carbs that make people fat and that low carb diets burn calories better. In other words, they set out to prove their conclusion. Just another variant of Atkins (keto, et. al.), really. They personally pocketed millions of dollars through NuSI, raising money from the people wanting to prove that eating fats was better for you than carbs.
All they generated was a study that disproved their claim. The NIH researcher they hired (Kevin Hall) ran a controlled study with folks in controlled wards (thus unable to stray from the diets provided). The result? Carbs or fats made no difference.
So just because Attia is *now* making the less crazy claim, doesn’t mean he is correct, either.
Of course, just because he was a quack ten years ago doesn’t mean he remains a quack or that his current claims are wrong.
However, I think he’s still a quack. Older folks need sufficient protein, true, but protein as in the West is almost always consumed in excess: 17% of dietary calories on average vs. 8%-10% of calories actually needed. On average, therefore, most of the public is already on a high protein diet – which is almost always accompanied by deadly quantities of saturated fats, given the types of proteins people actually eat.
And it is this standard (high protein) diet that aggressively promotes many of the primary causes of death – heart disease, cancer, stroke, diabetes – that would be far less prevalent on adequate protein diets – and in fact were relatively rare in societies that didn’t eat the levels of protein we do.
TL;DR: Attia has long been telling people what they want to hear so he can make millions off them.
Jim Glass
Feb 27 2024 at 2:41am
Give Attia some relative credit. Taubes is still quacking the same line. No number of controlled studies will ever reform him.
Andre
Feb 25 2024 at 1:54pm
Sorry about the formatting. Thought I had put in line breaks.
Grand Rapids Mike
Feb 25 2024 at 2:35pm
Another more basic area where experts disagree is the value of taking baby aspirin. Studies have shown opposite conclusions, with rather large sample studies. So one doctor recommended I stop taking baby aspirin and another recommended I should take it.
In Covid, political/institutional biases really played a huge role in all aspects of responses, leading to a greater distrust of expects.
For overall health, as already said, balance in eating, drinking and life style are important and its very clear smoking is a life reducer, tragically so. Also will add, with the growing acceptance and availability of cannabis in it various forms, wonder how long it will take for the alarm bells to start, or when information on its tragic consequences are allowed to filter out.
Thomas L Hutcheson
Feb 25 2024 at 11:01pm
Another heuristic: are the recommendations within the area of the person’s expertise. The COVID “experts” were making recommendations about NPI, when they could not know the cost of the measures. This made them suspect in that area.
Joel N Pollen
Feb 26 2024 at 4:43am
Consistent with what Steve said earlier, I think it may be that a better heuristic when two experts make diametrically opposing claims is to suspend judgment, in the assumption that both are overconfident. Often, incentives push experts in that direction (it’s difficult to convert your study of longevity into a bestselling book if your conclusion is that we don’t know much).
In this case, I happen to think that they’re both wrong. I have a moderate level of familiarity with their work, and both are prone to make strong statements where the evidence is pretty weak. We don’t really know whether high or low protein diets extend life in general, and the effect, if it even exists, is probably small.
robc
Feb 26 2024 at 8:59am
Its possible that both people are right. There may be local maxima of lifespan with both low meat and high meat diets, with a minima at moderation.
I have no reason to believe it is true, but it wouldn’t be the first time something like that was the case.
Jim Glass
Feb 27 2024 at 2:54am
Its possible that both people are right.
When “experts” argue for popularity and sales while both ignoring the solidly documented yet boring real expert mainstream view that contradicts them, it’s really much more likely they are both wrong.
“The Mafia killed JFK!” “No, the CIA killed JFK!”. Aw, Oswald did it alone.
Richard W Fulmer
Feb 26 2024 at 10:25am
Ways in which laymen can fact-check experts’ claims might include:
– Check their timelines (could Abraham Lincoln really have warned against believing everything that’s on the Internet?)
– Verify quotes
– Verify links
– Check claims for compatibility with the laws of nature such as the First and Second Laws of Thermodynamics.
– Check for claims that that money is being “left on the table” (if women really are paid %17 less than men for doing exactly the same jobs, then why are millions of entrepreneurs not hiring all-woman workforces?)
– Do the claims require thousands or hundreds of thousands of people to be “in on” a conspiracy?
– Are there discrepancies and contradictions?
– How likely are the claims?
– Are the experts employing any common logical fallacies?
– Do the numbers add up?
– Are the experts using sensational or misleading language or words and phrases designed to elicit emotions?
– Were quotes taken out of context?
– Can the claims be verified against other sources?
At least some of these checks could be performed by AI.
Jim Glass
Feb 27 2024 at 2:25am
When judging an “expert v expert” dispute, the first step is to check is if either of them actually is an expert. Neither Sinclair nor Attia has any professional expertise about protein. They have credentials on *other* subjects. That’s OK.
The next step is to find actual experts on the subject. While the Internet spews volcanoes of quackery and tribal invective about health issues, it also provides ready access to really top experts if one looks for them. When I went through this exercise for my own health some years ago I found the likes of world-known protein researchers Christopher Gardner, Stanford U; Donald Layman, U of Illinois; and Stuart Phillips, McGill U.
They all lead long-running university research groups on *protein*.
The third step is to see if the experts have a consensus among themselves on the subject, vis a vis the contradictory book writers. They do. Prof. Gardner here pretty clearly explains the basics of protein ….
‘… You must eat protein or die … the RDA of 0.8 grams per kg of lean body mass is sufficient for 97.5% of the population (excess for 97%) … bodybuilders and old people might need say 50% more … excess protein eaten is not used but turned into carbs and fat with no health problem other than that. AND the average American already eats twice the RDA, 1.6 grams, more than even ancient bodybuilders need! So for the average person today, eating a protein bar is pretty much the same as eating a candy bar with the same net calories. The entire “eat more protein for heath” sales industry is basically a huge scam…’ [Somehow, little of this gets into the general conversation.]
I have Attia’s book. It’s excellent except for the nutrition content. (“Most animals can’t get atherosclerosis.” What??) Peter’s admitted publicly that he’s flip-flopped on nutrition repeatedly and doesn’t like the topic. It shows. He’s a weight lifter and jock at heart, and admits it. As to all that extra protein he endorses, quoting Prof. Gardner:
I haven’t seen Sinclair’s book. My understanding is that he argues protein causes cancer and shortens life via mTOR activation. This is the “I think it could be, so it is!” fallacy. Maybe. But to be accepted, this must pass the tests of counterargument and empirical data. (If something is causing early death, people should be dying early!)
The counterargument as to mTOR and protein restriction is addressed by Dr. Layman: “Which do you want to take on first, the crappy epidemiology or the crappy animal studies?” [Them taken on.] As to the data, this long-term, large scale, diet-mortality study
finds: “Both high and low percentages of carbohydrate diets were associated with increased mortality” — a U-shaped curve with the balanced diet eaters living up to four years longer than the highest carb (lowest protein) and highest protein (lowest carb) eaters. A balanced diet. Go figure!
So both authors are wrong. If one pays attention to the real experts they have a strong consensus on these other things: “It’s the calories”, “It’s always the calories”, and “The poison is in the dose”. Layman pounds these at the link above. Sugar tastes good because it is good for you, a great energy source, until you eat too much, then it is terrible. Are eggs good or bad? Studies say both. It depends on if you are in caloric balance or not. Will protein put muscle or fat on you? Depends on how much you eat.
If you doubt all this, check out the Twinkie Diet. Prof. Mark Haub, Kansas State University, put himself on a Twinkies, Doritos and Oreos diet, lost 27 pounds and **improved all his blood markers**, got healthier! (“It’s the calories, stupid.“)
They also have a strong consensus that after calories, in spite of all of the fighting, in reality the big nutrition health issue is not protein v carbs, it is high quality protein & carbs vs low-quality protein & carbs.
And they agree that the public fights far too much about the small issues while slighting the few big ones. Attia did put this succinctly and well.
Jim Glass
Mar 14 2024 at 7:06pm
The verdict on Sinclair is in! Sinclair himself says: I’m incompetent! Trust me at your own cost! Because I’ll keep your money!
While his Harvard colleagues call him “the definition of a snake oil salesman”. Yikes!
The David Sinclair $720,000,000 Train Wreck!
Sinclair sold his “breakthrough longevity research” to Glaxo for $720 million (!!) in 2008. Guess what. I didn’t work!! Glaxo wanted its money back. Sinclair’s defense: ‘I did nothing wrong, you trusted my work. That’s on you. I didn’t defraud you. Look at all the mistakes I made, how incompetent I am. To commit fraud I’d have to be competent. So I keep the money!’
At the link above listen to him lie (to Atttia – at minute 8:13), and hear/read the statements (at minute 12:14) by Harvard Med School professor Matt Kaeberlien (Sinclair’s former research partner!)…
and former Dean, Jeffrey Flier…
So … there is the answer to your question. From Harvard itself. If one can trust Harvard these days. It seems to be having some credibility issues as an institution.
This is a little late. Maybe nobody else will read it (if you are). But for future reference, now you know.
Comments are closed.