Gerd Gigerenzer‘s Risk Savvy is strikingly consistent with Hansonian medical skepticism. One striking example: his cost-benefit analysis of prostate screening. His key analytical tool is the icon box.
An icon box brings transparency into health care. It can be used to communicate facts about screening, drugs, or any other treatment. The alternative treatments are placed directly next to each other in columns and both benefits and harms are shown. Most important, no misleading statistics such as survival rates and relative risks are allowed to enter the box. All information is given in plain frequencies.
Here’s what you get if you construct an icon box for prostate screening based on “all medical studies using the best evidence from randomized trials.” Each dot represents 1% of men 50 years and older over the course of ten years.
That’s right. Statistically speaking, prostate cancer screening is worthless. Over the course of ten years, 1-in-100 men dies of prostate cancer regardless of screening. 20-in-100 die for any reason, regardless of screening. The only difference: 2-in-100 screened men – and screened men alone – endure hellish treatments, and another 18-in-100 endure milder torments and a false alarm.
Details on the illusory benefits:
[I]s there any evidence that early detection reduces the number of deaths from prostate cancer? The answer is no: There was no difference… [I]s there evidence that detecting cancer at an early stage reduces the total number of deaths from any cause whatsoever? Again no.
Details on the grotesque costs:
There are two kinds of harms: for men without prostate cancer and for men with prostate cancer that is nonprogressive. When a man without cancer repeatedly has a high PSA level, doctors typically do a biopsy. But unlike a mammogram, a PSA test does not tell the doctor where to inject the needle. As a result, men are often subjected to the nightmare of multiple needle biopsies in search of a tumor that is not there in the first place. These false alarms occur frequently because many men without cancer have high PSA levels…
Men with nonprogressive prostate cancer suffer even more. If a biopsy showed any signs of cancer, most were pushed into unnecessary treatments, such as prostatectomy and radiation therapy… Between 20 and 70 percent of men who had no problems before treatment ended up incontinent or impotent for the rest of their lives.
Gigerenzer actually understates his case. Since 1-in-100 die of prostate cancer either way, we have to conclude that either (a) positive biopsies don’t lead to treatment, or (b) the treatments do not, on average, work. Since (a) is clearly wrong, (b) is the logical inference. If you have terminal prostate cancer, modern medicine won’t help you. It will however still hurt you during treatment, and quite possibly make you incontinent and/or impotent for your remaining years.
When faced with scary medical problems, retreating to “I have to trust my doctor” is very tempting. What Gigerenzer shows, though, is that this trust is misplaced. Relying on doctors is unreliable. Trust transparent statistics instead.
READER COMMENTS
Lawrence D'Anna
Aug 11 2015 at 2:51am
I kind of want to see the next significant digit in that “1 in 100”
David Jinkins
Aug 11 2015 at 3:05am
What about the obvious selection effect? Those that are screened are not a random sample of men. People with symptoms of prostate cancer are more likely to be screened. The relevant comparison is not “screened men” and “non-screened men”. It is “screened men” and “screened men, had they not been screened”.
Martin
Aug 11 2015 at 7:41am
Actually, that’s exactly what my doctor told me about screening for prostate cancer when I turned 50 and asked him about it. He said, he hadn’t had it done and he didn’t recommend it at all, unless there were some other significant symptoms of prostate cancer showing. So I guess you can trust your doctor, if you have the right doctor.
Pajser
Aug 11 2015 at 8:04am
I agree with both D’Anna and Jinkins criticism. Furthermore, why ten years period? It is not intended that prostate screening stops after ten years, and I expect that, longer the period of regular screening, relatively more cases will be detected early enough to make it beneficial.
Ben
Aug 11 2015 at 8:13am
Previous critiques seem on point to me. In addition, the icon box seems to ignore lifespan. Even if it were true that being screened does not decrease your risk of dying of prostate cancer at all, it could still be true that it would substantially prolong your lifespan (before you ultimately die of prostate cancer). Many people (although I am not among them) seem to have a “maximal lifespan at whatever cost” attitude, and would take the tradeoff of longer lifespan if they do in fact have prostate cancer, even when weighed against the lower quality of life they might experience due to false positives and unnecessary treatments if they don’t. Again, I am not among those who would accept that tradeoff; but people who would are common, and this icon box analysis seems to be worse than useless for them since it paves right over their value system.
Rich Berger
Aug 11 2015 at 8:15am
Not sure what David Jinkins is questioning. If the nonscreened have the same actual rate of prostate cancer as those who are screened, how are they different?
Brett Champion
Aug 11 2015 at 8:34am
One can presume that the medical profession has access to and is reading these same statistics. I would trust their ability to interpret the data better than I would someone not in the profession.
Dan ambrosino
Aug 11 2015 at 8:40am
[Comment removed pending confirmation of email address. Email the webmaster@econlib.org to request restoring this comment. A valid email address is required to post comments on EconLog and EconTalk.–Econlib Ed.]
John Thacker
Aug 11 2015 at 9:33am
So all previous commenters agree that the idea that all men should get a PSA screening upon turning 50 is worthless, and the only people that should get PSA screenings are those that have symptoms or strong risk factors? That is the position of Gigerenzer, Djulbegovic, and others. While people are framing it as a “critique” or disagreement, everyone really seems to be in agreement.
The studies cited and the book are arguing against “routine” PSA screenings. Thanks to this and other articles, expert recommendations have turned against routine PSA screenings, lowering the frequency at which they are recommended. That is a recent development, and not accepted by all physicians.
Except I would trust a statistician’s ability to interpret the data far better than I would a physician. Physicians in general are notoriously bad at interpreting statistics, because they have a lot of other skills required. There are physicians who are experts in reading statistics and who issue guidelines; many individual physicians ignore them. When your personal physician disagrees with the expert best practice, what do you go with?
Benjamin Djulbegovic, who authored the study recommending against routing PSA screening, is both a physician and an expert in statistics and risk. Surely by your argument you should trust him over the average medical doctor.
Ten years was the length of the randomized study performed. Ten year survival rate is typical in discussing cancer. I don’t understand your criticism at all; can you please elaborate? A “long period of regular screening” would not make it seem more beneficial; the number of extra cases detected would be equally balanced with extra false positives. In fact, a longer period of study would also result in people with false positives dying early of extra cancers caused by unnecessary radiation, surgery, CT scans, etc.
Mike W
Aug 11 2015 at 9:58am
It is probably not a good idea to get your information about prostate cancer from an economics blog.
I had a relatively high PSA…6…for a number of years combined with, according to three doctors, an only slightly enlarged prostate which caused some urinary issues (seemingly not major, weak stream and occasional urgency). Because there is no way prostate cancer can be detected other than by a biopsy (Martin: “unless there were some other significant symptoms of prostate cancer showing”…such symptoms would only appear after it was too late for treatment) my family doctor referred me to a urologist and he recommended a biopsy (he was one of the doctors who said my prostate seemed only slightly enlarged). There were other options: “watchful waiting”, to see if the trend in the PSA went up, and doing nothing because prostate cancer proceeds slowly and at my age (65) something else might take me first.
Because I am in good physical condition in all other areas I elected to do the biopsy. It’s an uncomfortable office procedure but no more so than a dentist visit and hardly a “nightmare of multiple needle biopsies”. I received the diagnosis of cancer throughout my prostate on December 31st of last year.
I elected to have the prostate removed and this is where the real choices come in…where and who to do the surgery. I wanted to have it done using the Da Vinci minimally invasive robotic surgical system. There are a number of local urologists that use the system but I wanted one that had done lots of these procedures. I ended up going out-of-state to large teaching hospital and a surgeon who had done more than 750 procedures.
During the surgery (Feb 9th) he found the cancer had progressed slightly outside the prostate capsule and so he “shaved” a portion of the nerve and had a biopsy done during the surgery to determine if he had gotten it all. This surgical experience and hospital capability is why I chose to go where I did. It appears now that he did get it all because my PSA has remained at zero. I am fully recovered from the surgery with no incontinence issues and only some slight sexual adjustments needed.
The advice I give my sons is: get the PSA test as part of regular checkups after 50…it’s better to know than not know. If a biopsy seems warranted, e.g., the PSA is trending up, get it…especially prior to age 65. If cancer is detected, deal with it.
Daniel Kuehn
Aug 11 2015 at 10:36am
Death is a very extreme bad outcome here. Anything about person-years or quality of life saved? If I were going to die of prostate cancer and could put it off even a modest amount to spend time with my family and prepare that would be a huge benefit of treatment, but it would still show up in that one dead guy the same as if it progressed quickly.
The other questions is why does this persist? Certainly doctors want to treat patients so they get paid, but the medical community is not statistically illiterate. What’s supposed to be going on here?
Jim Dow
Aug 11 2015 at 11:24am
I suppose I could read the paper, but the trouble with the icon box is that it is fairly course-grained (1% is the smallest probability) for issues involving death. If the treated group dies with a frequency of 0.5% and the untreated group dies with a frequency of 1.4% they both show up as one death. That difference in probabilities is meaningful for me.
I agree that the conclusion that treatment has no effect is disturbing, which I why you would, in addition, want to focus more closely on the treated group and actually look at what happened to them in terms of death and quality of life (as others were saying)
Hazel Meade
Aug 11 2015 at 12:21pm
Objection.
The icon box is for all men 50+, regardless of how old they are within that age cohort. It’s a huge age group. And it’s open ended on the “until death” side.
The prostate screening might make the difference between death at 56 and death at 90 and that would not show up in the statistic, because everyone in the “50+” age group is GOING to die of something.
I would like to see this broken down by death rate in the 50-60, 60-70, 70-80, and 80-90 age categories.
If it lowered the death rate in the 50-60 age group that would count as a win, to me.
Glenn
Aug 11 2015 at 12:22pm
Agreed with all above – the graphic in this post is (deliberately?) deceptive due to scale. My understanding is that – conditional on having prostate cancer – the difference in 10-year survival rates with and without aggressive treatment is on the order of 5% (with overall rates already near 90%). Given relative rarity of prostate cancer in the 50+ population, it is highly unlikely this change could be captured at the 1-in-100 level. On the other hand, the risk literature is highly suggestive that we care about probabilities of dieing at much finer levels of significance than that.
Whether or not that marginal gain in expected life time is worth the costs of treatment and the risks of false positive is certainly a serious question worthy of serious discussion.
But only a fool would conclude that the graphic discusses here is seriously informative.
Pajser
Aug 11 2015 at 1:02pm
John Thacker: “can you please elaborate?”
I can try. If research is done with same participants for 20 instead of 10 years, average age of the participant at screening would be higher. The probability that single screening of single individual detects cancer would be higher. If there is any reduction of mortality due to early detection, longer the period of regular screening, the benefit should be more pronounced.
The question “how many lives are saved in first ten years of screening” is less important. It is important “how many lives are saved with proposed regular screening from 50 to death.”
Ron M
Aug 11 2015 at 1:28pm
To the previous comments, I’d like to write that doctors do what doctors do until they are told different. If you think your doctor has all the relevant information is like trusting that someone pointing a gun at you saying there are no bullets, I’d rather check the information myself.
In my case the doctors wanted to remove my prostrate when I had a bad PSA test. Well that was 10 years ago and I am fine because the test gave a false positive because I had sex right before the test. Neither the doctors nor I at the time realized that having sex with multiple orgasms will raise your PSA test levels for a few days.
Same thing happened recently when my new doctor sprung a surprise PSA test on me. I told the doctor that I had sex that morning and he had no idea that it would raise the levels. Subsequently, the blood test showed an elevated PSA. He was not aware of common knowledge about the relationship of sex and elevated PSA levels. Furthermore he refuses to read the latest knowledge on the topic.
So if you are blindly trusting your doctor on this issue you are just asking for unnecessary pain and testing. In addition, I would trust an independent source way before I did a surgeon/doctor who went to school to do surgery and has bills to pay like the rest of us. I know some of think that doctors are above that but you are very wrong.
Rich Berger
Aug 11 2015 at 1:45pm
I am puzzled by the reactions to this graph. First, the assumption seems to be that the two groups are much different. Is that really true? Screening is a pretty normal thing as part of a physical, whether you believe that you are at higher risk or not. I would expect those that do not get screened are likely to be less diligent about their health and maybe less healthy in general. According to statistics from the CDC the rate is highest among AA men for example – is it likely that they are less likely to be screened? The incidence rates for all men seems to be about 150 per 100,000 or about 0.15%; for ten years this would be about 1.5% which is in the ball park for the one death in 100 over 10 years.
Of the 20 of 100 in the icon box who had some sort of treatment or biopsy, how many would have died without treatment? At most, 2? Prostate cancer seems to develop very slowly and it is possible to live for a long time with it.
If the groups are similar, it means that prostate cancer screening is almost worthless, causing unnecessary pain and disability, for no gain. As far as 100 being too small a metric, if you increased it to 1,000, the range could be 5-14.99. Big deal.
KLO
Aug 11 2015 at 2:57pm
Yes, the box could contain more information, but then it would be confusing to the average person.
The problem is simple: half of men over 60 have prostate cancer and most will not die from it. For lots of reasons, it is difficult for patients and doctors to resist treatment even though current treatment is minimally effective at prolonging life or improving quality of life.
The situation persists because: (a) there is a strong bias towards “doing something” in both patient and doctor; and (b) we improve treatments by subjecting patients to them and learning what works and what does not.
ThomasH
Aug 11 2015 at 5:56pm
There is no substitute for cost benefit analysis that takes account of both varying benefits (some kind of quality of life measure) and when those benefits accrue.
FWIW, a properly done biopsy (I’ve had 3) is no big deal. Radiation and chemotherapy (observed up close) are, and need to be justified by a substantial increase in QA longevity to be worth it.
The problem with screening may be more what is done in the aftermath.
Mark Bahner
Aug 11 2015 at 10:38pm
A PSA test is a blood sample and typically less than $100. That’s not a lot of pain, physically or financially.
Michael Tontchev
Aug 12 2015 at 2:59am
@David Jinkins
The data is from randomized controlled trials. If there’s any selection effect, it affects both groups, so that point is not relevant.
@Pajser
You asked “how many lives are saved with proposed regular screening from 50 to death.” But… wouldn’t this number necessarily be 0?
Comments are closed.