One of the first lessons you learn in statistics is to discount first-hand experience. “But I knew a guy who…” is weak evidence, for a long list of reasons:
1. Random error. When you only sample one person – yourself – there’s immense random error. The noise can easily drowns the signal.
2. Selection bias. Are you an exactly average human? Probably not. In fact, exactly average humans probably simply don’t exist. As a result, your first-hand experience systematically misrepresents reality. And the less you resemble this average human, the greater the systematic bias.
3. Availability bias. You are more likely to remember extreme, vivid events, so even if your first-hand experience were representative, beliefs based on your first-hand experience could still be seriously biased.
While these are all reasonable concerns, they dodge a critical question: “Compared to what?” When someone sets aside their first-hand experience, statisticians are hoping – perhaps even assuming – that they rely on random sampling instead. In the real world, however, almost no one does this. For the vast majority of human beings, the alternative to first-hand experience is not statistics, but news. And compared to news, first-hand experience is ultra-reliable, for a long list of reasons.
1. Random error. Since the news is a vast industry, this might seem like a minor problem. Due to severe media herding, however, the problem remains severe. Journalists are not independent draws, but echoes in a vast echo chamber.
2. Selection bias. Journalists are far from average humans. They are highly-educated and highly-left-wing. Even more importantly, they are desperately trying to grab people’s attention with shocking anecdotes and images. What’s more, they have impressive resources to hunt down these shocking anecdotes and images. The upshot is that media selection bias is literally off the charts. What they choose to show is outside the first-hand experience all humans on Earth. By which I mean that zero humans have personally experienced all – or even a tiny sliver – of the horrors on the news.
3. Availability bias. After filtering reality through the biases of their ideology and need to grab people’s attention, journalists take the distillate and run it through yet another filter: their own memories. So when they bring up old stories, or provide context for new stories, they are piling bias on bias.
As you may have heard, when you see moonshine marked “XXX,” this means that the liquid has been filtered three times. Each filtration raises the alcohol content. This is a fine metaphor for the media. Journalists filter their experience over and over until they have a final product strong enough to make you blind.
By comparison, then, first-hand experience is a fountain of truth. If statisticians tell you to fear something you’ve never experienced during decades of life, you may want to consider the possibility that you’ve led a charmed life. If the media tells you the same thing, however, the wise response is to roll your eyes and rely on your first-hand experience. While you’re not an average human, your first-hand experience almost certainty tells you that racism is rare, serious crime is ultra-rare, that terrorism is basically non-existent, and that the vast majority of people in rich countries are materially prosperous. The media are in no position to “correct” you – or anyone, really. Politics aside, they are practically the most biased source on Earth.
READER COMMENTS
astew
Jul 9 2021 at 4:46pm
When was the last time some random person approached you (also a random person) with a wild story that is relevant to some political narrative that they just had to tell you? Not recently? Consider that it happens to many media people all the time — especially if it’s something they’ve been reporting about recently.
Not only do media people have more resources to hunt down shocking anecdotes, people with shocking anecdotes actually seek them.
More importantly… you know, you can get rid of that watermark on this post’s banner image by actually paying for the stock photo 😂
john hare
Jul 9 2021 at 9:10pm
Actually I hear quite a number of stories that lack credibility. Not exactly from random strangers, but often from people I’ve known for minutes or a very few hour. Flat Earth, vaccines take 10 years, Covid is a government conspiracy to take over. I’ve collected dozens over the decades. People left, right, and center seem to have no trouble telling me their views. I’ve developed a fairly good Uh Huh response that doesn’t agree or start an argumentative diatribe.
suddyan
Jul 11 2021 at 8:16am
[…vaccines take 10 years, Covid is a government conspiracy to take over…]
Quite impressive how you have framed reasonable stances with such a blatantly derogatory slant.
For what it is worth, there are long-established scientific reasons why vaccine testing should take years. But you seem to imply people who rationally query the political override of exhaustive trials and the release of vaccines under (kindly note) only emergency authorisation should be categorised as deluded conspiracy theorists.
As to “a government conspiracy to take over,” the actions of governments are plain and visible to see. And of course they planned all their actions in their “cabinet meetings” and confidential “emergency advisory boards” and the like. “Never let the opportunity of a crisis go to waste” inevitably entails behind closed doors planning by politicians to maximise their interest (in other words, they are conspiring).
Critical thinkers can see what is happening, e.g.: https://www.dailymail.co.uk/debate/article-9776111/PETER-HITCHENS-think-freedom-youve-forgotten-life-like.html
Yet while you seem prone to treat people who are able to observe and think critically about what they see with haughty condescension, you appear to be the one who is so enraptured by the show being put on that he can not see past the superficial. “Oh look, here come the dancing girls! Pay no attention to the man behind the screen.”
By the way, the tone of your rhetoric does not endear you to civilised, polite conversation. So, if you feel I am not being fully polite, kindly consider what I am replying to.
Daniel Klein
Jul 9 2021 at 5:49pm
Tremendous post!
Alan Goldhammer
Jul 10 2021 at 7:23am
You might want to read the new book, “Noise: A Flaw in Human Judgement” by Kahneman, Sibony and Sunstein before jumping to this conclusion.
suddyan
Jul 11 2021 at 7:51am
[You might want to read the new book….]
Nah. Sunstein’s agenda shines through in everything he touches. Talk about bias!
Hope you didn’t fall for all of it….
Shane L
Jul 10 2021 at 10:13am
There’s a phenomenon in perceptions of crime that people tend to think society is plagued with rising crime – except in their own local area, which isn’t so bad. Perceptions of national crime rates derive from the news, perceptions of local crime rates derive from personal experience.
Scott Sumner
Jul 10 2021 at 12:57pm
Interestingly, my first hand experience does not suggest that “racism is rare”. I’ve seen hundreds of examples in the comment section of my blog. I suspect that racism is pretty common.
If by personal experience you mean “directed at me personally”, then yes, my experience is that it’s pretty rare.
robc
Jul 10 2021 at 1:11pm
Wouldn’t the comment section of your blog qualify as news instead of personal experience?
Mark Z
Jul 11 2021 at 5:19am
I assume he means “in real life.” Comments sections are selection bias on steroids. People ‘in real life’ don’t remotely resemble any online environment I’ve ever encountered, for better or worse, so there’s extraordinarily nonrandom sampling going on.
Brian
Jul 10 2021 at 3:06pm
Racism is not rare. Racism is the default mode of some boys. This condition happens after the original default when the boys were infants. The default is influenced by racist parents. When we are boys we know many boys. This means racism will be encountered often. It is uncertain how much those boys reform as they become adults.
Since the boys are evidence of racist parents, racism is more common than you would know by the racism of the boys alone.
Gil
Jul 12 2021 at 1:57pm
Just curious why you chose to use “boys” rather than “children”.
Michael Rulle
Jul 11 2021 at 10:45am
While there are obviously good studies which can provide useful information—-Bryan’s “compared to what” idea is superb. I have the same belief———primarily because I think each of us is an excellent sample—-even though we are a sample of one. And while compared to the “news” this seems obvious to me as well, in and of itself we are excellent samples. Plus, we can easily identify differences of beliefs (of facts and opinions) between ourselves and others.
Too bad few people have the patience to discuss those differences—even as we clearly observe them. When I observe other’s certainty as clearly wrong, I assume they think the same about me. What could be more interesting than to draw out what one’s differences are? Unfortunately we do not.
I doubt the media ever played this role—-but it would be a miracle if they ever tried.
James Richards
Jul 12 2021 at 3:56pm
” your first-hand experience almost certainty tells you that racism is rare,”
Brian,
How can you say that when your group is not the main target of racism? Since you discuss first hand experience a lot, let make ask you some questions to see if you are truly qualified to discuss racism based on your experiences.
How many times have you been in a room or sat a table with at least 10+ people and been the only White person there?
How many times have you hung out with at least 3+ Black people. I mean really hung out with them?
Have you ever been inside the house of one of your Black “friends” to eat lunch, dinner, watch a TV show, cookout, etc.?
Have you ever met a family member of one of your Black “friends”?
For the majority of White People, they can’t answer yes to any of those questions. If you haven’t spent quality time with Black people and other minorities, how can you be qualified to say that racism is rare based on your experience?
Did you know 75% of White People have zero Non-White friends? It’s true. How are those people getting “first hand experience” of the rareness of racism? It’s rare in their minds because they are rarely around Black people due to the segregated nature of America. How would you know about racism if 99% of your interactions are with other white people?
https://www.washingtonpost.com/news/wonk/wp/2014/11/27/three-quarters-of-whites-dont-have-any-non-white-friends-2/
Ben
Jul 12 2021 at 6:32pm
The Xs don’t stand for filtration; they stand for distillation. (Most moonshine is not filtered at all, though almost all conventional commercial spirits are.)
Blindness is not caused by moonshine with a high alcohol content; it’s caused by methanol. Methanol is removed (not added) by the distillation process.
For more useful information on moonshine, consult your local hillbilly.
Brian
Jul 17 2021 at 4:44pm
Today it was reported that the German Olympic soccer team walked off the field in protest of racism from the Honduras team. As you say, first hand experience is less biased than news. Here we have information of Torunarigha’s first hand experience.
Axios reported “Torunarigha has faced racism before.” Thus we have information of another first hand experience. If you talk to him you might find information of other first hand experiences. If you do it privately, it won’t be in the news. Given the gravity of the problem for individuals, racism is not rare enough. By “individuals” I mean billions of individuals. So the occurrence of racism is at least 2 times some factor to adjust for Torunarigha’s exceptionalism times billions. That does not seem rare.
Comments are closed.