Cowen and Hanson’s “Are Disagreements Honest?” summarizes the theoretical case against “agreeing to disagree.”

Yet according to well-known theory, such honest disagreement is impossible. Robert Aumann (1976) first developed general results about the irrationality of “agreeing to disagree.” He showed that if two or more Bayesians would believe the same thing given the same information (i.e., have “common priors”), and if they are mutually aware of each other’s opinions (i.e., have “common knowledge”), then those individuals cannot knowingly disagree. Merely knowing someone else’s opinion provides a powerful summary of everything that person knows, powerful enough to eliminate any differences of opinion due to differing information.

Aumann’s impossibility result required many strong assumptions, and so it seemed to have little empirical relevance. But further research has found that similar results hold when many of Aumann’s assumptions are relaxed to be more empirically relevant. His results are robust because they are based on the simple idea that when seeking to estimate the truth, you should realize you might be wrong; others may well know things that you do not.

It’s hard to find real human beings who reason this way.  What about fictional characters?  One suddenly came to mind tonight: Winston Smith from George Orwell’s 1984.  Smith is admirably meta-rational while brilliant Thought Policeman O’Brien intellectually bullies him.

O’Brien was a being in all ways larger than himself. There was no
idea that he had ever had, or could have, that O’Brien had not long ago
known, examined, and rejected. His mind CONTAINED Winston’s mind. But
in that case how could it be true that O’Brien was mad? It must be he,
Winston, who was mad.

As far as I understand, Aumann’s theorem only applies if both Smith and O’Brien are meta-rational.  Otherwise, Smith is epistemically selling himself short.  And the very fact that O’Brien combines intellectual argument with physical torture conveys extra negative information about O’Brien’s credibility.  What’s striking, though, isn’t that Smith takes Aumannian reasoning too far, but that he applies this reasoning in the first place.