EconLog reader Peter Hurley sent me some interesting information on testing the efficacy of debate strategies. Reprinted with Peter’s permission:
Hey
Prof. Caplan,
I
have been reading your recent posts on econlog about the lack of research on
debating tactics, and I wanted to propose a potential research avenue that you
or someone you know might be interested in.
I
moderate a forum on Reddit called “change my view” which is
essentially a debate forum, but specifically focused on people who have a view
but are open to changing it. As a part of the forum, we have a protocol
for people to note and reward changes in their views. Essentially, if
someone’s post changes your view, you include a greek delta symbol in the
comment, and a bot comes around and gives the person who changed your view some
internet points.
You
can access the site here: https://www.reddit.com/r/changemyview
The
upshot of all this is that we have a tabulation of thousands of debates, and
thousands of posts within those debates which were effective at changing
someone’s view. I don’t know exactly how to methodologically turn that
into a useful conclusion about what’s effective, but I figure you or one of
your students or colleagues might be interested in giving it a crack.
Good
luck in your future debating endeavors.
Best,
Peter
Hurley
Normal
0
false
false
false
EN-US
X-NONE
X-NONE
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:”Table Normal”;
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:””;
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin:0in;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:”Calibri”,sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;}
[Bryan again]
Sample bias is the obvious problem. People who frequent this reddit are almost certainly unusual human beings. But I still like the idea. If anyone makes a serious go of it, I will blog it.
READER COMMENTS
Peter H
Jul 14 2015 at 10:27am
I’m the one who emailed Bryan, on the question of the data quality, I’ll point out the following possible issues and advantages:
Issues:
CMV is heavily moderated. To keep it as a good forum, we need to enforce rules about civility and open-mindedness. As such many things get removed on a discretionary basis, though there are rules and guidelines about it.
DeltaBot isn’t perfect. It’s been broken for a few weeks now, though one of the other mods who is more adept than me has been keeping a crawler active to track deltas in the interim.
DeltaBot doesn’t allow all deltas, even when it’s working. The original poster of a submission can’t be awarded a delta by another user, and any delta must be accompanied by a certain number of characters (200 I think) to explain why the view was changed.
Advantages:
We actually get a lot of submissions from people who aren’t CMV regulars. So while the people who do the view changing may be regulars, the people whose views are changed are often not.
The data on removed submissions and disallowed deltas aren’t destroyed completely. A researcher who was given a level of privileged access could scrape them via a crawler or something.
Sieben
Jul 14 2015 at 10:40am
Another obvious problem is that people hate failure, so they are less likely to attempt to convince people of stickly propositions.
Radical libertarianism is definitely stickly.
Peter H
Jul 14 2015 at 10:47am
Sieben,
I’ve never seen the word “stickly” used before, so I can’t say exactly, though we do have plenty of posts and arguments about some more radically libertarian ideas (as well as some more radically statist ideas – and some just plain weird ones).
We do seem to get a hugely disproportionate number of posts about gender and transgenderism for whatever reason.
Dustin
Jul 14 2015 at 10:51am
3.5 million registered users, 164 million visitors (June 015), 731 million unique visitors (2013)
https://www.reddit.com/about/
http://thenextweb.com/insider/2013/12/31/reddits-2013-stats-56-billion-page-views-731-million-unique-visitors-40-million-posts/
‘Unusual’? This is the definition of ‘usual’ IMO.
It would be more accurate to say “People who frequent this EconLog site are almost certainly unusual human beings.”
Anon.
Jul 14 2015 at 11:22am
Well the first order of business is categorizing the arguments into different types. It’s boring, menial work… Sounds like something graduate students would do.
Kevin L
Jul 14 2015 at 12:09pm
Personally, I’ve heard arguments that changed my view on moderately important topics, only to have my views revert or change again. That is something that cannot be tracked in a debate forum where the delta responses are, I assume, posted shortly after first reading an argument.
Mark
Jul 14 2015 at 4:34pm
Perhaps it could serve as a useful test environment.
Grab a random group of test subjects, ask them their views on various topics, and how strongly they hold those views.
Then get them to post their views on change my view, and see what happens.
Some of the data that could be interesting:
– Which sorts of appeals do they respond to (length, emotional vs. data-based, writing style, etc)
– Which arguments do they ignore
– How many rounds of argument does it take for their view to be changed?
– Was their view changed?
– Did their view stay changed? (requires follow up later of course)
Of course, you could do all of this in a more controlled lab environment, but since arguments tend to be threaded, you’d need to have a lot of “pre-programmed” responses to cover all of the directions the subject might take it. In fact, you could use (or browse) CMV to collect some of those arguments to prep for a lab test.
Ricardo Cruz
Jul 14 2015 at 7:14pm
Cool stuff to apply some hard data mining techniques to. 🙂
Ben
Jul 14 2015 at 8:03pm
Of course, one might well end up with the depressing finding that demagoguery, ad hominem attacks, and outright lies are the best way to change minds. I would guess that politicians know what they’re doing; politics is a highly competitive environment in which those who are persuasive win and those who aren’t get weeded out mercilessly (ok, Donald Trump being an obvious exception, but his personal fortune makes him somewhat immune to “weeding” :->). So the state of politics today makes me think that the debate tactics that work the best are not likely to warm Bryan’s heart. But then, it seemed to me that Bryan’s post was not about what debate tactics are the most likely to change minds, but rather, what tactics are demanded by integrity and honor. Which is, of course, a big reason why I’ve always been a Bryan fan, and why he has changed my mind more than once. :->
db
Jul 15 2015 at 11:42am
A question for Peter H:
How do you verify that a given person actually holds the view they are purportedly willing to have changed by argument? That is, how would one control for disingenuous folks who hold already an opposing view and falsely claim to have been convinced by reason the error of their previous ways in an attempt to influence a debate in their favor? Said another way, how do you detect false flags?
Peter H
Jul 15 2015 at 6:45pm
db,
Human moderation. Such posts are prohibited under the rules, and have been removed in the past. Any user can report a post for violating one of our rules, and the moderators can act on that report, either unilaterally or after a discussion among ourselves.
Generally, false flags aren’t so common, though they do happen. It’s usually not too hard to tell, since people who aren’t false flagging won’t usually capitulate totally to the first replies they get.
Much more common are soapboxing posts where people post with zero intent or openness to changing their mind. We probably yank two or three of those on an average day.
Comments are closed.