Take This Ideological Turing Test

0 out of 5 for both. Wow, since I thought I aced it.

Zero out of 5.

I wouldn’t mind knowing what the self identified liberals are answering.

@Dangerman-

That’s the experience I had too. This is very interesting. I can’t wait to see the paper when it is finished with all the data compiled!

Has anyone got a result other than 0/5 ? I tried it as both a liberal and a conservative and got 0/5 both time despite giving what I thought were the most representative answers on both occasions.

Another zero out of five here. I’d really be curious to know how self described liberals actually answered these questions.

I’d like to join the 0 out of 5 club. I was sure I was being generous and smart about my answers to the liberal ones. Very interesting.

Add me to the list of double goose egg recipients. I hope we get to see a break down of the answers at some point.

Add another zero to the pile-on. I was going to make that my criticism of the test, but since everyone else has already noted it, I’ll add a different criticism:

The “free trade” or “laissez-faire” positions are all indicated as “conservative.” But aren’t these really the liberal positions on economics? Why does the quiz use opposite language to describe liberal trade policies?

Also 0 out of 5.

The methodology is flawed, IMHO — the survey never asks for our own views, but just what “liberals” or “conservatives” as a whole think. Even someone who identifies as liberal or conservative is being asked to judge based on their sense of the overall category rather than their personal views.

Also, it has the usual flaws for a multiple-choice survey — I would guess that most liberals, e.g., would have both fairness-related and outcome-related justifications for many of their positions, so picking a single one would be oversimplifying.

0 of 5, 1 of 5, identified as moderate. I tried to be generous to both sides as in Australia at least most politicians seem to hold their views sincerely, and beleive themselves on the side of right.

This test had multiple correct answers on both sides. For example, on the “conservative” side, I found myself agreeing with two or more answers on every question. On the “liberal” side, each question had multiple answers that describe views I have actually seen from self-identified “liberals.”

There were a couple of obvious strawman answers, but most simply reflect different views held to varying degrees by different people.

0/5 I did it as very conservative economically.

I tried to be really generous, but I thought there were several reasonable answers that liberals and conservatives could give.

I think the idea is a good one, but I think the implementation is likely off.

Another big fat 0 here also – I smell something going on here – lefties are gaming the system and not saying what they really think, hhmmm.

Concur with The_Original_CC and Dave: I picked the intelligent arguments for liberal policies, the ones that can be most strongly defended without appeals to emotion. If it turns out that the typical liberal is holding these beliefs based on the fluffy reasons, that lowers my estimation of the reasonable opposition.

I thought Ideological Turing Tests were supposed to use the most intelligent, well-informed advocates to compare with their caricatures?

(I’m guessing there’s a bug in the code.)

0/5 here too…are we all really this bad at understanding our opponents or is there perhaps some software error?

I think something is bugged. Either way, here are my answers.

Conservatives oppose raising the minimum wage because they believe it would:

force employers to pay employees more than their contribution to the firm or not hire them at all.

Liberals favor raising the minimum wage because they believe it would

increase the income of low-wage workers with a minimal loss of employment

Conservatives oppose high taxation of incomes because they believe such taxes would :

reduce economic growth.

Liberals favor high taxation of incomes because they believe that such taxes would:

permit socially desirable expenditures, paid for by those who disproportionately gain from the system.

Conservatives oppose the use of economic stimulus programs because they believe:

stimulus programs are ineffective at improving output and employment.

Liberals favor the use of economic stimulus programs because they believe:

stimulus programs are effective at improving output and employment.

Conservatives favor generally freer trade because they believe:

freer trade has benefits greater than costs for society.

Liberals oppose generally freer trade because they believe:

freer trade hurts workers while helping corporations.

Conservatives oppose accumulating a large national debt because:

they worry about the debt ultimately causing a financial collapse.

Liberals do not oppose accumulating a large national debt because:

they believe fears of financial collapse are overblown.

Picking what I thought was the most sophisticated argument for each one:

You identified yourself as very conservative. Your score in correctly choosing liberals’ reasoning (based on answers given by self-identified liberals to date) was 3 out of 5 questions.

Your score in identifying liberals’ reasoning is as good as or better than 48 percent of others who are very conservative.

0/5 here, also trying to give the most reasonable answers (i.e. answers that I could best justify using my knowledge of economics.)

I’m wondering if either there aren’t enough people answering yet for it to decide what the “liberal” or “conservative” dominant answers are, or if someone(s) are intentionally giving weird answers to screw up its calibration. Or, it could be a sociological experiment to see how people react to finding out they’re less capable of passing an ideological Turing test than they think they are. >.>

In the first two cases, redoing to test with different answers just to see if you can improve your match will potentially make it worse. (If, for example, they’re not filtering responses by IP address and ignoring those addresses when people retake the test.)

I think I’m going to post this on my facebook feed, and then post my own answers in the comment section to manually compare instead of just relying on their posted results and comparing from there.

FYI: 0 out of 5 is not the only result.

“You identified yourself as slightly conservative. Your score in correctly choosing liberals’ reasoning (based on answers given by self-identified liberals to date) was 4 out of 5 questions.

Your score in identifying liberals’ reasoning is as good as or better than 89 percent of others who are slightly conservative.”

Slightly conservative, 4/5. I hang out with socialists.

Seems like it’s fixed. I retook it with all my original answers (as written above) and got 4/5 (L), 3/5 (C). Self identified as a “Moderate”

“You identified yourself as very conservative.
2 out of 5 questions.
Your score in identifying liberals’ reasoning is as good as or better than 20 percent of others who are very conservative.”

Not an American though 🙂

Thank you very much for completing the survey!

You identified yourself as very conservative. Your score in correctly choosing liberals’ reasoning (based on answers given by self-identified liberals to date) was 3 out of 5 questions.
Your score in identifying liberals’ reasoning is as good as or better than 50 percent of others who are very conservative.

I only got 3 out of 5 not so good. I guess that I need to read more left blogs.

FWIW here is text that I got:

Thank you very much for completing the survey!

You identified yourself as very conservative. Your score in correctly choosing liberals’ reasoning (based on answers given by self-identified liberals to date) was 3 out of 5 questions.
Your score in identifying liberals’ reasoning is as good as or better than 49 percent of others who are very conservative.

4/5 here as very conservative economically.

So I clicked the button identifying myself as economically conservative and then the final page came back with,

“You identified yourself as very liberal.”

Something’s not right.

FWIW I matched 4/5 conservative arguments, which they said was better than 96% of other very liberal people. So that’s what’s happening on the other side.

“You identified yourself as very conservative. Your score in correctly choosing liberals’ reasoning (based on answers given by self-identified liberals to date) was 5 out of 5 questions.”

I think that’s luck to some extent (some choices were difficult), but I chose answers based on my assessment of what people actually tend to believe, not what was the best argument for the position.

Unfortunately, I did the same for the conservative answers.

I guess you could call this test an ideological Turing-Keynesian beauty contest?

I got a 3/5 for liberals because I was too generous to their position when my gut was probably more accurate. I meta-Turing-ed myself…

Pretty sure I was 5/5 for conservatives, which I should be since I hang out with them online much more.

I put “very conservative,” but that feels strange as an anarcho-capitalist…

I’m surprised at all the 0/5 scores reported in the comments. I got 5/5, self-identified as slightly conservative (I hate that way of thinking; I lean toward classical liberalism), and I just tried to think what the typical NY times reader would think. It seemed pretty straightforward to me.

Per Daniel Fountain:

For anyone wondering the correct answers (5/5 , 5/5) are:

D, A, C, C, A, A, A, A, A, B

If one answers as a “moderate,” then the survey scores both liberal and conservative responses.

My first attempt scored 4/5 on liberal and 5/5 on conservative. My “mistake” was assuming that liberals supported economic stimulus programs because they “prevent the most vulnerable from being hurt in a recession,” when they apparently believe that these programs “are effective at improving output and employment.”

It helps to remember that this is an economic turing test, not what you would get asking random students at college.

Moderate with conservative 3/5 and liberal 2/5. All of these views are represented so there should probably be a weight assigned (and you get more points if you choose something a higher share answered). I hope they make this data available.

It is very much a Keynesian beauty contest spin on it since its trained by liberals’ views of average liberal views.

This is a poorly conceived and executed survey because:

• It doesn’t clearly distinguish whether it is asking what people believe, or asking what they say. [A true Turing test would make clear it is asking about what people say]. What people will say when asked to justify their beliefs will often differ from their most important reasons for holding those beliefs, for instance if they think that exaggerating their beliefs will be more effective at persuading others or signalling group membership, or if they assume that a certain style of answer is conventionally expected.
• It doesn’t allow for the possibility of explanations being offered at different levels of specificity, or provide any criteria for choosing among levels. For example, people may believe that Policy X is bad for the economy, and may also believe that there is a particular mechanism through which this happens. Which is their “real” or “most important” reason? The question either does not make sense, or is unclear. [Again, instructions that are clearly based on what people say would provide such criteria, because presumably the frequency with which people offer general over specific explanations can be empirically measured. Whether general or specific explanations are more real or more important cannot.]
• It doesn’t allow for the fact that people can be “liberal” or “conservative” for different ethical reasons, or that there are different dimensions of liberal/conservative even when we stick to economic policy questions. For example, one could oppose Policy X either on grounds of utility or desert — cutting taxes will lead to economic growth, or it allows people to keep more of what they earn. Both are intellectually valid conservative reasons; conservatives may differ in which they think [sorry, which they say] is most important. Which answer is more frequent in the general population? The survey designers make no attempt to find out and so are on shaky ground telling people that their guesses are wrong.

I don’t know whether the authors of this survey hope to get a publication out of it, but if I were the referee, I would advise the editor to reject any article based on the current survey, and for the authors to think hard about observability issues — i.e., whether beliefs can in principle be observed using this methodology.

For what it is worth, I am a market-oriented economist who identifies as politically liberal and I got 1/5 on the test. [The test didn’t tell me which one.] You could say my views simply reflect wounded pride, but my guess [apparently validated by the reports of previous commentators] is that I would do just as poorly at identifying the “beliefs” of self-identified liberals.

“You identified yourself as very liberal. Your score in correctly choosing conservatives’ reasoning (based on answers given by self-identified conservatives to date) was 5 out of 5 questions.

Your score in identifying conservatives’ reasoning is as good as or better than 100 percent of others who are very liberal.”

Ignoring the bogus USA conservative/liberal economic axis (on which I identified as ‘very conservative’, meaning what ‘very liberal’ would mean elsewhere in the English-speaking world) I got 4/5 for the ‘liberal’ questions. Rather than try to game the test I tried to imagine what most educated ‘liberals’ (i.e. lefties) actually say when attempting to provide reasoning for their views. In fact I considered comments that I normally would come across on this blog. I struggled more with the ‘conservative’ questions, mainly because on a couple of them I couldn’t find the most likely answer. Which indicates to me that no ‘conservatives’ were involved in writing these questions.

The way I got 4/5 on liberal was by following WWPKA.

Apparently he would answer that high taxes “permit socially desirable expenditures, paid for by those who disproportionately gain from the system” rather than “raise large amounts of revenue without significantly affecting overall output”.

On further reflection, I might just agree that he still has enough economist in him to avoid the risk of deadweight loss in his answer.

So this is interesting: I tried getting a 4/5 for each side, once identifying as strongly liberal, once as strongly conservative.

A 4/5 on liberal answers gets you better than 81% of strongly conservative test takers. A 4/5 on conservative answers gets you better than 96% of strongly liberal test takers.

For slightly liberal and slightly conservative test takers – 4/5 as a slight liberal gets you better than, again, 96% of test takers. 4/5 as a slight conservative gets your better than 84% of slight conservatives.

Obviously this might be confounded by lots of things. Maybe conservatives are more inclined to retake the test to improve their score. Also these numbers will obviously change as more people take the test. Perhaps this trend will hold or perhaps not. It’ll be interesting to see.

5/5. Self described very conservative, picked 5/5 liberal positions. The key is to remember that the other side does not view e themselves as evil, they will choose a response that aligns to a self perception of being “good”. The answers are actually pretty obvious on both sides when applying that filter.

I always try and remember that – my ideological opponent does not see themselves as evil, though they do tend to see my position as evil. I tend to view their position as misguided with good intentions, rather than taking the evil approach

Ignoto: “Both are intellectually valid conservative reasons; conservatives may differ in which they think [sorry, which they say] is most important. Which answer is more frequent in the general population? The survey designers make no attempt to find out”

I think the test itself is the attempt, no?

Oh, wow. The anonymous from above here again – I tried the test again, this time getting 3/5 as each of a strong conservative and strong liberal, and found that 3/5 gets you better than 48% of strong conservatives, and 84% of strong liberals.

To be fair, I don’t know where this test has been shared. If it’s gotten a large proportion of its attempts from EconLog readers that might explain this skew.

I’m going to stop screwing up their results now. Hopefully they are logging IPs along with attempts and can remove all but the first attempt from each IP, or something similar. But… it is interesting that the results I can glean so far seem to support the conjecture that, if I recall, led Bryan to invent the concept of the Ideological Turing Test in the first place.

I took it yesterday and got 0/5. Took it today and got 4/5 in understanding liberals. I think I changed one answer from yesterday. ID as very conservative. Something odd is happening with this test. BTW, is there really only one answer that a liberal or conservative must give? I think that the answers would be more like Family Feud distributions.

So I said ‘Very Conservative’ and I got 3 out 5 which it told me was in the 48th percentile. Clearly we here at Econlog are NOT representative.

I rated myself as very liberal. I answered the questions from a classical conservative point of view and got 2 of 5 correct. Then I took it a second time from the perspective of my conservative friends and neighbors, and got 3 of 5 correct.

I think the fundamental problem with the test is that there are multiple types of conservatives that could be your frame of reference, and the more you about their opinions the harder it is to choose an answer.

For example, a conservative economist is likely to give a different answer from what you’d get from a conservative layman. A tea party conservative is likely to give a different answer from that of a mainstream conservative. And those of us who know GOP political strategies might choose yet another answer that reflects the goals of conservative political elites.

Take the deficit question. Tea partiers and gold bugs fret about deficit spending because they think it’ll lead to hyperinflation and an economic collapse, a la Germany between the wars.

Conservative economists are more likely to be concerned that deficit spending will crowd out private sector borrowing and undermine economic growth.

GOP political elites oppose deficit spending because it finances entitlement and other domestic spending programs, and they want to “starve the beast” (except, of course, when they’re in power, and then the deficit doesn’t matter, as Dick Cheney said, because the spending goes to the military and homeland security industrial complex–or to cement support among seniors, a la Medicare Part D.)

And if you’re a liberal answering the questions from the perspective of an “informed” conservative layman, there’s another problem: the conservative message machine. The ubiquitous dissemination of conservative talking points through their media channels has armed laymen with arguments that are prepackaged for use in political discussions.

I see this with my conservative friends. Sometimes it’s hard to tell what their opinion is and what they think is a good argument. And the fact that their arguments are sometimes inconsistent with each other further complicates determining what they really think.

A more interesting test would be one in which conservatives and liberals are asked to determine the truth or falsity of “facts” and statements that are commonly cited by each side.

I took the test as very conservative, got 3 out of 5 liberal questions correct.

I selected the reasons I felt were the most persuasive to me, so that’s probably why I missed a few. I deliberately avoided answers that sounded like they were from the HuffPost comments.

For most of the questions, there were multiple answers that mapped onto reasons I have seen liberals give for believing what they do, which made it difficult to select one.

4/5 as a very conservative.

You identified yourself as very conservative. Your score in correctly choosing liberals’ reasoning (based on answers given by self-identified liberals to date) was 4 out of 5 questions.

Your score in identifying liberals’ reasoning is as good as or better than 81 percent of others who are very conservative.

I identified as slightly conservative.

Got 3 out of 5.

“As good as or better than 54 percent of people who are slightly conservative.”

It’s disorienting when it asks me whether I’m “liberal” on economic issues. And my knee-jerk response was to click “very liberal” because I believe in liberating the economy! But I think in it’s terms I’m “Very conservative”

Anyway, I only got 2 out of 5 for my Turing test on liberals! Ouch!