Like many other people, I have enjoyed giving prompts to ChatGPT, an AI chatbot created by OpenAI. ChatGPT will respond to a wide range of prompts, often in amusing ways, yet often in ways that are surprisingly impressive.
I have seen many of my fellow academics express concern that students will use this type of AI to cheat. I can understand their concern. The AI wrote some material that I could imagine grading favorably. For example, at one point I asked the AI to write a Christopher J. Coyne article. It did not write an article Chris would write. However, it did write a fairly good summary of his excellent book After War: The Political Economy of Exporting Democracy.
However, as OpenAI directly warns users, ChatGPT “may occasionally generate incorrect information.” This is certainly true on economic topics. Every time I have asked ChatGPT about the Alchian-Allen effect, the AI has talked about different economic phenomena, never about the Alchian-Allen effect itself. I imagine the same is true for at least some other economic concepts.
Students should be careful about using ChatGPT as a substitute for real research and studying. They certainly should not treat it as though it’s “better than Google” for finding factual information. Instructors concerned about students using ChatGPT to cheat should try directly asking ChatGPT to answer the same questions they ask students. The instructors might be pleasantly surprised by how ineffective cheating using ChatGPT would be. Or they may choose to drop questions that the AI answers too reliably.
Does ChatGPT have implications for economic education beyond the academic integrity concerns it raises? Perhaps! I have been having fun asking ChatGPT to write song lyrics. So far I’ve had it write songs about a variety of economic topics, including microeconomics, public choice, and Elinor Ostrom. These types of whimsical prompts could offer educators new and memorable ways to present material, though students might find the results more cringeworthy than appealing.
That said, faculty should also be careful to fact check any educational materials they try to create using ChatGPT. Just as the AI’s errors could cause trouble for unscrupulous students, they might embarrass us too if we’re not careful!
Nathan P. Goodman is a Postdoctoral Fellow in the Department of Economics at New York University. His research interests include defense and peace economics, self-governance, public choice, institutional analysis, and Austrian economics.
READER COMMENTS
Thomas Lee Hutcheson
Dec 11 2022 at 12:02pm
Curious that Ai can’t do what I just did and lookup “Alchian-Allen effect” in Wikipedia.
Did you try asking the question with the phrase in quotes?
Nathan Goodman
Dec 13 2022 at 6:06pm
I have since tried this with “Alchian-Allen effect” in quotes and it still gave an incorrect answer.
My understanding is that ChatGPT is not enabled to search the internet. Instead, it was trained on a bunch of text data from up through 2021.
I think part of the issue is that ChatGPT can easily infer that Alchian and Allen were economists and can tell me that they coauthored Exchange and Production, but relatively few discussions of the principle called the “Alchian-Allen effect” use that phrase. Some call it “the third law of demand” and based on David Henderson’s blog posts I have learned that those trained by Alchian and Allen at UCLA call it the “oranges principle.”
Some good David Henderson blog posts on the topic:
https://www.econlib.org/archives/2014/04/freakonomics_di.html
https://www.econlib.org/high-quality-oranges-and-high-quality-hotels/
Garrett
Dec 11 2022 at 11:26pm
Post the public choice song lyrics!
Nathan Goodman
Dec 16 2022 at 11:33am
Here you go!
https://www.econlib.org/i-asked-chatgpt-to-write-songs-about-public-choice/
Joseph Hertzlinger
Dec 12 2022 at 3:12am
From a ChatGPT session:
Q: If landlords want to build more and landlords want to raise rents, what will happen if landlords build more?
A: If landlords build more, it is likely that rents will increase due to increased demand for housing.
ChatGPT is a NIMBY.
Don Boudreaux
Dec 12 2022 at 6:12am
My (very computer-literate) son is quite impressed with ChatGPT, and I trust his judgment. But I’ve so far asked it only one question, which it got wrong. The question was: “If the price of a jar of pickles rises from $4.99 to $5.99 and as a result the revenue earned from sales of jars of pickles falls, what can we conclude about the elasticity of demand for jars of pickles over the price-change range $4.99 to $5.99?”
ChatGPT answered that demand over this price-change range is inelastic.
Mark Barbieri
Dec 12 2022 at 9:14am
If it generates lyrics to a song, who owns the copyright on those lyrics? I find it curious in the art world that artists are complaining that their pictures were used to train AI image generators and that they should be compensated. But it is perfectly legal for me to study their art and then mimic their style, so why can’t a computer?
Also, I think we can assume that it will get better and more accurate over time, so even if it is unsuitable for cheating now, eventually that will change.
Comments are closed.