ChatGPT has made some poor attempts to be a journalist and a therapist, but could it be a good student? Professor Christian Terwiesch from the Wharton School of the University of Pennsylvania thinks so. After Terwiesch proctored a final exam from a Master of Business Administration course for the chatbot, he found that ChatGPT passed.
Terswiesch published his findings in a report last week and revealed that not only did ChatGPT pass, it scored a B- to a B. Terswiesch did describe three main trends in ChatGPT’s performance on the exam. First, he claims that ChatGPT performed well on open-ended questions based on case studies. Second, the AI apparently struggled with some 6th grade arithmetic, with answers incorrect by magnitudes of ten, for example.
Finally, the chatbot couldn’t handle “process analysis” questions which included “process flows with multiple products and problems with stochastic effects such as demand variability.” ChatGPT was able to improve its results when provided with hints from humans, much the way a student might receive hints from their professor when asking a question during an exam.
“This has important implications for business school education, including the need for exam policies, curriculum design focusing on collaboration between human and AI, opportunities to simulate real world decision making processes, the need to teach creative problem solving, improved teaching productivity, and more,” Terwiesch wrote. Terwiesch is a Wharton professor and researcher with the school’s Mack Institute for Innovation Management.
Based on his work, Terwiesch argues that ChatGPT could be an invaluable tool for education, but professors and administrators need to be mindful of what the AI can and cannot do before baking it into a curriculum. Likewise, Terwiesch believes that teaching the foundations of various subjects is crucial to student success, even if ChatGPT can master those foundations with ease.
“In my view of education, an elementary school student still needs to learn that 7 x 7=49 and that the capital of Pennsylvania is Harrisburg, even though calculators have been widely used for over 50 years and students can use Google or Wikipedia to find answers for most factual questions,” Terwiesch wrote.
ChatGPT has proven itself as an incredibly versatile tool, which has fooled scientific reviewers, planned a holiday party, and even written a Gizmodo article. With that said, its role in higher education is questionable. While OpenAI believes schools need to get with the times, some school systems, like the New York City Department of Education, have banned the AI over fears of cheating.