I, like many of you, have been following the inception and progress of AI, especially as it relates to education. Like many of you, I have been experimenting with it in my daily life and have been introduced to its benefits and pitfalls. I have also been reading widely about AI, especially ChatGPT, and the effect that it is having on student learning. Clearly, the field is evolving, and the final verdict on AI's efficacy is not in, but there are some troubling signs.
In a September 2024 article of
The Hechinger Report entitled
Kids Who Use ChatGPT as a Study Assistant Do Worse on Tests, author Jill Barshay shares the results of researchers who compared the math progress of almost 1,000 high school students.
Researchers at the University of Pennsylvania found that Turkish students who used ChatGPT to assist them as they did math problems did worse than students who did not use ChatGPT. Their findings:
"Those with ChatGPT solved 48 percent more of the practice problems correctly, but they ultimately scored 17 percent worse on a test of the topic that the students were learning."
The researchers attributed these findings to the theory that the first group of students were using ChatGPT as "a crutch" – searching for the answers without truly understanding the process or the work involved.
To further examine their hypothesis, the researchers examined a third group of students who used ChatGPT as a tutor. This was done with a revised version of ChatGPT designed for this very purpose, providing suggestions, but not revealing the answers. Their results:
"The students who used it did spectacularly better on the practice problems, solving 127 percent more of them correctly compared with students who did their practice work without any high-tech aids. But on a test afterwards, these AI-tutored students did no better. Students who just did their practice problems the old-fashioned way – on their own – matched their test scores.”
The study,
Generative AI Without Guardrails Can Harm Learning: Evidence From High School Mathematics digs more deeply into the pitfalls of student usage of generative AI.
The researchers concluded that parents and educators (and students) need to be wary of the currently free crop of AI Chatbots. They cited these reasons:
- Chatbots discourage students from building the skills that they need to solve problems themselves.
- Chatbots are not error free; they make errors in outlining the mathematical process and in generating final answers.
- Chatbots can generate overconfidence in students. Surprisingly, students did not think that ChatGPT had caused them to learn less. Even students with the AI tutor thought that they had done better on the exam than they had.
AI is becoming ubiquitous in education and in our daily lives. For this reason, I am still pondering this thought:
"A key question that remains is how generative AI affects how humans learn novel skills, both in educational settings and through the course of performing their jobs. This process of skill acquisition is critical for safeguarding long-term productivity. However, many generative AI deployments are designed to automate tasks without consideration for impact on learning."
So, what does this mean for learning? What does it mean for us as educators and parents? And what does it mean for our students? When is AI a tool? When is it an impediment to learning? How do we decide when and where to use it? How do we help our gifted students decide when and how to use it? If you're looking for answers, maybe we need to consult with ChatGPT (just kidding).
As always, I welcome your thoughts. Together we grow.
- Jackie Drummer, Past President and Current Advisor to the WATG Board
Thank you to Esther Vazquez of the Appleton Schools for the translation of this article for our Spanish-speaking educators and families.
