Professor Jason Lodge from The University of Queensland explores why many AI-powered tutorbots fall short of supporting meaningful learning in his article What most AI tutorbots get wrong about learning, examining how educational AI can better develop students’ independence, critical thinking and ability to take ownership of their learning.
The AI tutoring hype
Why does almost every new LLM-based tutor arrive with an assurance that it is not about replacing teachers? A product that keeps telling you what it isn't for invites an obvious question. Whatever the intentions of the people building these tools, the pitch itself, a personal tutor for every student on the planet, casts the teacher as the expendable part of the system.
Even if we accept that the aim is not to replace teachers, AI tutorbots are seen by some as a solution to a very real shortage of educators. The UNESCO Global Report on Teachers estimates that the world will need 44 million additional teachers by 2030. You can see why AI is tempting. If you are a minister responsible for a system that cannot recruit or keep enough teachers, a technology that promises personalised instruction at scale, at a fraction of the cost of a human tutor, is an easy sell.
AI teachers are a simple answer to a complex problem. Education has seen plenty of technologies promised as the answer to systemic problems, and as Justin Reich has documented, the gains have tended to be more modest and more uneven than promised, and rarely achieved by the technology on its own. These findings, as my colleague Jared Cooney Horvath, PhD, MEd collates in his book, The Digital Delusion, drive the growing trend toward screen time limits and technology bans in schools.
How are AI tutorbots justified?
Two ideas do most of the work here:
The first is Bloom's two-sigma problem. In 1984, Benjamin Bloom reported that students who received one-to-one tutoring performed two standard deviations better than students taught in a conventional classroom, enough to move the average student to around the 98th percentile. Bloom framed this as a problem because one-to-one tutoring is far too expensive to provide to everyone. His challenge to the field was to find methods of group instruction that could come close. The AI tutor pitch reads it the other way: a tutor for everyone, made affordable by taking the human out. The two-sigma figure has become a headline number in that pitch, including in Sal Khan's widely viewed 2023 TED talk launching Khanmigo.
The second is a vaguer appeal to scaffolding, usually with a sprinkling of Vygotsky and the zone of proximal development. The idea is that the AI provides just enough support to keep the learner working at the edge of what they can do. Vygotsky didn't use the term, for what it's worth. It is usually credited to Wood, Bruner and Ross in 1976, and the link to the zone of proximal development was made later. As the concept has since been defined, scaffolding involves support that is contingent on what the learner needs at that moment, that fades over time, and that transfers responsibility to the learner. The handing over is the whole point of the metaphor. Very few tutorbots I have seen are designed to hand anything back.
Have they been working?
The evidence is mixed, and thinner than the marketing suggests. There are some promising trials, among them a Harvard physics study in which a carefully designed AI tutor outperformed an in-class active learning session, and a World Bank trial in Nigeria that produced meaningful gains over six weeks with teachers guiding the sessions. Both were short, carefully engineered and closely supported. Beyond trials like these, a recent Stanford review found the evidence base on AI in schools to be very limited.
The most candid assessment has come from Sal Khan himself. Three years after Khanmigo launched, he told Chalkbeat that for a lot of students 'it was a non-event'. Most of them simply didn't use it. Khan Academy has since rebuilt its product so that Khanmigo sits inside practice problems rather than waiting to be asked, because students weren't seeking out its help. Its chief learning officer, Kristen Eignor DiCerbo, put part of the problem down to students not being very good at asking questions. Khan now describes AI as part of the solution rather than the whole of it, and says the bigger lever is investing in the human systems around students (exactly what many of us have been saying all along). That second point about asking questions matters more than it might seem, and I'll come back to it.
None of this is helped by the sub-standard evaluation approaches being used. I've heard enough mangling of effect size estimates to last a lifetime. Effects on narrow, developer-built tests get compared with benchmarks drawn from broad standardised tests, even though effects measured on narrow tests tend to be around twice as large. Engagement metrics stand in for learning, and gains among the students who chose to use a tool are presented as though they tell us what it does for everyone. Many of the claims being made are not what I would have tolerated in a second-year stats class, let alone in major, internationally renowned, well-funded work.
The situation is compounded by the goal resting on a myth, or at least on a claim that has drifted a long way from its evidence. Bloom's experiments ran for three weeks, taught topics that were unfamiliar to the students, and measured outcomes on tests of exactly that material. The tutored students also got extra quizzes, corrective feedback and retesting, specially trained tutors and more instructional time. As Paul von Hippel has shown, the feedback and retesting alone account for roughly half the effect. The result has never been replicated. A 2020 meta-analysis of 96 randomised tutoring studies found an average effect of 0.37 standard deviations, and none produced two sigmas. That is a substantial effect for tutoring. It is also a long way from the sales pitch. How far findings from three weeks of intensively supported human tutoring carry over to a chatbot is another question entirely.
Where the approach goes wrong
The dominant design choice is to stop AI tutorbots from providing answers with some sort of system prompt. Instead, they respond with prompts, hints and questions. This usually goes by the name of a 'Socratic tutor'. The label borrows from the method Plato attributes to Socrates, who professed to know nothing and drew out, or dismantled, other people's views through questioning. In tutorbot form, a student asks how to solve an equation and gets back something like 'What do you think the first step might be?' The theory is that the student constructs the answer themselves and learns more as a result, and that the bot can't simply do the work for them.
There is something to this. Generating information tends to be remembered better than reading it. In a large field experiment in Turkish high schools, students given unrestricted access to GPT-4 during practice went on to perform worse on exams without it than students who never had access, and a version designed to give hints rather than answers largely removed that harm. But removing a harm is not the same as producing learning, and withholding answers is not always the right call. For novices, studying a worked solution is often more effective than struggling to produce one, and support that helps a novice can become redundant or even counterproductive as expertise grows. Even approaches built deliberately around struggle, such as productive failure, depend on learners getting the canonical explanation via explicit instruction afterwards.
Expertise doesn't develop in a straight line. Studies of how problem solving develops show learners using several strategies at once and shifting between them unevenly over time. Supporting the development of knowing, doing and being is nuanced, personalised and messy. There are times when a learner needs the kind of scaffolded support a Socratic tutor can provide. There are other times when the process is well in hand and what the learner needs is the answer to a very specific question. A geometry teacher who trialled Khanmigo with her class found her students frustrated by a bot that would sometimes get things wrong but still wouldn't tell them the answer. Being stonewalled by something that is itself unreliable is not most people's idea of productive struggle.
Other kinds of support matter too: help with setting goals, planning, monitoring progress and switching strategies, and with managing time. Every Socratic tutorbot I have seen does a poor job of responding to these shifting needs. Maybe there are more sophisticated tools out there; I just haven’t seen them.
The bigger problem is agency. Socratic tutorbots end up making many of the decisions learners need to make for themselves. The bot decides what the next question is, how big the next step should be and when the student has got it. These judgements and decisions are not overhead sitting on top of the learning. In many cases, they are the learning. Working out what you don't yet understand, deciding which strategy to try next and judging whether a piece of work is any good are what self-regulated learning consists of, and the capability to judge the quality of your own work and other people's is one of the main things education is meant to develop. Offloading cognitive work to AI isn't harmful in itself; what matters is what gets offloaded. Offloading these decisions is the costly kind. A tutorbot that takes them away is taking away some of the core elements of what it means to learn.
There is an obvious objection. Learners are often not very good at these judgements. Students routinely misjudge how well they understand something and make poor choices about how to study as a result. Simply handing learners control does little for achievement on its own. But these capabilities can be explicitly taught, and interventions that teach them improve achievement. The answer to poor judgement is to support learners in exercising it, not to exercise it on their behalf. DiCerbo's point about students struggling to ask good questions is the same problem from the other direction. Research on tutoring dialogue found that students do try to spot gaps in their knowledge and ask questions to fill them, but need explicit help to get good at it. A tutorbot that waits passively to be asked does nothing to build that capability. One that takes over the steering entirely is still doing the thinking for the student, just more politely.
Experts are not experts just because they know a lot of things. That is utterly critical, but it isn't the whole package. Experts have a nuanced relationship with knowledge and knowing. It allows them to integrate new knowledge, to know the limits of what they know and to adapt what they know to genuinely new problems. None of that is easily supported algorithmically, so the tendency is to brute force the knowledge acquisition piece. That, unfortunately, comes at the expense of the adaptive capabilities I've described here.
To be fair, a lot of careful work has gone into these tools, and I am likely overstating the simplicity of the approach. The better ones are considerably more sophisticated than 'never give the answer'. The point stands, though. Most don't align with the complex, uneven ways in which people move from being novices to being experts.
What should happen instead?
Stop aiming at the two-sigma problem. It isn't helping. Bloom's figure has anchored expectations for effect sizes that very few educational interventions come close to, and chasing it encourages inflated promises and inflated claims.
Stop treating Socratic tutoring as a blanket solution. The secret sauce is supporting metacognitive and self-regulated learning: helping learners figure out where they are, decide what to do next, and notice when a strategy isn't working. Good teachers can make these dynamic shifts in support, moving between giving an answer, asking a question, prompting a change of approach or leaving a student to work something out. It is hard to do well, and classroom observation suggests genuinely contingent support is less common than we might hope, which is all the more reason not to design it out.
The highest risk version of the path we are on is a two-tier system: privileged people get access to the best human tutors, inside or beyond their formal education, while everyone else gets AI tutors and teachers as a substitute. There are already warning signs. Learning technologies have often benefited affluent students most, and in the World Bank's Nigeria trial the largest gains went to students who started out ahead. We cannot allow that to happen.
Tutorbots should never be seen as a replacement for teachers. Learning is social. The quality of students' relationships with their teachers is consistently associated with their engagement and achievement, and much of what education does, socialising people into ways of knowing and being, happens in interaction with other people. We shouldn't trade that away, no matter how sophisticated AI gets.
That leaves two legitimate paths for tutorbots. The first is to drop any notion that they can do anything beyond knowledge acquisition. There's nothing inherently wrong with that. Knowledge acquisition is important and it matters. You can't think critically if you have nothing to think critically about. This way, the human, social and adaptive capabilities that sit around the acquisition piece stay where they are likely to have the most impact, in the hands of skilled educators.
The second path is harder. If tutorbots are going to go beyond knowledge acquisition, they need to support judgement and decision-making in a way that does not take agency or metacognitive demands away from learners. Those processes might not look central to the task at hand, but they are critical for the development of expertise, and often they are the point.
We have further unpacked the issues associated with AI companions/tutors for supporting actual learning in a recent paper led by Hassan Khosravi.
This article was written by and is the sole intellectual property of the author. Claude (Opus 5.5) was used to edit the article for clarity.
Professor Jason M. Lodge
Professor of Educational Psychology | Founding Co-Chair, Learning & Instruction Society of Australasia (LISA)