Nigeria and Harvard tested AI tutoring—and the lesson is in the experiment

A Nigerian after-school programme and a Harvard physics experiment show why tutoring results belong to a defined teaching design, population and comparison.

An AI tutor can look convincing in a demonstration because the demonstration ends when it gives a useful answer. A learning experiment has to continue: what did the student understand, compared with what, and for how long? Recent work in Nigerian secondary schools and a Harvard physics course makes that distinction unusually concrete.

The studies concern different students, subjects and teaching arrangements. Neither establishes that a chatbot can replace a school. Their value is showing how an educational intervention can be designed and tested as a whole: the learning material, interaction, teacher support and assessment all influence what a result means.

Six weeks in Edo was a supported programme

The [World Bank's May 2025 working paper](https://documents1.worldbank.org/curated/en/099548105192529324/pdf/IDU-c09f40d8-9ff8-42dc-b315-591157499be7.pdf) evaluates an after-school programme involving first-year senior-secondary students from nine public schools in Benin City, Edo State, Nigeria. It combined generative-AI tutoring with teacher guidance. English was the main focus, alongside AI knowledge and digital skills. The intervention took place in June and July 2024.

The reported overall assessment effect was 0.31 standard deviations. That is a comparison on measured outcomes in the evaluation. It is not a statement that every participant advanced through two school years. Converting a test effect into an equivalent duration of typical schooling adds a separate assumption that readers should keep visible.

The implementers describe the delivery work

The [January 2025 World Bank account](https://blogs.worldbank.org/en/education/From-chalkboards-to-chatbots-Transforming-learning-in-Nigeria) describes teacher support, pen-and-paper testing and practical attendance challenges, including flooding and competing commitments. This is the implementers' account of the same programme, not an independent replication.

Those details matter to interpretation. Providing structured additional learning time is different from opening a general chatbot account. The result belongs to the supported package that was evaluated. It does not isolate the effect of the model from every other component, nor demonstrate that a longer programme would preserve the same gains.

Harvard tested a carefully designed physics tutor

A separate [Harvard physics study in Scientific Reports](https://www.nature.com/articles/s41598-025-97652-6), published on 3 June 2025, reports a 194-student experiment involving two lessons in consecutive weeks, with conditions reversed between groups. It compared a pedagogically designed AI tutor with an active-learning classroom arrangement. This was a comparison on defined topics, not between a public chatbot and all human teaching.

The researchers reported stronger learning outcomes for the AI-supported condition within their experiment. That is meaningful evidence about a particular instructional design and comparison. It does not establish superiority across all subjects, ages or universities, or answer whether the advantage would persist across a full academic year.

Put unlike results beside one another carefully

| Programme | What was tested | What cannot be inferred | | --- | --- | --- | | Edo, Nigeria | Supported after-school learning over six weeks | A model acting alone or guaranteed years of progress | | Harvard physics | A designed tutor against a specified classroom comparison | Universal replacement of teachers | | A new school pilot | Its own content, support and student population | Automatic inheritance of either study's result |

The studies can inform better questions without being pooled into a league table. An effect measured on one test cannot be ranked casually against an effect on another. The comparison condition also matters: additional supported practice, a familiar lesson and a well-designed active-learning class are different baselines.

A fictional pilot starts with one learning objective

Consider a fictional teacher-training college in Accra planning a screen-only demonstration for adult trainees. This is an illustrative exercise, not a Ghanaian deployment of either research system. The trainees choose a short passage and a single objective: distinguish an author's claim from the evidence offered for it.

They design a tutor that asks the learner to identify the relevant sentence before offering a hint. A teacher-reviewed answer guide explains several plausible mistakes. The demonstration uses invented learner responses and licensed material. It collects no schoolchild data and awards no real course grade.

Test the explanation, not just the final answer

The trainees write three possible interactions. In one, the learner selects the correct claim for the wrong reason. In another, the learner notices an ambiguity the answer guide missed. In the third, the system cannot interpret the response. These cases expose different educational needs.

A useful tutor should not treat the correct selection as proof of understanding or punish a justified alternative interpretation. If it cannot continue, it should preserve the learner's work and explain that the exercise needs review. That is a proposed design requirement, not an effect measured in the Nigerian or Harvard studies.

The teacher then examines whether the hint encourages a meaningful next step. A hint that merely disguises the answer may make the interaction feel successful while removing the reasoning the lesson was intended to develop. The instructional objective gives the team a concrete way to judge that trade-off.

A fair comparison includes the ordinary alternative

In a future authorised evaluation, the team would need to specify what learners otherwise receive. Comparing an elaborate new tutor with no additional practice answers a different question from comparing it with the same practice supported by a teacher or a printed worked example. Time, content and available support should be described.

Measure more than completion. A short explanation in the learner's own words can reveal a misconception concealed by a multiple-choice answer. Later assessment can investigate durability, but its timing should be reported rather than described vaguely as long-term learning. Enjoyment and confidence are useful outcomes that should remain distinct from understanding.

The fictional exercise does not estimate an effect size. A small convenience demonstration cannot support a claim about national attainment. If a real study is contemplated, qualified educators and researchers should choose an appropriate design and determine the necessary permissions before recruiting participants.

Access changes the intervention itself

The Nigerian account's attendance difficulties are a reminder that access involves more than possessing a device. Travel, power, connectivity, available time and language can change who participates. Students absent from a programme may also be the students a school most hopes to support.

For learners under 18, any real implementation needs the institution's approved service, appropriate safeguarding and privacy arrangements, and a usable non-AI route. This article does not assert a universal age rule or legal-compliance checklist. The adult training exercise can explore interface choices without opening accounts for children.

An unavailable connection should not erase a response or tell the teacher that a learner failed the concept. Distinguishing technical interruption from a learning difficulty is part of preserving an honest record of the lesson.

What the evidence changes for teachers and students

Teachers contribute the sequence of ideas, likely misconceptions and interpretation of ambiguous work. Developers make those decisions survive the interaction. Students can learn to ask whether an assistant is helping them explain a concept or only finish a screen. These are complementary tasks, not evidence that teaching becomes unnecessary.

The next valuable studies will examine sustained implementation, varied populations and outcomes beyond immediate topic tests. No staffing reduction, guaranteed grade improvement or universal learning effect follows from the records discussed here. This is educational, non-promotional reporting checked on 15 October 2026. The advance is a more testable approach to tutoring, with its educational design treated as seriously as its model.

Evidence diagram and readable guide


Separate study lanes show why supported tutoring results belong to a defined teaching arrangement, followed by questions for a new evaluation.

[Open the complete evidence diagram](/assets/education-workflow-tutoring-experiments-20261015.svg)

Read the diagram

**What was tested in AI tutoring?**

Two studies, two teaching arrangements. Keep each result attached to its setting and comparison.

- **Edo, Nigeria.** Six weeks in nine public schools. First-year senior secondary students.

- **Harvard physics.** 194 students. Two lessons in consecutive weeks; conditions reversed.

- **Supported package.** AI tutoring, teacher guidance and structured after-school learning.

- **Designed tutor.** Pedagogical AI tutor compared with a specified active-learning class.

- **Defined assessment.** Reported overall effect: 0.31 standard deviations in this evaluation.

- **Topic learning.** Stronger outcomes in the AI-supported condition within this comparison.

Questions for a new evaluation

- **Define and review.** Choose a learning objective and review the instructional material.

- **Specify the interaction.** Describe tutor support, learning time and the ordinary comparison.

- **Check understanding.** Use an independent learning check rather than completion alone.

- **Keep open questions visible.** What lasts later, and who can participate? A new pilot does not inherit either result.

**Reading the evidence.** Read each study on its own terms. Neither establishes a general model-only effect or replacement of teachers.

*Sources: World Bank, May 2025; Scientific Reports, June 2025.*

**Keep the comparisons separate.** Different studies and populations. Different instructional arrangements. Outcomes are not pooled or ranked.