Evaluating the impact of AI assistants on students’ learning

HTI has partnered with UTS High Resolves to evaluate the impact of AI assistants on high school students’ learning motivation and engagement.

This project uses Bayesian Adaptive Trial (BAT) methodology to understand which features of AI assistants work best to improve students’ motivation, and whether these features are different for subgroups of the student population. The use of BATs in a social policy context is an Australian first and, to the best of our knowledge, a global first. 

The AI Assistants, developed by High Resolves in alignment with the Australian curriculum, are designed to support classroom tasks and discussions. Students from 16 schools across Australia have engaged with the AI Assistants in their classrooms as part of the ongoing trial.

AI Assistants offer transformative benefits in education by making personalised learning accessible to all students. They remove cost barriers to one-on-one tutoring, enable highly individualised curriculum pathways that account for diverse learning needs, and provide real-time, adaptive feedback that deepens engagement and understanding.

Bayesian Adaptive Trials

BATs are a rigorous and efficient framework for social policy impact evaluation, which uses AI algorithms to synthesise and learn from a range of different data sources, allowing researchers to gain insights in real time and adapt or change as needed. 

BATs are increasingly used in clinical research, particularly in oncology, where they support decision-making in high-cost, high-variability, time-sensitive, and ethically sensitive contexts – features common to many social policy environments.