Unlocking the power of AI to deliver next generation learning

In recent years, we have witnessed a rush toward studying and developing e-learning technological tools that enhance teaching and learning. The scientific literature clearly shows that these tools can play an important role in various processes, such as creating content that better engages students, enabling collaboration and interactivity even in large classrooms, and providing tutoring tools that support learning.

In the complex process we undergo from knowing nothing about a particular topic to acquiring reasonable awareness of it, there is a particularly valuable moment where we can achieve the maximum transformative effect.

It’s the moment of the feedback given by an expert, which addresses the correctness and completeness of the knowledge and skills demonstrated by the learner during an assessment, exercise, or even an oral discussion. In corporate contexts, for instance, this occurs when a new hire is paired with a senior colleague to expedite their apprenticeship.

While highly effective, the one-on-one approach is not applicable in educational settings, where each teacher faces many students in the classroom, typically 20 to 25 in primary and secondary schools and 50 to 300 in universities. In all of these cases, a significant portion of the feedback must be self-generated by the student through tests and exercises, while the main feedback from the teacher often comes only at the final exam.

Furthermore, a very imbalanced ratio between the number of students and the teacher makes it more difficult to explain course materials effectively, especially if we consider that students have radically different abilities and needs. It would be important to differentiate explanations depending on the difficulties of each and every learner, but this proves to be an impossible task.

MyLearningTalk: New AI-based tool empowering students and teachers

To tackle this challenge, Politecnico di Milano in Milan (Polytechnic University of Milan), Italy, is designing and developing MyLearningTalk (MLT), an AI-based virtual assistant that facilitates learning by providing personalised student support.

MLT does not replace teachers. Rather, it empowers students to understand and process content in a personalised way on the basis of their abilities, allowing them to explore content dialogically, receive appropriate examples, and obtain tailored feedback on the tests taken.

More specifically, in the methodology we are refining, the teacher personalises the course content for MLT use. This means the teacher is still the central figure in the classroom, giving lectures and helping learners carry out didactic activities.

Students leverage MLT to complement the lessons, clarify their doubts, and deepen topics of interest. Exams remain standard and do not allow the use of the tool. MLT thus becomes a way to augment classroom content and enhance learning without, however, resulting in flawed exams.

To develop a truly innovative tool, we gave great importance to three key elements.

The first concerns the interactive paradigm for content access and fruition. From this perspective, large language models (LLMs) represent the most appropriate technology, enabling natural language interaction between users and the tool.

Furthermore, the use of generative artificial intelligence allows for extreme flexibility in response, automatically controlling the language (an essential feature to support university internationalisation in full), the depth of answers, and the constant generation of new questions and examples.

It is observed, however, that the freedom offered by LLMs may be disorienting for many students, who may need or prefer guided interaction. For this reason, we’ve decided to equip MLT with interface elements that facilitate access to knowledge by suggesting additional interactions complementing the formulation of natural-language requests. Examples include buttons for automatic generation of quizzes, insights and concept maps.

The second element pertains to the completeness and correctness of content. Each university course is based on content customised by individual teachers, who usually draw on material from various sources, from textbooks, notes and slides to quizzes and videos. In many cases, the content is private, and it is not desirable to make it publicly available.

MLT benefits from the retrieval augmented generation (RAG) approach, through which information retrieval is implemented on the individual teacher’s content to build a prompt for the LLM. It should be noted that RAG enables greater control over the answers produced by the LLM and reduces the likelihood of hallucinations, which, in an educational context, would hinder the tool reliability and, in turn, students’ trust.

One of the technical problems to solve within the RAG approach concerns the use of mathematics. Many scientific and engineering courses rely on mathematics for definitions, theorems and algorithms. Currently, MLT uses documents written in LaTex, the language commonly employed for scientific articles, to build prompts for LLMs.

The results of LLMs containing symbols or mathematical elements are, in turn, produced in LaTex, and symbols can be displayed on screen through appropriate libraries. This allows users to formulate queries related to mathematical content and the tool to render the answers to their questions.

The problem of generating automatically numerical exercises and the solutions to them, however, remains open. This task is possible for basic maths problems, but not for more advanced exercises, which need to be managed by using external code scripts connected to the LLM.

The third key aspect regards the personalisation of the tool based on the individual student.

This requires tracking user behaviour in terms of both the content that has been explored (for example, which parts of the course the student focused on and how much time they dedicated to them) and how content has been explored (for instance, whether the student is used to starting from examples and exercises or from theory).

An analysis of user behaviour can be leveraged to build a recommendation system that suggests which content the student should pay more attention to and how to best approach it, that is, whether theory or practice should be the starting point. Tracking user behaviour also enables teachers to assess the student’s skills and facilitates their self-assessment in preparation for exams.

Preliminary results and next steps

A preliminary version of MLT has been experimented with small groups of students from the Smart Learning Design doctoral course – a humanities course – and the Algorithmic Game Theory master’s degree, a course in mathematics and computer science.

We conducted two main tests. The first focused on the correctness of the answers produced by MLT. Teachers evaluated the answers to many questions, and the results were found to be very relevant and far more accurate and complete than those provided by a LLM without RAG. This preliminarily demonstrates that the RAG approach is valid for achieving reliable outcomes.

The second test aimed to understand how students approach MLT and whether it plays an effective role in cognitive processes. It was noticed that students use the tool to search for information, aggregate the results, and learn faster than with traditional methods. The observed behaviour suggests that MLT strengthens learning effectiveness. More extensive studies will help us further investigate its impact.

Besides conducting continuous evaluation, the next steps to make MLT a successful learning companion include extending the user interface with other interactive mechanisms for knowledge work, consolidating the architecture scalability to an ever-increasing number of university courses, and defining a methodological framework to support teachers in monitoring students' progress and better calibrate their lessons.

It is no exaggeration to say that MLT is set to unlock a whole new world of possibilities within the education sector.

Luca Alessandrelli is an AI engineer at Polimi Artificial Intelligence Research and Innovation Center, Politecnico di Milano, Italy. Tommaso Bianchi is a project manager and senior AI research engineer at Polimi Artificial Intelligence Research and Innovation Center, Politecnico di Milano, Italy. Daniela Casiraghi is a project manager at METID (Methods for Innovative Technologies for Learning), Politecnico di Milano, Italy. Ludovica Piro is a PhD student in the department of computer engineering at Politecnico di Milano, Italy. Maristella Matera is a full professor of computer science and engineering in the department of electronics, information and bioengineering at Politecnico di Milano, Italy. Susanna Sancassani is the managing director of METID, Politecnico di Milano, Italy. Nicola Gatti is a full professor of computer science and engineering in the department of electronics, information and bioengineering at Politecnico di Milano, Italy.