The digital transformation in education has created many opportunities but also made it challenging to navigate the growing landscape of digital learning materials. The volume and diversity of learning resources create challenges for both educators and learners to identify and utilize the most relevant resources based on specific learning contexts. In light of this, there is a critical demand for systems capable of effectively connecting different learning materials to support teaching and learning activities and, for that purpose, natural language processing can be used to provide some of the essential building blocks for educational content recommendation systems. Hence, this thesis explores the use of natural language processing techniques for automatically linking and recommending relevant learning resources in the form of textbook content, exercises and curriculum goals. A key question is how to represent diverse learning materials effectively and, to that end, various language models are explored; the obtained representations are then used for measuring semantic textual similarity between learning materials. Learning materials can also be represented based on educational concepts, which is investigated in an ontology-based linking approach. To further enhance the representations and improve linking performance, different language models can be combined and augmented using external knowledge in the form of knowledge graphs and knowledge bases. Beyond approaches based on semantic textual similarity, prompting large language models is explored and a method based on retrieval-augmented generation (RAG) to improve linking performance is proposed. 

The thesis presents a systematic empirical evaluation of natural language processing techniques for representing and linking digital learning content, spanning different types of learning materials, use cases, and subjects. The results demonstrate the feasibility of unsupervised approaches based on semantic textual similarity of representations derived from pre-trained language models, and that contextual embeddings outperform traditional text representation methods. Furthermore, zero-shot prompting of large language models can outperform methods based on semantic textual similarity, leveraging RAG to exploit an external knowledge base in the form of a digital textbook. The potential practical applications of the proposed approaches for automatic linking of digital learning materials pave the way for the development of intelligent and adaptive learning systems, including intelligent textbooks.

Using AI for automatically linking exercises to curriculum goals can support many educational use cases and facilitate teaching-to-the-curriculum by ensuring that exercises adequately reflect and encompass the curriculum goals, ultimately enabling curriculum-based assessment. Here, we introduce this novel task and create a manually labeled dataset where two types of diagnostic tests are linked to curriculum goals for Biology G7-9 in Sweden. We cast the problem both as an information retrieval task and a multi-class text classification task and explore unsupervised approaches to both, as labeled data for such tasks is typically scarce. For the information retrieval task, we employ SOTA embedding model ADA-002 for semantic textual similarity (STS), while we prompt a large language model in the form of ChatGPT to classify diagnostic tests into curriculum goals. For both task formulations, we investigate different ways of using textbook content as a pivot and provide additional context for linking diagnostic tests to curriculum goals. We show that a combination of the two approaches in a retrieval-augmented generation model, whereby STS is used for retrieving textbook content as context to ChatGPT that then performs zero-shot classification, leads to the best classification accuracy (73.5%), outperforming both STS-based classification (67.5%) and LLM-based classification without context (71.5%). Finally, we showcase how the proposed method could be used in pedagogical practices.

Automated Essay Scoring (AES) automatically allocates scores to essays at scale and may help teachers reduce the heavy burden during grading activities. Recently, researchers have deployed neural-based AES approaches to improve upon the state-of-the-art AES performance. These neural-based AES methods mainly take student essays as the sole input and focus on learning the relationship between student essays and essay scores through deep neural networks. However, their only product, the predicted holistic score, is far from providing adequate pedagogical information, such as automated writing evaluation (AWE). In this work, we propose Topic-aware BERT, a new method of learning relations among scores, student essays, as well as topical information in essay instructions. Beyond improving the AES benchmark performance, Topic-aware BERT can automatically retrieve key topical sentences in student essays by probing self-attention maps in intermediate layers. We evaluate the performance of Topic-aware BERT of different variants to (i) perform AES and (ii) retrieve key topical sentences using the open dataset Automated Student Assessment Prize and a manually annotated dataset. Our experiments show that Topic-aware BERT achieves a strong AES performance compared with the previous best neural-based AES methods and demonstrates effectiveness in identifying key topical sentences in argumentative essays.

As an integral part of AI-enhanced learning, a content recommender automatically filters and recommends relevant learning materials to the learner or the instructor in a learning system. It can effectively help instructors in pedagogical practices and support students in self-regulated learning. Content recommendation technologies and applications have been studied extensively, however, the SOTA technologies have not adequately adapted to the education domain and there is very limited research on how different models and solutions can be applied in the Swedish context and for multiple subjects. In this paper, we develop a text similarity-based content recommender system. Specifically, given a quiz, we automatically recommend supportive learning resources as a reference to the answer and link back to the textbook sections where the examined knowledge points reside. We present a generic method for Swedish educational content recommendations using the most representative models, evaluate and analyze in multi-dimensions such as model types, pooling methods, subjects etc. The best results are obtained by Sentence-BERT (SBERT) with max paragraph-level pooling, outperforming traditional Natural Language Processing (NLP) models and knowledge graph-based models, obtaining on average 95% in Recall@3 and 82% in MRR, and outstanding in dealing with texts containing symbols, equations or calculations. This research provides empirical evidence and analysis, and can be used as a guidance when building a Swedish educational content recommender.

Practice and exam-style questions, as essential educational tools, contribute to educators’ effective teaching. Automatic question generation (QG) is a promising technique that can eliminate the manual effort of constructing questions and boost technology-enhanced education systems. Recently, deep neural network-based question-generation approaches have significantly improved upon state-of-the-art of question generation. Nevertheless, these approaches are often developed based on huge and non-educational datasets consisting of over 100,000 examples, which negatively affect the scalability and reliability of the educational QG systems. This study proposes a prompt-based learning QG approach that could generate questions in a data-effective way. The proposed prompt-based learning QG approach is trained and evaluated on a general dataset SQuAD, and an educational dataset SciQ. Experiment results demonstrate that our approach outperforms existing best QG models by a vast margin in data-effective scenarios and could generate high-quality educational questions with as few as 1,000 training examples.

Educational concepts are the core of teaching and learning. From the perspective of educational technology, concepts are essential meta-data, represen- tative terms that can connect different learning materials, and are the foundation for many downstream tasks. Some studies on automatic concept extraction have been conducted, but there are no studies looking at the K-12 level and focused on the Swedish language. In this paper, we use a state-of-the-art Swedish BERT model to build an automatic concept extractor for the Biology subject using fine- annotated digital textbook data that cover all content for K-12. The model gives a recall measure of 72% and has the potential to be used in real-world settings for use cases that require high recall. Meanwhile, we investigate how input data fea- tures influence model performance and provide guidance on how to effectively use text data to achieve the optimal results when building a named entity recognition (NER) model.

Automated Essay Scoring (AES) automatically assigns scores to essays at scale and may help to support teachers' grading activities. Recently, AES methods based on deep neural networks (DNN) have significantly improved upon the state-of-the-art performance by learning relations between holistic essay scores and student essays. However, DNN-based AES methods function like black-box, negatively affecting the ability to provide automated writing evaluation (AWE). In this work, we proposed a new method, topic-aware BERT, based on fine-tuning the pre-trained language model to learn relations between essay scores and text representations of student essays as well as topical information in essay writing instructions. Moreover, we propose an approach to automatically retrieve key topical sentences in student essays by probing self-attention maps in intermediate layers of topic-aware BERT. We evaluate the performance of topic-aware BERT to (i) perform AES and (ii) retrieve key topical sentences using the open dataset Automated Student Assessment Prize and a manually annotated dataset, respectively. Our model achieves a strong AES performance compared with previous state-of-the-art DNN-based methods and shows effectiveness in identifying key topical sentences in argumentative essays.

Today, collaborative learning has become quite central as a method for learning, and over the past decades, a large number of studies have demonstrated the benefits from various theoretical and methodological perspectives. This study proposes a novel approach that utilises Natural Language Processing(NLP) methods, particularly pre-trained word embeddings, to automatically create homogeneous or heterogeneous groups of students in terms of knowledge and knowledge gaps expressed in assessments. The two different ways of creating groups serve two different pedagogical purposes: (1) homogeneous group formation based on students’ knowledge can support and make teachers’ pedagogical activities such as feedback provision more time efficient, and (2) the heterogeneous groups can support and enhance collaborative learning. We evaluate the performance of the proposed approach through experiments with a dataset from a university course in programming didactics.

Knowledge tracing—where a machine models the students’ knowledge as they interact with coursework—is a well-established area in the field of Artificial Intelligence in Education. In this paper, an ensemble approach is proposed that addresses existing limitations in question-centric knowledge tracing and achieves the goal of predicting future question correctness. The proposed approach consists of two models; one is Light Gradient Boosting Machine (LightGBM) built by incorporating all relevant key features engineered from the data. The second model is a Multiheaded-Self-Attention Knowledge Tracing model (MSAKT) that extracts historical student knowledge of future question by calculating their contextual similarity with previously attempted questions. The proposed model’s effectiveness is evaluated by conducting experiments on a big Kaggle dataset achieving an Area Under ROC Curve (AUC) score of 0.84 with 84% accuracy using 10fold cross-validation.

In this paper, we propose a counterfactual explanations-based approach to provide an automatic and intelligent recommendation that supports student's self-regulation of learning in a data-driven manner, aiming to improve their performance in courses. Existing work under the fields of learning analytics and AI in education predict students' performance and use the prediction outcome as feedback without explaining the reasons behind the prediction. Our proposed approach developed an algorithm that explains the root causes behind student's performance decline and generates data-driven recommendations for action. The effectiveness of the proposed predictive model that constitutes the intelligent recommendations is evaluated, with results demonstrating high accuracy.