CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Explainable AI for Data-Driven Feedback and Intelligent Action Recommendations to Support Student Self-Regulation
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences.ORCID iD: 0000-0003-2054-0971
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Self-regulated learning (SRL) is a cognitive ability with demonstrable significance in facilitating students’ ability to effectively strategize, monitor, and assess their own learning actions. Studies have indicated that a lack of selfregulated learning skills negatively impacts students’ academic performance. Effective data-driven feedback and action recommendations are considered crucial for SRL and significantly influence student learning and performance. However, the task of delivering personalised feedback to every student poses a significant challenge for teachers. Moreover, the task of identifying appropriate learning activities and resources for individualised recommendations poses a significant challenge for teachers, given the large number of students enrolled in most courses.

To address these challenges, several studies have examined how learning analytics-based dashboards can support students’ self-regulation. These dashboards offered several visualisations (as feedback) on student success and failure. However, while such feedback may be beneficial, it does not offer insightful information or actionable recommendations to help students improve academically. Explainable artificial intelligence (xAI) approaches have been proposed to explain such feedback and generate insights from predictive models, with a focus on the relevant actions a student needs to take to improve in ongoing courses. Such intelligent activities could be offered to students as data-driven behavioural change recommendations.

This thesis offers an xAI-based approach that predicts course performance and computes informative feedback and actionable recommendations to promote student self-regulation. Unlike previous research, this thesis integrates a predictive approach with an xAI approach to analyse and manipulate students’ learning trajectories. The aim is to offer detailed, data-driven actionable feedback to students by providing in-depth insights and explanations for the predictions provided by the approach. The technique provides students with more practical and useful knowledge compared to the predictions alone.

The proposed approach was implemented in the form of a dashboard to support self-regulation by students in university courses, and it was evaluated to determine its effects on the students’ academic performance. The results revealed that the dashboard significantly enhanced students’ learning achievements and improved their self-regulated learning skills. Furthermore, it was found that the recommendations generated by the proposed approach positively affected students’ performance and assisted them in self-regulation.

Place, publisher, year, edition, pages
Stockholm: Department of Computer and Systems Sciences, Stockholm University , 2024. , p. 109
Series
Report Series / Department of Computer & Systems Sciences, ISSN 1101-8526 ; 24-007
Keywords [en]
Self-regulated learning, Explainable artificial intelligence, Counterfactual explanations, Intelligent recommendations, Self-regulation, Informative feedback
National Category
Computer and Information Sciences
Research subject
Computer and Systems Sciences
Identifiers
URN: urn:nbn:se:su:diva-232365ISBN: 978-91-8014-883-2 (print)ISBN: 978-91-8014-884-9 (electronic)OAI: oai:DiVA.org:su-232365DiVA, id: diva2:1889043
Public defence
2024-09-30, L30, NOD-huset, Borgarfjordsgatan 12, Kista, 13:00 (English)
Opponent
Supervisors
Available from: 2024-09-05 Created: 2024-08-14 Last updated: 2024-08-28Bibliographically approved
List of papers
1. A Systematic Review of Software for Learning Analytics in Higher Education
Open this publication in new window or tab >>A Systematic Review of Software for Learning Analytics in Higher Education
2024 (English)In: International Journal of Emerging Technologies in Learning (iJET), E-ISSN 1863-0383Article in journal (Refereed) Accepted
Abstract [en]

Learning analytics (LA) is an important area of research in technology-enhanced learning that has emerged during the last decade. In earlier years, several systematic reviews have been conducted that focused on the theories behind LA or on empirical studies that utilised LA-based methods to improve learning and teaching processes in higher education. However, to date, there has been no systematic review of papers that have adopted a software perspective to report on the many forms of learning analytics software (LAS) that have been developed, despite these being used more frequently than before in higher education to support learning and teaching processes. To fill this gap, this paper presents a systematic review of LAS with the aim of critically scrutinising the ways in which the use of interactive software in real-world settings may both support students in improving their academic performance and assist teachers in various pedagogical practices. A thematic analysis of 75 articles was conducted, resulting in the identification of three categories of LAS: at-risk student identification software; self-regulation software; and collaborative learning software. For each of these categories, we analysed (i) the embedded functionality; (ii) the stakeholder (teacher and student) for which the functionality is intended; (iii) the analytical and visualisation approaches implemented; and (iv) the limitations of the software that require future attention. Based on the findings of our review, we propose future directions for the development of LAS.

Keywords
learning analytics, learning analytics software, systematic review, identification of at-risk students, computer-supported collaborative learning, self-regulated learning
National Category
Computer and Information Sciences
Identifiers
urn:nbn:se:su:diva-232298 (URN)
Available from: 2024-08-12 Created: 2024-08-12 Last updated: 2024-09-13
2. Generation of Automatic Data-Driven Feedback to Students Using Explainable Machine Learning
Open this publication in new window or tab >>Generation of Automatic Data-Driven Feedback to Students Using Explainable Machine Learning
Show others...
2021 (English)In: Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021, Proceedings, Part II / [ed] Ido Roll; Danielle McNamara; Sergey Sosnovsky; Rose Luckin; Vania Dimitrova, Springer , 2021, p. 37-42Conference paper, Published paper (Refereed)
Abstract [en]

This paper proposes a novel approach that employs learning analytics techniques combined with explainable machine learning to provide automatic and intelligent actionable feedback that supports students self-regulation of learning in a data-driven manner. Prior studies within the field of learning analytics predict students’ performance and use the prediction status as feedback without explaining the reasons behind the prediction. Our proposed method, which has been developed based on LMS data from a university course, extends this approach by explaining the root causes of the predictions and automatically provides data-driven recommendations for action. The underlying predictive model effectiveness of the proposed approach is evaluated, with the results demonstrating 90 per cent accuracy.

Place, publisher, year, edition, pages
Springer, 2021
Series
Lecture Notes in Computer Science (LNCS), ISSN 0302-9743, E-ISSN 1611-3349 ; 12749
Keywords
Learning analytics, Explainable machine learning, Feedback provision, Recommendations generation, Dashboard
National Category
Computer Sciences
Research subject
Computer and Systems Sciences
Identifiers
urn:nbn:se:su:diva-200483 (URN)10.1007/978-3-030-78270-2_6 (DOI)978-3-030-78270-2 (ISBN)
Conference
22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14–18, 2021
Available from: 2022-01-05 Created: 2022-01-05 Last updated: 2024-08-14Bibliographically approved
3. Explainable AI for Data-Driven Feedback and Intelligent Action Recommendations to Support Students Self-Regulation
Open this publication in new window or tab >>Explainable AI for Data-Driven Feedback and Intelligent Action Recommendations to Support Students Self-Regulation
Show others...
2021 (English)In: Frontiers in Artificial Intelligence, E-ISSN 2624-8212, Vol. 4, article id 723447Article in journal (Refereed) Published
Abstract [en]

Formative feedback has long been recognised as an effective tool for student learning, and researchers have investigated the subject for decades. However, the actual implementation of formative feedback practices is associated with significant challenges because it is highly time-consuming for teachers to analyse students’ behaviours and to formulate and deliver effective feedback and action recommendations to support students’ regulation of learning. This paper proposes a novel approach that employs learning analytics techniques combined with explainable machine learning to provide automatic and intelligent feedback and action recommendations that support student’s self-regulation in a data-driven manner, aiming to improve their performance in courses. Prior studies within the field of learning analytics have predicted students’ performance and have used the prediction status as feedback without explaining the reasons behind the prediction. Our proposed method, which has been developed based on LMS data from a university course, extends this approach by explaining the root causes of the predictions and by automatically providing data-driven intelligent recommendations for action. Based on the proposed explainable machine learning-based approach, a dashboard that provides data-driven feedback and intelligent course action recommendations to students is developed, tested and evaluated. Based on such an evaluation, we identify and discuss the utility and limitations of the developed dashboard. According to the findings of the conducted evaluation, the dashboard improved students’ learning outcomes, assisted them in self-regulation and had a positive effect on their motivation.

Keywords
self-regulated learning, recommender system, automatic data-driven feedback, explainable machine learning-based approach, dashboard, learning analytics, AI
National Category
Computer Sciences
Research subject
Computer and Systems Sciences
Identifiers
urn:nbn:se:su:diva-200479 (URN)10.3389/frai.2021.723447 (DOI)000751704800142 ()
Available from: 2022-01-05 Created: 2022-01-05 Last updated: 2024-08-14Bibliographically approved
4. Informative Feedback and Explainable AI-Based Recommendations to Support Students' Self-regulation
Open this publication in new window or tab >>Informative Feedback and Explainable AI-Based Recommendations to Support Students' Self-regulation
2024 (English)In: Technology, Knowledge and Learning, ISSN 2211-1662, E-ISSN 2211-1670, Vol. 29, no 1, p. 331-354Article in journal (Refereed) Published
Abstract [en]

Self-regulated learning is an essential skill that can help students plan, monitor, and reflect on their learning in order to achieve their learning goals. However, in situations where there is a lack of effective feedback and recommendations, it becomes challenging for students to self-regulate their learning. In this paper, we propose an explainable AI-based approach to provide automatic and intelligent feedback and recommendations that can support the self-regulation of students' learning in a data-driven manner, with the aim of improving their performance on their courses. Prior studies have predicted students' performance and have used these predicted outcomes as feedback, without explaining the reasons behind the predictions. Our proposed approach is based on an algorithm that explains the root causes behind a decline in student performance, and generates data-driven recommendations for taking appropriate actions. The proposed approach was implemented in the form of a dashboard to support self-regulation by students on a university course, and was evaluated to determine its effects on the students' academic performance. The results revealed that the dashboard significantly enhanced students' learning achievements and improved their self-regulated learning skills. Furthermore, it was found that the recommendations generated by the proposed approach positively affected students' performance and assisted them in self-regulation

Keywords
Self-regulated learning, Explainable artificial intelligence, Counterfactual explanations, Intelligent recommendations, Self-regulation, Informative feedback
National Category
Learning Information Systems
Identifiers
urn:nbn:se:su:diva-217020 (URN)10.1007/s10758-023-09650-0 (DOI)000975441100001 ()2-s2.0-85153386727 (Scopus ID)
Available from: 2023-05-15 Created: 2023-05-15 Last updated: 2024-08-14Bibliographically approved
5. A Transformer-Based Approach for the Automatic Generation of Concept-Wise Exercises to Provide Personalized Learning Support to Students
Open this publication in new window or tab >>A Transformer-Based Approach for the Automatic Generation of Concept-Wise Exercises to Provide Personalized Learning Support to Students
2023 (English)In: Responsive and Sustainable Educational Futures: 18th European Conference on Technology Enhanced Learning, EC-TEL 2023, Aveiro, Portugal, September 4–8, 2023, Proceedings / [ed] Olga Viberg; Ioana Jivet; Pedro J. Muñoz-Merino; Maria Perifanou; Tina Papathoma, Cham: Springer, 2023, p. 16-31Conference paper, Published paper (Refereed)
Abstract [en]

Providing personalized support to students during courses is essential to facilitate them in their desired learning goals and reduce the dropout rate. Although teachers can play an effective role in providing personalized support, achieving individual-level assistance for massive courses becomes challenging. To overcome this challenge, this paper proposes a transformer-based approach that first models students’ knowledge of various course concepts based on their performance in various assessed tasks. Afterwards, the students’ concept-wise knowledge level derived from the models is combined with the available course material, leading to the generation of personalized concept-wise exercises by employing fine-tuned Text-to-Text Transfer Transformer (T5) architecture. These generated exercises help students to improve their knowledge about different course concepts. The proposed approach has been evaluated with various university courses to determine its quality, utility and effects on students’ academic performance. The evaluation results revealed that teachers and students were satisfied with the quality of the generated exercises, and these were found to be helpful for students to improve their concept-wise understanding. Furthermore, the generated exercises positively impacted students’ academic performance. 

Place, publisher, year, edition, pages
Cham: Springer, 2023
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 14200
National Category
Information Systems, Social aspects
Identifiers
urn:nbn:se:su:diva-232364 (URN)10.1007/978-3-031-42682-7_2 (DOI)2-s2.0-85171985863 (Scopus ID)978-3-031-42681-0 (ISBN)978-3-031-42682-7 (ISBN)
Conference
18th European Conference on Technology Enhanced Learning, (EC-TEL 2023), Aveiro, Portugal, September 4–8, 2023
Available from: 2024-08-14 Created: 2024-08-14 Last updated: 2024-09-18Bibliographically approved

Open Access in DiVA

Explainable AI for Data-Driven Feedback and Intelligent Action Recommendations to Support Student Self-Regulation(4317 kB)56 downloads
File information
File name FULLTEXT01.pdfFile size 4317 kBChecksum SHA-512
e54280b34d8ca43581c065dcf0729df3ab94bd2ea453e185e63adcbf2155b7d9e2e89c46b32aebb1e5f1ac474dbeb560c6e0fab9a1a70e603a7fcbe090c4d7d8
Type fulltextMimetype application/pdf

Authority records

Afzaal, Muhammad

Search in DiVA

By author/editor
Afzaal, Muhammad
By organisation
Department of Computer and Systems Sciences
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 56 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 760 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf