Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Hello GPT! Goodbye home examination? An exploratory study of AI chatbots impact on university teachers' assessment practices
Stockholm University, Faculty of Social Sciences, Department of Education.ORCID iD: 0000-0001-7601-3850
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences.ORCID iD: 0000-0001-6389-0467
Stockholm University, Faculty of Social Sciences, Department of Education.ORCID iD: 0000-0002-3345-3810
Stockholm University, Faculty of Social Sciences, Department of Education.ORCID iD: 0000-0002-8215-3646
Number of Authors: 42024 (English)In: Assessment & Evaluation in Higher Education, ISSN 0260-2938, E-ISSN 1469-297X, Vol. 49, no 3, p. 363-375Article in journal (Refereed) Published
Abstract [en]

AI chatbots have recently fuelled debate regarding education practices in higher education institutions worldwide. Focusing on Generative AI and ChatGPT in particular, our study examines how AI chatbots impact university teachers' assessment practices, exploring teachers' perceptions about how ChatGPT performs in response to home examination prompts in undergraduate contexts. University teachers (n = 24) from four different departments in humanities and social sciences participated in Turing Test-inspired experiments, where they blindly assessed student and ChatGPT-written responses to home examination questions. Additionally, we conducted semi-structured interviews in focus groups with the same teachers examining their reflections about the quality of the texts they assessed. Regarding chatbot-generated texts, we found a passing rate range across the cohort (37.5 - 85.7%) and a chatbot-written suspicion range (14-23%). Regarding the student-written texts, we identified patterns of downgrading, suggesting that teachers were more critical when grading student-written texts. Drawing on post-phenomenology and mediation theory, we discuss AI chatbots as a potentially disruptive technology in higher education practices.

Place, publisher, year, edition, pages
2024. Vol. 49, no 3, p. 363-375
Keywords [en]
AI-chatbots, assessment, higher education, home examination, Turing test
National Category
Pedagogy
Identifiers
URN: urn:nbn:se:su:diva-220860DOI: 10.1080/02602938.2023.2241676ISI: 001040685700001Scopus ID: 2-s2.0-85166200345OAI: oai:DiVA.org:su-220860DiVA, id: diva2:1796534
Available from: 2023-09-12 Created: 2023-09-12 Last updated: 2026-03-18Bibliographically approved
In thesis
1. Emerging AI and Ethics in Higher Education: A Technology Mediation Perspective
Open this publication in new window or tab >>Emerging AI and Ethics in Higher Education: A Technology Mediation Perspective
2026 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The emergence of AI has become a defining issue for higher education worldwide, and Sweden is no exception. At the same time, emerging AI technologies reconfigure priorities and valuations within educational practice by mediating teaching and learning and opening new paths for how teachers and students relate to higher education practices. In this context, AI-mediated practices raise ethical questions that are often presented as unprecedented yet are deeply rooted in longstanding practices in higher education. This thesis undertakes an empirical exploration of AI-mediated practices in higher education, foregrounding teachers’ perspectives and focusing on the ethical issues arising from such mediations. Drawing on postphenomenology and technology mediation theory, the thesis examines how teachers perceive and experience emerging AI artefacts (automated grading systems (AGS) and generative AI (GAI) chatbots) in relation to their practices, and how these artefacts mediate their understandings of what they ought to do and how they ought to act when balancing sometimes competing demands of autonomy and accountability.

The thesis is a compilation of four complementary studies. Study I examines the ethical considerations of AGS, reviewing the literature on AGS and analysing their specificities through a relational ethics approach. This study highlighted that AGS not only introduce technical and procedural considerations but also reconfigure educational practices and relationships in ways that demand ongoing, situated, and relationally attuned ethical reflection. Study II is an interview study with AGS developers who are also university teachers using these systems. It examines their expectations, experiences, and the disruptions that AGS introduce into assessment practices. The findings underscore the ambivalent role of AGS as both promising and disruptive, offering efficiency and consistency, but also introducing ‘new’ frictions and ethical dilemmas. Study III is a study inspired by the Turing test, followed by focus group interviews with university teachers. It explores how GAI chatbots mediate teachers’ perceptions of their assessment practices. The findings indicate that the presence of GAI chatbots, allowing the possibility of AI-generated writing, shapes evaluation practices, prompting teachers to question authorship and, in some cases, reinforcing mistrust within the student–teacher relationship. Study IV is a focus group interview study examining how teachers experience and interpret the emergence of GAI and how it mediates their perceptions of their professional roles. Participants described GAI as both disruptive and potentially transformative. They were compelled to reconsider assessment formats, teaching priorities, and their responsibility to foster critical and ethical engagement with technology. 

The combined findings of the four studies show that the emergence of AI unsettles established practices and intensifies the uncertainties that characterise educational situations, placing greater demands on teachers’ professional judgment. The thesis also argues that the emergence of AI exposes and amplifies longstanding ethical issues, such as fairness, academic integrity, and equity, reshaping how these issues are understood and enacted as the technologies become embedded in higher education practices.

Place, publisher, year, edition, pages
Stockholm: Department of Education, Stockholm University, 2026. p. 124
Series
Doktorsavhandlingar från Institutionen för pedagogik och didaktik ; 89
Keywords
Higher education, emerging technologies, ethics, technology mediation theory, university teachers, artificial intelligence, relational ethics, postphenomenology, automation, generative artificial intelligence, chatbots, automated grading systems
National Category
Educational Sciences Artificial Intelligence Ethics
Research subject
Education
Identifiers
urn:nbn:se:su:diva-253560 (URN)978-91-8107-552-6 (ISBN)978-91-8107-553-3 (ISBN)
Public defence
2026-05-08, Lilla hörsalen, Naturhistoriska riksmuset, Frescativägen 40, Stockholm, 10:00 (English)
Opponent
Supervisors
Projects
Ethical and Legal Challenges in Relationship to AI-Driven Practices in Higher Education
Funder
Wallenberg AI, Autonomous Systems and Software Program – Humanity and Society (WASP-HS), MMW2020.0138
Available from: 2026-04-15 Created: 2026-03-18 Last updated: 2026-03-31Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Farazouli, AlexandraCerratto-Pargman, TeresaBolander Laksov, KlaraMcGrath, Cormac

Search in DiVA

By author/editor
Farazouli, AlexandraCerratto-Pargman, TeresaBolander Laksov, KlaraMcGrath, Cormac
By organisation
Department of EducationDepartment of Computer and Systems Sciences
In the same journal
Assessment & Evaluation in Higher Education
Pedagogy

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 534 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf