Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Assessing Performance Competence in Training Games
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences. University of Skövde, Sweden; University of Colombo School of Computing, Sri Lanka.
Stockholm University, Faculty of Social Sciences, Department of Computer and Systems Sciences.
Show others and affiliations
2011 (English)In: Affective Computing and Intelligent Interaction: Proceedings, Part II / [ed] Sidney D’Mello, Arthur Graesser, Björn Schuller, Jean-Claude Martin, Springer Berlin/Heidelberg, 2011, 518-527 p.Conference paper, Published paper (Refereed)
Abstract [en]

In-process assessment of trainee learners in game-based simulators is a challenging activity. This typically involves human instructor time and cost, and does not scale to the one tutor per learner vision of computer-based learning. Moreover, evaluation from a human instructor is often subjective and comparisons between learners are not accurate. Therefore, in this paper, we propose an automated, formula-driven quantitative evaluation method for assessing performance competence in serious training games. Our proposed method has been empirically validated in a game-based driving simulator using 7 subjects and 13 sessions, and accuracy up to 90.25% has been achieved when compared to an existing qualitative method. We believe that by incorporating quantitative evaluation methods like these future training games could be enriched with more meaningful feedback and adaptive game-play so as to better monitor and support player motivation, engagement and learning performance.

Place, publisher, year, edition, pages
Springer Berlin/Heidelberg, 2011. 518-527 p.
Series
Lecture Notes in Computer Science, ISSN 0302-9743 ; 6975
Keyword [en]
Serious games, Performance, Evaluation, Motivation, Driver training
National Category
Information Science
Research subject
Computer and Systems Sciences
Identifiers
URN: urn:nbn:se:su:diva-65043DOI: 10.1007/978-3-642-24571-8_65ISBN: 978-3-642-24570-1 (print)ISBN: 978-3-642-24571-8 (print)OAI: oai:DiVA.org:su-65043DiVA: diva2:460719
Conference
Fourth International Conference, ACII 2011, Memphis, TN, USA, October 9–12, 2011
Available from: 2011-12-01 Created: 2011-12-01 Last updated: 2015-11-11Bibliographically approved
In thesis
1. Validating User Engagement and Effectiveness of Training Simulations: A mixed-methods approach informed by embodied cognition and psychophysiological measures
Open this publication in new window or tab >>Validating User Engagement and Effectiveness of Training Simulations: A mixed-methods approach informed by embodied cognition and psychophysiological measures
2015 (English)Doctoral thesis, comprehensive summary (Other academic)
Alternative title[sv]
Validering av användarengagemang och effektivitet hos träningssimulatorer : En kombinerad metodansats informerad av kroppslig kognition och psyko-fysiologiska mått
Abstract [en]

Simulation-based training has gained widespread attention recently as a response to drawbacks associated with traditional training approaches, such as high training costs (instructors, equipment, etc.), high risks (e.g. pilot training), and ethical issues (e.g. medical training), as well as a lack of availability of certain training environments (e.g. space exploration). Apart from their target training domains, many of aspects of simulations differ, such as their degree of physical realism (fidelity), scenarios (e.g. story), and pedagogical aspects (e.g. after-action reviews and collaborative learning). Among those aspects, designers have mostly focused on developing high-fidelity simulations with the expectation of increasing the effectiveness of training. However, some authors suggest that the above belief is a myth as researchers have failed to identify a linear relationship between the (physical) fidelity and training effectiveness of simulations.  Most researchers have therefore evaluated the correspondence between the behaviours of trainees in both real world and simulated contexts, however, the existing methods of simulation validation using behavioural measures have a number of drawbacks, such as the fact that they do not address certain complex phenomena of skills acquisition.

Bridging the above knowledge gap, this research reports on empirical investigations using an improved methodology for validating training simulations. This research includes an investigation of the user experience of trainees, with respect to the acceptance of virtual scenarios provoking a similar psychophysiological response as in real world scenarios, and the training potential of simulations with respect to the positive transfer of training from a simulator to real world operational contexts. The most prominent features of the proposed methodology include the use of psychophysiological measures in addition to traditional behavioural measures and the use of natural (quasi-) experiments. Moreover, its conceptual framework was influenced by contemporary theories in cognitive science (e.g. constructivism and embodied cognition). The results of this research have several important theoretical and methodological implications, involving, for example, the dependency of the effectiveness of simulations on the perceived realism of trainees, which is more embodied than has been predicted by previous researchers, and the requirement of several different types/levels of adaptive training experience, depending on the type of trainee.

Abstract [sv]

Träning i simulatorer har på senare år fått ökad uppmärksamhet som en respons på problem och svårigheter förknippade med traditionella träningsansatser, såsom höga kostnader (instruktörer och utrustning, etc.), hög risk (t.ex. träning av piloter), och etiska aspekter (t.ex. träning av kirurger), likaväl som avsaknaden av träningsmöjligheter och miljöer (t.ex. forskning om rymden). Bortsett från vad som specifikt tränas så skiljer sig simuleringar åt i ett flertal olika aspekter såsom fysisk realism (eng. fidelity), scenarier (handling) och pedagogiska aspekter (t.ex. genomgång efter övning och kollaborativt lärande).  Bland dessa aspekter så har designers ofta fokuserat att utveckla simuleringar med hög realism med förväntningen att detta ska göra träningen mer effektiv. Litteraturen antyder dock att denna föreställning inte stämmer och att de flesta simuleringar med hög realism inte har lyckats uppnå denna målsättning. En slutsats är därför att det finns ett behov av metoder som kan validera potentialen hos simuleringar avsedda att stödja träning – redan innan dessa används.

Enligt litteraturen så är utbildningspotentialen hos en simulering starkt kopplad till hur väl den psykologiska effekten en simulering har, stämmer överens med en verklig upplevelse. Forskning har emellertid identifierat ett flertal svagheter hos existerande ansatser för att validera simuleringar; de är oftast baserade på prestations- och/eller subjektiva mätningar; de har fokuserat en eller ett fåtal psykologiska aspekter; och de bygger på traditionella teorier. Baserat på resultat från studier av en kör-simulator presenteras och föreslås i denna avhandling ett förbättrat ramverk för utvärdering. De mest centrala egenskaperna hos det föreslagna ramverket inbegriper användandet av psyko-fysiologiska mått tillsammans med mer traditionella mått; det konceptuella ramverket bygger på samtida teoretiska ansatser (tex konstruktivism och kroppslig kognition); samt användandet av fält (kvasi-) experiment. Utöver uppnåendet av uppsatta mål för forskningen så har resultaten ett flertal teoretiska och metodologiska implikationer. Bland dessa återfinns beroendet mellan effektiviteten hos en simulering och den upplevelse av realitet som de tränade har, vilken är mer grundläggande än vad som rapporterats i tidigare forskning, samt kravet på flera och olika typer av anpassning av träningsupplevelse för den tränade för att förhöja potentialen hos träningssimulatorer.

Place, publisher, year, edition, pages
Stockholm: Department of Computer and Systems Sciences, Stockholm University, 2015. 125 p.
Series
Report Series / Department of Computer & Systems Sciences, ISSN 1101-8526 ; 15-015
Keyword
training simulators, simulation validation, psychological fidelity, psychophysiological measures, embodied cognition, electroencephalography (EEG), galvanic skin response (GSR), träningssimulatorer, validering av simuleringar, psykologisk realism, psyko-fysiologiska mått, kroppslig kognition, elektroencefalografi (EEG), galvanisk hudrespons (GSR)
National Category
Human Computer Interaction
Research subject
Computer and Systems Sciences
Identifiers
urn:nbn:se:su:diva-122546 (URN)978-91-7649-305-2 (ISBN)
Public defence
2015-12-09, L30, NOD-huset, Borgarfjordsgatan 12, Kista, 13:00 (English)
Opponent
Supervisors
Projects
SIDA Funded National e-Learning Centre Project at the University of Colombo School of Computing, Sri Lanka
Available from: 2015-11-17 Created: 2015-11-04 Last updated: 2015-11-09Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Ekanayake, HiranRamberg, Robert
By organisation
Department of Computer and Systems Sciences
Information Science

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 68 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf