The ubiquity of wearable devices together with areas like internet of things, big data and machine learning have promoted the development of solutions for personalized healthcare that use digital sensors. However, there is a lack of an implemented framework that is technically feasible, easily scalable and that provides meaningful variables to be used in applications for translational medicine. This paper describes the implementation and early evaluation of a physiological sensing tool that collects and processes photoplethysmography data from a wearable smartwatch to calculate heart rate variability in real-time. A technical open-source framework is outlined, involving mobile devices for collection of heart rate data, feature extraction and execution of data mining or machine learning algorithms that ultimately deliver mobile health interventions tailored to the users. Eleven volunteers participated in the empirical evaluation that was carried out using an existing mobile virtual reality application for mental health and under controlled slow-paced breathing exercises. The results validated the feasibility of implementation of the proposed framework in the stages of signal acquisition and real-time calculation of heart rate variability (HRV). The analysis of data regarding packet loss, peak detection and overall system performance provided considerations to enhance the real-time calculation of HRV features. Further studies are planned to validate all the stages of the proposed framework.

Crucial elements for police firearms training include mastering very specific psychophysiological responses associated with controlled breathing while shooting. Under high-stress situations, the shooter is affected by responses of the sympathetic nervous system that can impact respiration. This research focuses on how frontal oscillatory brainwaves and cardiovascular responses of trained police officers (N = 10) are affected during a virtual reality (VR) firearms training routine. We present data from an experimental study wherein shooters were interacting in a VR-based training simulator designed to elicit psychophysiological changes under easy, moderate and frustrating difficulties. Outcome measures in this experiment include electroencephalographic and heart rate variability (HRV) parameters, as well as performance metrics from the VR simulator. Results revealed that specific frontal areas of the brain elicited different responses during resting states when compared with active shooting in the VR simulator. Moreover, sympathetic signatures were found in the HRV parameters (both time and frequency) reflecting similar differences. Based on the experimental findings, we propose a psychophysiological model to aid the design of a biocybernetic adaptation layer that creates real-time modulations in simulation difficulty based on targeted physiological responses.

Bodily signals can complement subjective and behavioral measures to analyze human factors, such as user engagement or stress, when interacting with virtual reality (VR) environments. Enabling widespread use of (also the real-time analysis) of bodily signals in VR applications could be a powerful method to design more user-centric, personalized VR experiences. However, technical and scientific challenges (e.g., cost of research-grade sensing devices, required coding skills, expert knowledge needed to interpret the data) complicate the integration of bodily data in existing interactive applications. This paper presents the design, development, and evaluation of an open-source software framework named Excite-O-Meter. It allows existing VR applications to integrate, record, analyze, and visualize bodily signals from wearable sensors, with the example of cardiac activity (heart rate and its variability) from the chest strap Polar H10. Survey responses from 58 potential users determined the design requirements for the framework. Two tests evaluated the framework and setup in terms of data acquisition/analysis and data quality. Finally, we present an example experiment that shows how our tool can be an easy-to-use and scientifically validated tool for researchers, hobbyists, or game designers to integrate bodily signals in VR applications.

In this paper, we present a method to classify head motion trajectories using recent time-series methods. The analysis of motion data with machine learning is a common technique to solve problems in Virtual Reality (VR), such as adaptive rendering or user behavioral modeling. Motion data are initially collected as time series, but they are usually transformed into tabular features compatible with traditional feature-based classifiers. Data mining research has proposed several time-series classifiers that can directly exploit the temporal relationship of the data without requiring manual feature extraction. Nevertheless, the effectiveness of these time-series methods still requires validation on real-life problem domains. Therefore, this paper demonstrates how a pipeline that combines a recent time-series classifier with two rotation space representations (quaternion and Euler) can successfully analyze head motion in VR applications. We test the proposed method on two public datasets containing head rotations, resulting in higher prediction accuracy than other feature-based and time-series classifiers. We also discuss some limitations, guidelines for future work, and concluding remarks.

Designing affect-based personalized technology involves dealing with large datasets. Machine learning (ML) algorithms are employed to predict affect and similar human factors from in-game metrics, behavioral patterns, or physiological responses. The classification performance is usually presented as a global point estimate without providing user-specific interpretations. This approach is incompatible with effective personalization in games because it disregards the variability of body responses between players. This paper proposes a methodology to classify subjective human factors from large multimodal data. A public VR dataset (CEAP-360VR) was used to extensively compare three ML classifiers and five feature importance techniques. The produced models could reduce the original feature space by 82% (from 113 to 20 features) without compromising predictive performance (F1 score). A random forest (RF) using forward sequential feature selection (fSFS) yielded the best prediction of binary valence (F1=0.761) and arousal (F1=0.748). Finally, feature importance rankings are discussed with emphasis on global and user-specific patterns that may improve affect recognition. The proposed methodology is envisioned to help game designers and researchers create customized user-centric games and VR experiences inferring possible explanations from multimodal datasets.