Background: Research suggests that individuals with autism spectrum disorder (ASD) have difficulties in emotion recognition (ER), which could lead to social difficulties. ER can be enhanced through targeted interventions, but generalization to everyday functioning poses a challenge. Using dynamic multimodal emotional expressions for training may increase similarities to everyday situations. This pilot study investigated the feasibility of internet-based multimodal emotion recognition training (iMERAT) for adolescents with ASD.

Method: Eight adolescents with ASD and nine typically developing (TD) adolescents took part in the iMERAT intervention, which included brief online training sessions conducted each weekday during a 3-week period. Training was performed on dynamic facial, vocal and multimodal emotional expressions, with outcome feedback provided after each response. A survey was conducted to explore participants' experiences of the training. ER was measured pre- and post-training using a multimodal ER test.

Results: Participants reported that the training was moderately difficult, instructions were relatively easy to understand, and the duration of training was appropriate. Content analysis of open-ended responses suggested further adaptations, such as providing more explanations of emotions and further tailoring content and language for adolescents. ER increased from pre- to post-intervention, with large effect sizes for both ASD and TD adolescents.

Conclusion: Results suggest that the iMERAT intervention is feasible for adolescents with ASD. Gains in ER ability were observed, but the small sample size and lack of a control group render these findings tentative. Further research is required to assess the effectiveness of the iMERAT and possible impact on broader social skills.

An important task in criminal justice is to evaluate the accuracy of eyewitness testimony. In this study, we examined if machine learning could be used to detect accuracy. Specifically, we examined if support vector machines (SVMs) could accurately classify testimony statements as correct or incorrect based purely on the nonverbal aspects of the voice. We analyzed 3,337 statements (76.61% accurate) from 51 eyewitness testimonies along 94 acoustic variables. We also examined the relative importance of each of the acoustic variables, using Lasso regression. Results showed that the machine learning algorithms were able to predict accuracy between 20 and 40% above chance level (AUC = 0.50). The most important predictors included acoustic variables related to the amplitude (loudness) of speech and the duration of pauses, with higher amplitude predicting correct recall and longer pauses predicting incorrect recall. Taken together, we find that machine learning methods are capable of predicting whether eyewitness testimonies are correct or incorrect with above-chance accuracy and comparable to human performance, but without detrimental human biases. This offers a proof-of-concept for machine learning in evaluations of eyewitness accuracy, and opens up new avenues of research that we hope might improve social justice.

Conditions like congenital adrenal hyperplasia (CAH) and complete androgen insensitivity syndrome (CAIS) may provide information contributing to the explanation of sex differences in cognition. Using online tests and questionnaires, we examined how prenatal androgen exposure and/or sex chromosomes influence spatial ability, episodic memory, and emotion recognition in women with classic CAH (C-CAH; n = 29), non-classic CAH (NC-CAH; n = 13), CAIS (n = 11), and female (n = 147) and male (n = 142) controls. Results showed that (1) female and male controls differed on most cognitive tasks, whereas (2) women with C-CAH or CAIS did not consistently differ from either female or male controls. Investigating the relative advantage on either the female (episodic memory, emotion recognition) or male-favoring tasks (spatial ability), indicated that women with (3) C-CAH had a cognitive profile that was different from female and male controls, (4) CAIS were not different from male controls, whereas (5) NC-CAH had a relative advantage on female-favoring tasks. These findings suggest that excessive prenatal androgen exposure (C-CAH) may shift cognitive performance toward a male-typical pattern, though not to the male level. Additionally, aspects associated with having 46,XY karyotype, but lacking prenatal androgen receptivity (CAIS) may also influence cognition in a male-typical direction, providing mixed support for the prenatal androgen hypothesis.

Criminal-law workers must evaluate the accuracy of eyewitness testimony, which includes nonverbal speech characteristics that may carry clues. Using a large collection of testimony statements (N = 3344), Study 1 showed that eyewitnesses use a higher speech rate, fewer pauses, and greater acoustic amplitude when remembering correctly and more confident in their answers. In Study 2, observers with vs. without language comprehension (Swedish vs. American and Indian samples) judged the accuracy of testimony statements and judged the eyewitness’ confidence. Participants performed at above-chance levels regardless of nation or language comprehension, suggesting universal cues to accuracy. Participants gave higher confidence ratings to correct statements, and this calibration was higher with greater cultural proximity to the testimony source. Observers’ confidence judgments were significantly correlated with amplitude, which suggests they used valid acoustical cues. Taken together, results demonstrate distinct vocal markers of eyewitness accuracy, which are detected by listeners from diverse cultures.

Researchers have proposed that variation in sex hormones across the menstrual cycle modulate the ability to recognize emotions in others. Existing research suggests that accuracy is higher during the follicular phase and ovulation compared to the luteal phase, but findings are inconsistent. Using a repeated measures design with a sample of healthy naturally cycling women (N = 63), we investigated whether emotion recognition accuracy varied between the follicular and luteal phases, and whether accuracy related to levels of estrogen (estradiol) and progesterone. Two tasks assessed recognition of a range of positive and negative emotions via brief video recordings presented in visual, auditory, and multimodal blocks, and non-linguistic vocalizations (e.g., laughter, sobs, and sighs). Multilevel models did not show differences in emotion recognition between cycle phases. However, coefficients for estrogen were significant for both emotion recognition tasks. Higher within-person levels of estrogen predicted lower accuracy, whereas higher between-person estrogen levels predicted greater accuracy. This suggests that in general having higher estrogen levels increases accuracy, but that higher-than-usual estrogen at a given time decreases it. Within-person estrogen further interacted with cycle phase for both tasks and showed a quadratic relationship with accuracy for the multimodal task. In particular, women with higher levels of estrogen were more accurate in the follicular phase and middle of the menstrual cycle. We propose that the differing role of within- and between-person hormone levels could explain some of the inconsistency in previous findings.

The present study aimed to increase understanding of how singing activities may be initiated in primary school, and what support and assistance teachers require to conduct singing activities as an integrated part of the school day. Five music teachers participated in a focus group interview. The following main themes were identified: 1) pedagogical and methodological flexibility, 2) the role of routines and familiarity, 3) the embodied and multimodal dimensions of singing, 4) the importance of accompaniment and instruments, 5) the experience of insecurity and obstacles and 6) the perceived synergies between singing and other learning activities. This knowledge may be important to integrate within music teacher education in order to secure singing’s place in schools.

Studies have reported substantial variability in emotion recognition ability (ERA) – an important social skill – but possible neural underpinnings for such individual differences are not well understood. This functional magnetic resonance imaging (fMRI) study investigated neural responses during emotion recognition in young adults (N=49) who were selected for inclusion based on their performance (high or low) during previous testing of ERA. Participants were asked to judge brief video recordings in a forced-choice emotion recognition task, wherein stimuli were presented in visual, auditory and multimodal (audiovisual) blocks. Emotion recognition rates during brain scanning confirmed that individuals with high (vs. low) ERA received higher accuracy for all presentation blocks. fMRI-analyses focused on key regions of interest (ROIs) involved in the processing of multimodal emotion expressions, based on previous meta-analyses. In neural response to emotional stimuli contrasted with neutral stimuli, individuals with high (vs. low) ERA showed higher activation in the following ROIs during the multimodal condition: right middle superior temporal gyrus (mSTG), right posterior superior temporal sulcus (PSTS), and right inferior frontal cortex (IFC). Overall, results suggest that individual variability in ERA may be reflected across several stages of decisional processing, including extraction (mSTG), integration (PSTS) and evaluation (IFC) of emotional information.

People frequently report feeling more than one emotion at the same time (i.e., blended emotions), but studies on nonverbal communication of such complex states remain scarce. Actors (N = 18) expressed blended emotions consisting of all pairwise combinations of anger, disgust, fear, happiness, and sadness – using facial gestures, body movement, and vocal sounds – with the intention that both emotions should be equally prominent in the resulting expression. Accuracy of blended emotion recognition was assessed in two preregistered studies using a combined forced-choice and rating scale task. For each recording, participants were instructed to choose two scales (out of 5 available scales: anger, disgust, fear, happiness, and sadness) that best described their perception of the emotional content and judge how clearly each of the two chosen emotions were perceived. Study 1 (N = 38) showed that all emotion combinations were accurately recognized from multimodal (facial/bodily/vocal) expressions, with significantly higher ratings on scales corresponding to intended vs. non-intended emotions. Study 2 (N = 51) showed that all emotion combinations were also accurately perceived when the recordings were presented in unimodal visual (facial/bodily) and auditory (vocal) conditions, although accuracy was lower in the auditory condition. To summarize, results suggest that blended emotions, including combinations of both same-valence and other-valence emotions, can be accurately recognized from dynamic facial/bodily and vocal expressions. The validated recordings of blended emotion expressions are freely available for research purposes. 

Background: Psychopathic traits have been associated with impaired emotion recognition in criminal, clinical and community samples. A recent study however, suggested that cognitive impairment reduced the relationship between psychopathy and emotion recognition. We therefore investigated if reasoning ability and psychomotor speed were impacting emotion recognition in individuals with psychotic spectrum disorders (PSD) with and without a history of aggression, as well as in healthy individuals, more than self-rated psychopathy ratings on the Triarchic Psychopathy Measure (TriPM).

Methods: Eighty individuals with PSD (schizophrenia, schizoaffective disorder, delusional disorder, other psychoses, psychotic bipolar disorder) and documented history of aggression (PSD+Agg) were compared with 54 individuals with PSD without prior aggression (PSD-Agg) and with 86 healthy individuals on the Emotion Recognition Assessment in Multiple Modalities (ERAM test). Individuals were psychiatrically stable and in remission from possible substance use disorders. Scaled scores on matrix reasoning, averages of dominant hand psychomotor speed and self-rated TriPM scores were obtained.

Results: Associations existed between low reasoning ability, low psychomotor speed, patient status and prior aggression with total accuracy on the ERAM test. PSD groups performed worse than the healthy group. Whole group correlations between total and subscale scores of TriPM to ERAM were found, but no associations with TriPM scores within each group or in general linear models when accounting for reasoning ability, psychomotor speed, understanding of emotion words and prior aggression.

Conclusion: Self-rated psychopathy was not independently linked to emotion recognition in PSD groups when considering prior aggression, patient status, reasoning ability, psychomotor speed and emotion word understanding.

In two studies, we examined vocal characteristics of accuracy. Participants watched a staged-crime film and were interviewed as eyewitnesses. A mega- analysis showed that correct responses were uttered with 1) a higher pitch, 2) greater energy in the first formant region, 3) higher speech rate and 4) shorter pauses.