his.sePublications
Change search
Refine search result
1 - 10 of 10
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Bevilacqua, Fernando
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. Federal University of Fronteira Sul.
    Game-calibrated and user-tailored remote detection of emotions: A non-intrusive, multifactorial camera-based approach for detecting stress and boredom of players in games2017Report (Other academic)
    Abstract [en]

    Questionnaires and physiological measurements are the most common approach used to obtain data for emotion estimation in the field of human-computer interaction (HCI) and games research. Both approaches interfere with the natural behavior of users, which affects any research procedure. Initiatives based on computer vision and remote extraction of user signals for emotion estimation exist, however they are limited. Experiments of such initiatives have been performed under extremely controlled situations with few game-related stimuli. Users had a passive role with limited possibilities for interaction or emotional involvement, differently than game-based emotion stimuli, where users take an active role in the process, making decisions and directly interacting with the media. Previous works also focus on predictive models based on a group perspective. As a consequence, a model is usually trained from data of several users, which in practice describes the average behavior of the group, excluding or diluting key individualities of each user. In that light, there is a lack of initiatives focusing on non-obtrusive, user-tailored emotion detection models, in particular regarding stress and boredom, within the context of games research that is based on emotion data generated from game stimuli.

    This thesis proposal presents a research that aims to fill that gap, providing the HCI and the games research community with an emotion detection process, instantiated as a software tool, which can be used to remotely study user's emotions in a non-obtrusive way within the context of games. The main knowledge contribution of this research is a novel process for emotion detection that is remote (non-contact) and constructed from a game-based, multifactorial, user-tailored calibration phase. The process relies on computer vision and remote photoplethysmography (rPPG) to read user signals, e.g. heart rate (HR) and facial actions, without physical contact during the interaction with games to perform the detection of stress/boredom levels of users. The approach is automated and uses an ordinary camera to collect information, so specialized equipment, e.g. HR sensors, are not required.

    Current results of this research show that individualities can be detected regarding facial activity, e.g. increased number of facial actions during the stressful part of games. Regarding physiological signals, findings are aligned with and reinforce previous research that indicates higher HR mean during stressful situations in a gaming context. The findings also suggest that changes in the HR during gaming sessions are a promising indicator of stress, which can be incorporated into a model aimed at emotion detection. The literature reviews, the experiments conducted so far and the planned future tasks support the idea of using a set of signals, e.g. facial activity, body movement, and HR estimations as sources of information in a multifactorial analysis for the identification of stress and boredom in games. It will produce a novel user-tailored approach for emotion detection focused on the behavioral particularities of each user instead of the average group pattern. The proposed approach will be implemented as a software tool, which can be used by researchers and practitioners for games research.

  • 2.
    Bevilacqua, Fernando
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Game-calibrated and user-tailored remote detection of emotions: A non-intrusive, multifactorial camera-based approach for detecting stress and boredom of players in games2018Doctoral thesis, monograph (Other academic)
    Abstract [en]

    Questionnaires and physiological measurements are the most common approach used to obtain data for emotion estimation in the field of human-computer interaction (HCI) and games research. Both approaches interfere with the natural behavior of users. Initiatives based on computer vision and the remote extraction of user signals for emotion estimation exist, however they are limited. Experiments of such initiatives have been performed under extremely controlled situations with few game-related stimuli. Users had a passive role with limited possibilities for interaction or emotional involvement, instead of game-based emotion stimuli, where users take an active role in the process, making decisions and directly interacting with the media. Previous works also focus on predictive models based on a group perspective. As a consequence, a model is usually trained from the data of several users, which in practice describes the average behavior of the group, excluding or diluting key individualities of each user. In that light, there is a lack of initiatives focusing on non-obtrusive, user-tailored emotion detection models, in particular regarding stress and boredom, within the context of games research that is based on emotion data generated from game stimuli. This research aims to fill that gap, providing the HCI and the games research community with an emotion detection process that can be used to remotely study user's emotions in a non-obtrusive way within the context of games.

    The main knowledge contribution of this research is a novel process for emotion detection that is non-obtrusive, user-tailored and game-based. It uses remotely acquired signals, namely, heart rate (HR) and facial actions (FA), to create a user-tailored model, i.e. trained neural network, able to detect the emotional states of boredom and stress of a given subject. The process is automated and relies on computer vision and remote photoplethysmography (rPPG) to acquire user signals, so that specialized equipment, e.g. HR sensors, is not required and only an ordinary camera is needed. The approach comprises two phases: training (or calibration) and testing. In the training phase, a model is trained using a user-tailored approach, i.e. data from a given subject playing calibration games is used to create a model for that given subject. Calibration games are a novel emotion elicitation material introduced by this research. These games are carefully designed to present a difficulty level that constantly and linearly progresses over time without a pre-defined stopping point. They induce emotional states of boredom and stress, accounting for particularities at an individual level. Finally, the testing phase occurs in a game session involving a subject playing any ordinary, non-calibration game, e.g. Super Mario. During the testing phase, the subject's signals are remotely acquired and fed into the model previously trained for that particular subject. The model subsequently outputs the estimated emotional state of that given subject for that particular testing game.

    The method for emotion detection proposed in this thesis has been conceived on the basis of established theories and it has been carefully evaluated in experimental setups. Results show a statistical significance classification of emotional states with a mean accuracy of 61.6\%. Finally, this thesis presents a series of systematic evaluations conducted in order to understand the relation between psychophysiological signals and emotions. Facial behavior and physiological signals, i.e. HR, are analyzed and discussed as indicators of emotional states. This research reveals that individualities can be detected regarding facial activity, e.g. an increased number of facial actions during the stressful part of games. Regarding physiological signals, findings are aligned with and reinforce previous research that indicates higher HR mean during stressful situations in a gaming context. Results also suggest that changes in HR during gaming sessions are a promising indicator of stress. The method for the remote detection of emotions, presented in this thesis, is feasible, but does contain limitations. Nevertheless, it is a solid initiative to move away from questionnaires and physical sensors into a non-obtrusive, remote-based solution for the evaluation of user emotions.

  • 3.
    Bevilacqua, Fernando
    et al.
    Federal University of Fronteira Sul, Chapecó, Brazil.
    Backlund, Per
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Engström, Henrik
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Proposal for Non-contact Analysis of Multimodal Inputs to Measure Stress Level in Serious Games2015In: VS-Games 2015: 7th International Conference on Games and Virtual Worlds for Serious Applications / [ed] Per Backlund, Henrik Engström & Fotis Liarokapis, Red Hook, NY: IEEE Computer Society, 2015, p. 171-174Conference paper (Refereed)
    Abstract [en]

    The process of monitoring user emotions in serious games or human-computer interaction is usually obtrusive. The work-flow is typically based on sensors that are physically attached to the user. Sometimes those sensors completely disturb the user experience, such as finger sensors that prevent the use of keyboard/mouse. This short paper presents techniques used to remotely measure different signals produced by a person, e.g. heart rate, through the use of a camera and computer vision techniques. The analysis of a combination of such signals (multimodal input) can be used in a variety of applications such as emotion assessment and measurement of cognitive stress. We present a research proposal for measurement of player’s stress level based on a non-contact analysis of multimodal user inputs. Our main contribution is a survey of commonly used methods to remotely measure user input signals related to stress assessment.

  • 4.
    Bevilacqua, Fernando
    et al.
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Backlund, Per
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Engström, Henrik
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Variations of Facial Actions While Playing Games with Inducing Boredom and Stress2016In: 2016 8th International Conference on Games and Virtual Worlds for Serious Applications (VS-Games), IEEE, 2016Conference paper (Refereed)
    Abstract [en]

    This paper presents an experiment aimed at empirically exploring the variations of facial actions (FA) during gaming sessions with induced boredom and stress. Twenty adults with different ages and gaming experiences played three games while being recorded by a video camera and monitored by a heart rate sensor. The games were carefully designed to have a linear progression from a boring to a stressful state. Self-reported answers indicate participants perceived the games as being boring at the beginning and stressful at the end. The 6 hours of recordings of all subjects were manually analyzed and FA were annotated. We annotated FA that appeared in the recordings at least twice; annotations were categorized by the period when they happened (boring/stressful part of the games) and analysed on a group and on an individual level. Group level analysis revealed that FA patterns were related to no more than 25% of the subjects. The individual level analysis revealed particular patterns for 50% of the subjects. More FA annotations were made during the stressful part of the games. We conclude that, for the context of our experiment, FA provide an unclear foundation for detection of boredom/stressful states when observed from a group level perspective, while the individual level perspective might produce more information.

  • 5.
    Bevilacqua, Fernando
    et al.
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. Federal University of Fronteira Sul, Chapecó, Brazil.
    Engström, Henrik
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Backlund, Per
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Accuracy Evaluation of Remote Photoplethysmography Estimations of Heart Rate in Gaming Sessions with Natural Behavior2018In: Advances in Computer Entertainment Technology: 14th International Conference, ACE 2017, London, UK, December 14-16, 2017, Proceedings / [ed] Adrian David Cheok, Masahiko Inami, Teresa Romão, Springer, 2018, 1, p. 508-530Conference paper (Refereed)
    Abstract [en]

    Remote photoplethysmography (rPPG) can be used to remotely estimate heart rate (HR) of users to infer their emotional state. However natural body movement and facial actions of users significantly impact such techniques, so their reliability within contexts involving natural behavior must be checked. We present an experiment focused on the accuracy evaluation of an established rPPG technique in a gaming context. The technique was applied to estimate the HR of subjects behaving naturally in gaming sessions whose games were carefully designed to be casual-themed, similar to off-the-shelf games and have a difficulty level that linearly progresses from a boring to a stressful state. Estimations presented mean error of 2.99 bpm and Pearson correlationr = 0.43, p < 0.001, however with significant variations among subjects. Our experiment is the first to measure the accuracy of an rPPG techniqueusing boredom/stress-inducing casual games with subjects behaving naturally.

  • 6.
    Bevilacqua, Fernando
    et al.
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. Federal University of Fronteira Sul, Chapecó, Brazil.
    Engström, Henrik
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Backlund, Per
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Automated analysis of facial cues from videos as a potential method for differentiating stress and boredom of players in games2018In: International Journal of Computer Games Technology, ISSN 1687-7047, E-ISSN 1687-7055, article id 8734540Article in journal (Refereed)
    Abstract [en]

    Facial analysis is a promising approach to detect emotions of players unobtrusively, however approaches are commonly evaluated in contexts not related to games, or facial cues are derived from models not designed for analysis of emotions during interactions with games. We present a method for automated analysis of facial cues from videos as a potential tool for detecting stress and boredom of players behaving naturally while playing games. Computer vision is used to automatically and unobtrusively extract 7 facial features aimed to detect the activity of a set of facial muscles. Features are mainly based on the Euclidean distance of facial landmarks and do not rely on pre-dened facial expressions, training of a model or the use of facial standards. An empirical evaluation was conducted on video recordings of an experiment involving games as emotion elicitation sources. Results show statistically signicant dierences in the values of facial features during boring and stressful periods of gameplay for 5 of the 7 features. We believe our approach is more user-tailored, convenient and better suited for contexts involving games.

  • 7.
    Bevilacqua, Fernando
    et al.
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. Federal University of Fronteira Sul, Chapecó, Brazil.
    Engström, Henrik
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Backlund, Per
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Changes in heart rate and facial actions during a gaming session with provoked boredom and stress2018In: Entertainment Computing, ISSN 1875-9521, E-ISSN 1875-953X, Vol. 24, p. 10-20Article in journal (Refereed)
    Abstract [en]

    This paper presents an experiment aimed at exploring the relation between facial actions (FA), heart rate (HR) and emotional states, particularly stress and boredom, during the interaction with games. Subjects played three custom-made games with a linear and constant progression from a boring to a stressful state, without pre-defined levels, modes or stopping conditions. Such configuration gives our experiment a novel approach for the exploration of FA and HR regarding their connection to emotional states, since we can categorize information according to the induced (and theoretically known) emotional states on a user level. The HR data was divided into segments, whose HR mean was calculated and compared in periods (boring/stressful part of the games). Additionally the 6 h of recordings were manually analyzed and FA were annotated and categorized in the same periods. Findings show that variations of HR and FA on a group and on an individual level are different when comparing boring and stressful parts of the gaming sessions. This paper contributes information regarding variations of HR and FA in the context of games, which can potentially be used as input candidates to create user-tailored models for emotion detection with game-based emotion elicitation sources.

    The full text will be freely available from 2020-02-01 00:01
  • 8.
    Bevilacqua, Fernando
    et al.
    Computer Science, Federal University of Fronteira Sul, Chapecó 89802 112, Brazil.
    Engström, Henrik
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Backlund, Per
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre.
    Game-Calibrated and User-Tailored Remote Detection of Stress and Boredom in Games2019In: Sensors, ISSN 1424-8220, E-ISSN 1424-8220, Vol. 19, no 13, p. 1-43, article id 2877Article in journal (Refereed)
    Abstract [en]

    Emotion detection based on computer vision and remote extraction of user signals commonly rely on stimuli where users have a passive role with limited possibilities for interaction or emotional involvement, e.g., images and videos. Predictive models are also trained on a group level, which potentially excludes or dilutes key individualities of users. We present a non-obtrusive, multifactorial, user-tailored emotion detection method based on remotely estimated psychophysiological signals. A neural network learns the emotional profile of a user during the interaction with calibration games, a novel game-based emotion elicitation material designed to induce emotions while accounting for particularities of individuals. We evaluate our method in two experiments (n = 20 and n = 62) with mean classification accuracy of 61.6%, which is statistically significantly better than chance-level classification. Our approach and its evaluation present unique circumstances: our model is trained on one dataset (calibration games) and tested on another (evaluation game), while preserving the natural behavior of subjects and using remote acquisition of signals. Results of this study suggest our method is feasible and an initiative to move away from questionnaires and physical sensors into a non-obtrusive, remote-based solution for detecting emotions in a context involving more naturalistic user behavior and games.

  • 9.
    Eduardo Venson, José
    et al.
    Institute of Informatics – INF, Universidade Federal do Rio Grande do Sul – UFRGS, Av. Bento Gonçalves, Porto Alegre, RS, Brazil / Animati - Computação aplicada à Saúde, Santa Maria, RS, Brazil.
    Bevilacqua, Fernando
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. Universidade Federal da Fronteira Sul – UFFS, Chapecó, SC, Brazil.
    Onuki, Fabio
    Medvia Diagnósticos, Porto Alegre, RS, Brazil.
    Cordeiro d’Ornellas, Marcos
    Laboratory for Applied Computing – LaCA, Universidade Federal de Santa Maria – UFSM, Santa Maria, RS, Brazil.
    Anderson, Maciel
    Institute of Informatics – INF, Universidade Federal do Rio Grande do Sul – UFRGS, Av. Bento Gonçalves, Porto Alegre, RS, Brazil.
    Efficient medical image access in diagnostic environments with limited resources2016In: Research on Biomedical Engineering, ISSN 2446-4732, Vol. 32, no 4, p. 347-357Article in journal (Refereed)
    Abstract [en]

    Introduction

    A medical application running outside the workstation environment has to deal with several constraints, such as reduced available memory and low network bandwidth. The aim of this paper is to present an approach to optimize the data flow for fast image transfer and visualization on mobile devices and remote stationary devices.

    Methods

    We use a combination of client- and server-side procedures to reduce the amount of information transferred by the application. Our approach was implemented on top of a commercial PACS and evaluated through user experiments with specialists in typical diagnosis tasks. The quality of the system outcome was measured in relation to the accumulated amount of network data transference and the amount of memory used in the host device. Besides, the system's quality of use (usability) was measured through participants’ feedback.

    Results

    Contrarily to previous approaches, ours keeps the application within the memory constraints, minimizing data transferring whenever possible, allowing the application to run on a variety of devices. Moreover, it does that without sacrificing the user experience. Experimental data point that over 90% of the users did not notice any delays or degraded image quality, and when they did, they did not impact on the clinical decisions.

    Conclusion

    The combined activities and orchestration of our methods allow the image viewer to run on resource-constrained environments, such as those with low network bandwidth or little available memory. These results demonstrate the ability to explore the use of mobile devices as a support tool in the medical workflow.

  • 10.
    Venson, José Eduardo
    et al.
    Instituto de Informática, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil.
    Bevilacqua, Fernando
    University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. Universidade Federal da Fronteira Sul, Chapecó, Brazil.
    Berni, Jean
    Animati Computação Aplicada à Saúde, Santa Maria, Brazil.
    Onuki, Fabio
    Medvia Diagnóstico, Porto Alegre, Brazil.
    Maciel, Anderson
    Instituto de Informática, Universidade Federal do Rio Grande do Sul (UFRGS), Porto Alegre, Brazil.
    Diagnostic concordance between mobile interfaces and conventional workstations for emergency imaging assessment2018In: International Journal of Medical Informatics, ISSN 1386-5056, E-ISSN 1872-8243, Vol. 113, p. 1-8Article in journal (Refereed)
    Abstract [en]

    Introduction

    Mobile devices and software are now available with sufficient computing power, speed and complexity to allow for real-time interpretation of radiology exams. In this paper, we perform a multivariable user study that investigates concordance of image-based diagnoses provided using mobile devices on the one hand and conventional workstations on the other hand.

    Methods

    We performed a between-subjects task-analysis using CT, MRI and radiography datasets. Moreover, we investigated the adequacy of the screen size, image quality, usability and the availability of the tools necessary for the analysis. Radiologists, members of several teams, participated in the experiment under real work conditions. A total of 64 studies with 93 main diagnoses were analyzed.

    Results

    Our results showed that 56 cases were classified with complete concordance (87.69%), 5 cases with almost complete concordance (7.69%) and 1 case (1.56%) with partial concordance. Only 2 studies presented discordance between the reports (3.07%). The main reason to explain the cause of those disagreements was the lack of multiplanar reconstruction tool in the mobile viewer. Screen size and image quality had no direct impact on the mobile diagnosis process.

    Conclusion

    We concluded that for images from emergency modalities, a mobile interface provides accurate interpretation and swift response, which could benefit patients' healthcare.

1 - 10 of 10
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf