his.sePublikationer
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Challenges in face expression recognition from video
Högskolan i Skövde, Institutionen för informationsteknologi. Högskolan i Skövde, Forskningscentrum för Informationsteknologi. (Interaction Lab)
Högskolan i Skövde, Institutionen för informationsteknologi. Högskolan i Skövde, Forskningscentrum för Informationsteknologi. (Interaction Lab)ORCID-id: 0000-0002-6568-9342
2017 (Engelska)Ingår i: SweDS 2017: The 5th Swedish Workshop on Data Science / [ed] Alexander Schliep, 2017Konferensbidrag, Muntlig presentation med publicerat abstract (Refereegranskat)
Abstract [en]

Identication of emotion from face expressions is a relatively well understood problem where state-of-the-art solutions perform almost as well as humans. However, in many practical applications, disruptingfactors still make identication of face expression a very challenging problem. Within the project DREAM1- Development of Robot Enhanced Therapy for Children with Autism Spectrum Disorder (ASD), we areidentifying face expressions from children with ASD, during therapy. Identied face expressions are usedboth in the online system, to guide the behavior of the robot, and o-line, to automatically annotate videofor measurements of clinical outcomes.

This setup puts several new challenges on the face expression technology. First of all, in contrast tomost open databases of face expressions comprising adult faces, we are recognizing emotions from childrenbetween the age of 4 to 7 years. Secondly, children with ASD may show emotions dierently, compared totypically developed children. Thirdly, the children move freely during the intervention and, despite the useof several cameras tracking the face of the child from dierent angles, we rarely have a full frontal view ofthe face. Fourthly, and nally, the amount of native data is very limited.

Although we have access to extensive video recorded material from therapy sessions with ASD children,potentially constituting a very valuable dataset for both training and testing of face expression implemen-tations, this data proved to be dicult to use. A session of 10 minutes of video may comprise only a fewinstances of expressions e.g. smiling. As such, although we have many hours of video in total, the data isvery sparse and the number of clear face expressions is still rather small for it to be used as training data inmost machine learning (ML) techniques.

We therefore focused on the use of synthetic datasets for transfer learning, trying to overcome thechallenges mentioned above. Three techniques were evaluated: (1) convolutional neural networks for imageclassication by analyzing separate video frames, (2) recurrent neural networks for sequence classication tocapture facial dynamics, and (3) ML algorithms classifying pre-extracted facial landmarks.

The performance of all three models are unsatisfactory. Although the proposed models were of highaccuracy, approximately 98%, while classifying a test set, they performed poorly on the real-world data.This was due to the usage of a synthetic dataset which had mostly a frontal view of faces. The models whichhave not seen similar examples before failed to classify them correctly. The accuracy decreased drasticallywhen the child rotated her head or covered a part of her face. Even if the frame clearly captured a facialexpression, ML algorithms were not able to provide a stable positive classication rate. Thus, elaborationon training datasets and designing robust ML models are required. Another option is to incorporate voiceand gestures of the child into the model to classify emotional state as a complex concept.

Ort, förlag, år, upplaga, sidor
2017.
Nationell ämneskategori
Datorteknik
Forskningsämne
Interaction Lab (ILAB)
Identifikatorer
URN: urn:nbn:se:his:diva-14581Scopus ID: 2-s2.0-85030706059OAI: oai:DiVA.org:his-14581DiVA, id: diva2:1166244
Konferens
SweDS 2017: The 5th Swedish Workshop on Data Science. Data Science Division Department of Computer Science and Engineering University of Gothenburg Chalmers, Gothenburg, Sweden, December 12–13, 2017
Projekt
DREAM, FP7 funded project, #611391Tillgänglig från: 2017-12-14 Skapad: 2017-12-14 Senast uppdaterad: 2019-09-09Bibliografiskt granskad

Open Access i DiVA

fulltext(258 kB)59 nedladdningar
Filinformation
Filnamn FULLTEXT01.pdfFilstorlek 258 kBChecksumma SHA-512
3c0077274be2283de6c0f9ca3c6d1ae0e363966bb86fecdae6ee3c2db25a7647998198ec2c46831ddf464bffcfd646388af2438ea6a98d5965ce60589ac9a4d0
Typ fulltextMimetyp application/pdf

Övriga länkar

ScopusAbstracts

Personposter BETA

Redyuk, SergeyBilling, Erik A.

Sök vidare i DiVA

Av författaren/redaktören
Redyuk, SergeyBilling, Erik A.
Av organisationen
Institutionen för informationsteknologiForskningscentrum för Informationsteknologi
Datorteknik

Sök vidare utanför DiVA

GoogleGoogle Scholar
Totalt: 59 nedladdningar
Antalet nedladdningar är summan av nedladdningar för alla fulltexter. Det kan inkludera t.ex tidigare versioner som nu inte längre är tillgängliga.

urn-nbn

Altmetricpoäng

urn-nbn
Totalt: 808 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf