Analyze Face Emotions with R

Human faces provide various information about emotionsMicrosoft launched a free service in December 2015 to analyze human faces detecting their emotions. The emotions detected are anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise. These emotions are understood to be cross-culturally and universally communicated with particular facial expressions.

The Emotion API takes an facial expression in an image as an input, and returns the confidence across a set of emotions for each face in the image, as well as bounding box for the face, using the Face API.

The implementation in R allows to analyze human faces in a structured way. Note, that one have to create an account to use the Face API.

The example is referred to a simple example:





You need four R packages: httrXMLstringrggplot2.

Analyze Face Emotions with R:


Daniel Boller

To keep it simple: R makes the world intelligible. Data - as part and results of human interaction - can not be used without intelligent systems and inferential/statistical tools - thus, R provides an optimal framework to get use of the data - or, to understand human interaction. I worked with R for several years and like it. I studied Economics with a strong focus on empirical research methods and behavioral economics. Currently, I work as Research Associate (Doctoral Candidate) at the University of St. Gallen.

  • Lena

    thank you for this tutorial! It was great and easy to follow.
    I have two questions.
    If I’m not mistaken line 56 of your code should refer to the “face api” instead of the “emotion api”. So one has actually subscribe to two different apps.
    And secondly, how do you interpret the results? the picture of Obama shows a smile, right? Does it indicate happiness? Hence, I was quite surprised by the generated bar chart (if higher bars suggests higher amounts of that emotional expression)..and I think there might be an issue with your previous transformation (line 38) given the scientific notation (e-). If you use the non-scientific display of numbers (options(scipen=999)) the bar chart looks quite different and suggests happiness (as expected) instead of higher amounts of fear, disgust and surprise. What do you think? maybe I’m mistaken, but it’d be interesting to hear how you interpret the results (the accuracy of the outcome?) and also how the results of the face api could be interpreted?

  • Karan Singh Aulakh

    A great tutorial. However, I am unable to get along with the initial steps on how to use the Face API. Please help.

  • Abhishek Palit

    How can I access the local image file & study the Face Emotion Score?

  • Samson agboola

    I think there are no object (variable) called scores with Obama object. the names associated with Obama is “code” and “message” no scores