Quantifying Emotional States Based on Body Language Data Using Non Invasive Sensors

Determining participant engagement is an important issue across a large number of fields, ranging from entertainment to education. Traditionally, feedback from participants is taken after the activity has been completed. Alternately, continuous observation by trained humans is needed. Thus, there is a need for an automated real time solution. In this paper, the authors propose a data mining driven approach that models a participant’s engagement, based on body language data acquired in real time using non-invasive sensors. Skeletal position data, that approximates human body motions, is acquired from participants using off the shelf, non-invasive sensors. Thereafter, machine learning techniques are employed to detect body language patterns representing emotions such as delight, interest, boredom, frustration, and confusion. The methodology proposed in this paper enables researchers to predict the participants’ engagement levels in real time with high accuracy above 98%. A case study involving human participants enacting eight body language poses, is used to illustrate the effectiveness of the methodology. Finally, this methodology highlights the potential of a real time, automated engagement detection using non-invasive sensors which can ultimately have applications in a large variety of areas such as lectures, gaming and classroom learning.