Have you found yourself alone in the dark, working on your computer feeling a tad depressed and wishing your computer could respond in some way to your mental and emotional state? You know, maybe your computer could send you some funny comics to lighten your day, or perhaps send you an inspiring quote to keep you in the fight for life....These ideas might sound a bit far fetched given our current reality with our machines. But there is actually an active community, whose goal is precisely for machines to understand human emotion and social interaction.
Having machines understand emotion and social interactions, is beneficial for:
- Social Scientists, Psychologists and Doctors: As they all are very interested in observing and quantifying human behaviour. For Psychologists and Doctors this could help them in diagnosing and rehabilitating their patients.
- Intelligent Algorithms: that by understanding their user better, could respond to semantic queries and retrieve more relevant material. As humans we are very familiar with interacting with different meanings given different contexts, think for example of that popular 60's song titled: "it's the same old song, but with a different meaning since you've been gone
- Ambient intelligence: Environments can become more responsive to the social context. Perhaps the room detects that the crowd at a small reunion is bored, and so the room could maybe start playing whimsical animal figures on the walls to entertain the audience.
Overall having machines being able to interpret human emotion can improve Human Computer Interaction, as it increases the computer's sensitivity to the user's mental and emotional state [5].
Now, given that we understand the benefits of machines that can understand our emotions betters. The question is, so... how can this be enabled?
Recent investigations focus on something called SOCIAL SIGNALS.
But what is exactly a social signal?
-A social signal (According to Poggi and D'Errico) is a communicative or informative signal that conveys information about social actions, social interactions, social emotions and attitudes.
Where the heck does this idea of "Social Signal" come from?
The term "Social Signal" was inspired by various psychology studies, that analysed how non-verbal behaviour relates to social interactions. Psychologists were studying things such as:
- How does non-verbal behaviour effect the formation of impressions? For example, apparently when you smile a lot and have rapid movements, people take this as if you are an extrovert.
- How does non-verbal behaviour reinforce the nature of a relationship? Apparently men tend to lean forward more and gaze toward the person they are talking with, when the other person so happens to be a female, this behaviour is experienced even more if they are having an intimate conversation, rather than the general boring interpersonal water cooler chit chat.
- Can facial movements be mapped into emotional signals and conversation signals? This is an area greatly studied in deception detection, as there are certain muscles that are expected to be moved when someone is angry, happy, sad etc. Therefore a person might unwillingly move those muscles, and show their true feelings. Or not move them, and therefore give clues as to the fact that they are being deceitful ( See [4]).
Now the big question is: O.K., So how do social signals help machines understand human emotion?
In 2007, a professor and researcher from MIT's Media Lab,Alex Pentland, introduced the notion of “social signal processing”. Which is about applying traditional signal processing techniques to social signals, and use this processing and analysis to predict human social behaviour. For example, his group created a machine that was able to autonomously predict the outcome of a negotiation or of a speed date within its very first minute (see [1] and [2]).
The main goal of social signal processing, is to enable analysis of Human behaviour by computers. For this advanced pattern recognition techniques are utilized to automatically interpret complex human behavioural patterns. In the next days, we will talk more about these techniques that are used to interpret human behaviour. Stay tuned! n_n
References:
1)I. Poggi and F. DÉrrico, "Social signals: A psychological perspective." . Springer Verlag’s Advances in Pattern Recognition series, 2011, pp. 185-225.
2)Pentland, A.: Social signal processing. IEEE Signal Process. Mag. 24(4), 108–111 (2007)
3)Curhan, J., Pentland, A.: Thin slices of negotiation: predicting outcomes from conversational
dynamics within the first five minutes. J. Appl. Psychol. 92, 802–811 (2007)
4)Ekman, P., Friesen, W.V.: Nonverbal leakage and clues to deception. Psychiatry 32, 88–106
(1969)
5) A. A. Salah, M. Pantic, and A. Vinciarelli, "Recent Developments in Social Signal Processing," in Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, 2011, pp. 380-385.
