Prediction of Culture Based on Automated Detection of Multimodal Social Signals

Users require conversational virtual characters to be socially and culturally aware in order to develop trust and relationship(s). This is dependent on the goal and setting of the interaction. In previous research It has been suggested that cultures that require high context (increased use of nonverbal signals) and low context (more direct verbal communication) during communication differ in their use of nonverbal displays. The aim of this paper is to investigate the following: 1) whether group membership of high or low context cultures can be predicted based on detection of nonverbal signals in the impression formation phase of a dialogue; and 2) whether multimodal approaches would lead to better and improved accuracy compared to unimodal approaches. As part of this study and to collect data, nonverbal signals were captured during two media skills training workshops. Analysis of 35 media interviews revealed that a multimodal approach produces a higher F measure at predicting high and low context cultures (0.772 to 0.848 vs 0.733 to 0.771 for unimodal channels). These findings contribute to the development of conversational virtual characters that demonstrate more culturally aware communication styles in the initial stages of an interaction with their users.