Real-Time Backchannel Selection for ECAs According to User’s Level of Interest Etienne de Sevin and Catherine Pelachaud CNRS - Telecom ParisTech 37/39, rue Dareau 75014 Paris, France {etienne.de-sevin,catherine.pelachaud}@telecom-paristech.fr

1 Action Selection Algorithm for ECA Backchannels A great challenge that is to be faced in the design of virtual agents is the issue of credibility, not only in the agent's aspect but also in its behavior [1]. To be believable, the agent has to decide what to do next according to the internal and external variables of the agent. Besides others, we have to deal with the problem of action selection which can be resumed to choose the most appropriate action among all possible (conflicting) ones [2]. In our case, actions are backchannels. This work is part of the STREP EU SEMAINE project1 in which a real-time Embodied Conversational Agent (ECA) will be a Sensitive Artificial Listener (SAL) [3]. This project aims to build an autonomous talking agent able to exhibit autonomously appropriate verbal and non verbal behaviors in real-time when it plays the role of the listener in a conversation with a user.

Agent’s Mental State

Triggering of backchannels

Reactive Backchannels

Mimicry Backchannels

Perceived User’s Level of Interest

Backchannel Selection

ECA

User

Fig. 1. Schematic view of the Backchannel architecture including a BC selection module

The proposed work is part of a pre-existing system for the generation of backchannels for an ECA listener [1][4] (see figure 1). Backchannel modules will generate potential conflicting actions and send them with their priorities to the backchannel 1

http://www.semaine-project.eu

Zs. Ruttkay et al. (Eds.): IVA 2009, LNAI 5773, pp. 494–495, 2009. © Springer-Verlag Berlin Heidelberg 2009

Real-Time Backchannel Selection for ECAs According to User’s Level of Interest

495

selection algorithm. For example, the mimicry module could generate a head nod to mimic the user's head movement, whereas the reactive backchannel module could generate a head shake determined by the communicative function “disagree”. There is a conflict between both head signals and as just one signal can be actually displayed, a choice has to be done. The first step of our backchannel selection algorithm is to compute the probability of displaying backchannels (mimicry or reactive backchannels). Then the algorithm receives all potential reactive backchannels and mimicry. It calculates their updated priorities as a function of the user’s interest level. These new priorities are the basis of selection because they allow us to compare different backchannels coming from the action proposer module. The priorities of backchannels can also be modified afterward according the contextual information of the interaction such as the estimated gaze of the user, the user’s level of the disinterest and the phase of the interaction. The selection is event-based and in real-time. If backchannels are triggered, then a choice is made. Finally, the selection algorithm chooses the most appropriate backchannels based on the priority values according to the user’s interest level and the context of the interaction.

2 Conclusion The preliminary evaluation shows that our listener ECA behaves like we expected. The action selection chooses the appropriate action according to the user’s interest level as perceived by the ECA. The user's estimated interest level is well-adapted to adjust the decisions of the ECA to choose which backchannel is the best to display in real-time. It is also a good indicator for inferring information about the user’s intention to interact. As our action selection algorithm is generic, it can also be used in an application to select actions in a gaze-based sharing attention context [5].

Acknowledgments This research was supported by the STREP SEMAINE IST-211486.

References 1. Bevacqua, E., Mancini, M., Pelachaud, C.: A listening agent exhibiting variable behaviour. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 262–269. Springer, Heidelberg (2008) 2. de Sevin, E., Thalmann, D.: A motivational Model of Action Selection for Virtual Humans. In: SocietyPress, I. (ed.) Computer Graphics International (CGI), pp. 213–220 (2005) 3. Douglas-Cowie, E., Cowie, R., Cox, C., Amir, N., Heylen, D.: The Sensitive Artificial Listener: an induction technique for generating emotionally coloured conversation (2008) 4. Peters, C., Pelachaud, C., Bevacqua, E., Poggi, I., Mancini, M., Chafai, N.: A Model of Attention and Interest Using Gaze Behavior. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 229–240. Springer, Heidelberg (2005) 5. Peters, C., Asteriadis, S., Karpouzis, K., Sevin, E.: Towards a Real-time Gaze-based Shared Attention for a Virtual Agent. In: Workshop on Affective Interaction in Natural Environments in the Tenth International Conference on Multimodal Interfaces (2008)