Real-time Multi-map Saliency-driven Gaze Behavior for Non-conversational Characters

Ific Goudé (1) , Alexandre Bruckert (2) , Anne-Hélène Olivier (3, 1) , Julien Pettré (1) , Rémi Cozot (4, 5) , Kadi Bouatouch (1) , Marc Christie (1) , Ludovic Hoyet (1)

1 VIRTUS - Nous, virtuels
2 LS2N - équipe IPI - Image Perception Interaction
3 M2S - Laboratoire Mouvement Sport Santé
4 ULCO - Université du Littoral Côte d'Opale
5 LISIC - Laboratoire d'Informatique Signal et Image de la Côte d'Opale

Gaze behavior of virtual characters in video games and virtual reality experiences is a key factor of realism and immersion. Indeed, gaze plays many roles when interacting with the environment; not only does it indicate what characters are looking at, but it also plays an important role in verbal and non-verbal behaviors and in making virtual characters alive. Automated computing of gaze behaviors is however a challenging problem, and to date none of the existing methods are capable of producing close-to-real results in an interactive context. We therefore propose a novel method that leverages recent advances in several distinct areas related to visual saliency, attention mechanisms, saccadic behavior modelling, and head-gaze animation techniques. Our approach articulates these advances to converge on a multi-map saliency-driven model which offers real-time realistic gaze behaviors for non-conversational characters, together with additional user-control over customizable features to compose a wide variety of results. We first evaluate the benefits of our approach through an objective evaluation that confronts our gaze simulation with ground truth data using an eye-tracking dataset specifically acquired for this purpose. We then rely on subjective evaluation to measure the level of realism of gaze animations generated by our method, in comparison with gaze animations captured from real actors. Our results show that our method generates gaze behaviors that cannot be distinguished from captured gaze animations. Overall, we believe that these results will open the way for more natural and intuitive design of realistic and coherent gaze animations for real-time applications.

Mots clés: Gaze behavior; simulation; animation; neural networks; eye-tracking data; dataset

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]; Multimédia [cs.MM]; Modélisation et simulation

Ific Goudé, Alexandre Bruckert, Anne-Hélène Olivier, Julien Pettré, Rémi Cozot, et al.. Real-time Multi-map Saliency-driven Gaze Behavior for Non-conversational Characters. 2022. ⟨hal-03796523v1⟩