cs.CV cs.LG
Spatio Temporal Analysis of Facial Actions using Lifecycle Aware Capsule Networks. Most state of the art approaches for Facial Action Unit (AU) detection rely upon evaluating facial expressions from static frames, encoding a snapshot of heightened facial activity. In real world interactions, however, facial expressions are usually more subtle and evolve in a temporal manner requiring AU detection models to learn spatial as well as temporal information. In this paper, we focus on both spatial and spatio temporal features encoding the temporal evolution of facial AU activation. For this purpose, we propose the Action Unit Lifecycle Aware Capsule Network (AULA Caps) that performs AU detection using both frame and sequence level features. While at the frame level the capsule layers of AULA Caps learn spatial feature primitives to determine AU activations, at the sequence level, it learns temporal dependencies between contiguous frames by focusing on relevant spatio temporal segments in the sequence. The learnt feature capsules are routed together such that the model learns to selectively focus more on spatial or spatio temporal information depending upon the AU lifecycle. The proposed model is evaluated on the commonly used BP4D and GFT benchmark datasets obtaining state of the art results on both the datasets.