Unsupervised feature learning of human actions as trajectories in pose embedding manifold

Kundu, JN and Gor, M and Uppala, PK and Babu, RV (2019) Unsupervised feature learning of human actions as trajectories in pose embedding manifold. In: UNSPECIFIED, pp. 1459-1467.

PDF Pro-Iee-Win_Con_App_Com_Vis_1459_2019.pdf - Published Version Restricted to Registered users only Download (706kB) | Request a copy

Abstract

An unsupervised human action modeling framework can provide useful pose-sequence representation, which can be utilized in a variety of pose analysis applications. In this work we propose a novel temporal pose-sequence modeling framework, which can embed the dynamics of 3D human-skeleton joints to a continuous latent space in an efficient manner. In contrast to end-to-end framework explored by previous works, we disentangle the task of individual pose representation learning from the task of learning actions as a trajectory in pose embedding space. In order to realize a continuous pose embedding manifold with improved reconstructions, we propose an unsupervised, manifold learning procedure named Encoder GAN, (or EnGAN). Further we use the pose embeddings generated by EnGAN to model human actions using a bidirectional RNN auto-encoder architecture, PoseRNN. We introduce first-order gradient loss to explicitly enforce temporal regularity in the predicted motion sequence. A hierarchical feature fusion technique is also investigated for simultaneous modeling of local skeleton joints along with global pose variations. We demonstrate state-of-the-art transfer-ability of the learned representation against other supervisedly and unsupervisedly learned motion embeddings for the task of fine-grained action recognition on SBU interaction dataset. Further, we show the qualitative strengths of the proposed framework by visualizing skeleton pose reconstructions and interpolations in pose-embedding space, and low dimensional principal component projections of the reconstructed pose trajectories. © 2019 IEEE.

Item Type:	Conference Paper
Publication:	Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision, WACV 2019
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	Copyright for this article belongs to Institute of Electrical and Electronics Engineers Inc.
Keywords:	3D modeling; Computer vision; Machine learning; Musculoskeletal system; Repair; Signal encoding; Trajectories, Action recognition; Hierarchical features; Learning actions; Manifold learning; Principal Components; Sequence modeling; Simultaneous model; Unsupervised feature learning, Embeddings
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	29 May 2019 11:57
Last Modified:	29 May 2019 11:57
URI:	http://eprints.iisc.ac.in/id/eprint/62220

Actions (login required)

View Item