Unsupervised feature learning with discriminative encoder

Pandey, Gaurav and Dukkipati, Ambedkar (2017) Unsupervised feature learning with discriminative encoder. In: 17th IEEE International Conference on Data Mining (ICDMW), NOV 18-21, 2017, New Orleans, LA, pp. 367-376.

PDF ICDM_367_2017.pdf - Published Version Restricted to Registered users only Download (356kB) | Request a copy

Abstract

In recent years, deep discriminative models have achieved extraordinary performance on supervised learning tasks, significantly outperforming their generative counterparts. However, their success relies on the presence of a large amount of labeled data. How can one use the same discriminative models for learning useful features in the absence of labels? We address this question in this paper, by jointly modeling the distribution of data and latent features in a manner that explicitly assigns zero probability to unobserved data. Rather than maximizing the marginal probability of observed data, we maximize the joint probability of the data and the latent features using a two step EM-like procedure. To prevent the model from overfitting to our initial selection of latent features, we use adversarial regularization. Depending on the task, we allow the latent features to be one-hot or real-valued vectors, and define a suitable prior on the features. For instance, one-hot features correspond to class labels, and are directly used for unsupervised and semi-supervised classification task, whereas real-valued feature vectors are fed as input to simple classifiers for auxiliary supervised discrimination tasks. The proposed model, which we dub dicriminative encoder (or DisCoder), is flexible in the type of latent features that it can capture. The proposed model achieves state-of-the-art performance on several challenging tasks. Qualitative visualization of the latent features shows that the features learnt by the DisCoder are indeed meaningful.

Item Type:	Conference Proceedings
Series.:	IEEE International Conference on Data Mining
Publisher:	IEEE, 345 E 47TH ST, NEW YORK, NY 10017 USA
Additional Information:	Copy right for the article belong toIEEE, 345 E 47TH ST, NEW YORK, NY 10017 USA
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	04 Apr 2018 18:51
Last Modified:	04 Apr 2018 18:51
URI:	http://eprints.iisc.ac.in/id/eprint/59474

Actions (login required)

View Item