P-SIF: Document embeddings using partition averaging

Gupta, V and Saw, A and Nokhiz, P and Netrapalli, P and Rai, P and Talukdar, P (2020) P-SIF: Document embeddings using partition averaging. In: AAAI 2020 - 34th AAAI Conference on Artificial Intelligence, 7- 12 February 2020, New York, pp. 7863-7870.

PDF AAAI_2020.pdf - Published Version Restricted to Registered users only Download (969kB) | Request a copy

Abstract

Simple weighted averaging of word vectors often yields effective representations for sentences which outperform sophisticated seq2seq neural models in many tasks. While it is desirable to use the same method to represent documents as well, unfortunately, the effectiveness is lost when representing long documents involving multiple sentences. One of the key reasons is that a longer document is likely to contain words from many different topics; hence, creating a single vector while ignoring all the topical structure is unlikely to yield an effective document representation. This problem is less acute in single sentences and other short text fragments where the presence of a single topic is most likely. To alleviate this problem, we present P-SIF, a partitioned word averaging model to represent long documents. P-SIF retains the simplicity of simple weighted word averaging while taking a document’s topical structure into account. In particular, PSIF learns topic-specific vectors from a document and finally concatenates them all to represent the overall document. We provide theoretical justifications on the correctness of P-SIF. Through a comprehensive set of experiments, we demonstrate P-SIF’s effectiveness compared to simple weighted averaging and many other baselines. Copyright © 2020, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Item Type:	Conference Paper
Publication:	AAAI 2020 - 34th AAAI Conference on Artificial Intelligence
Publisher:	AAAI press
Additional Information:	The copyright for this article belongs to AAAI press.
Keywords:	Statistical methods, Document Representation; Most likely; Neural models; Short texts; Single vectors; Topical structure; Weighted averaging; Word vectors, Artificial intelligence
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	07 Feb 2023 06:16
Last Modified:	07 Feb 2023 06:16
URI:	https://eprints.iisc.ac.in/id/eprint/79988

Actions (login required)

View Item