ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

Improving answer selection and answer triggering using hard negatives

Kumar, S and Mehta, K and Garg, S and Rasiwasia, N (2020) Improving answer selection and answer triggering using hard negatives. In: 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, 3-7 November 2019, Hong Kong; China, pp. 5911-5917.

[img] PDF
CON_EMP_MET_NAT_LAN_PRO_PRO_5911-5917_2020.pdf - Published Version
Restricted to Registered users only

Download (314kB) | Request a copy
Official URL: http://dx.doi.org/10.18653/v1/D19-1604

Abstract

In this paper, we establish the effectiveness of using hard negatives, coupled with a siamese network and a suitable loss function, for the tasks of answer selection and answer triggering. We show that the choice of sampling strategy is key for achieving improved performance on these tasks. Evaluating on recent answer selection datasets - InsuranceQA, SelQA, and an internal QA dataset, we show that using hard negatives with relatively simple model architectures (bag of words and LSTM-CNN) drives significant performance gains. On InsuranceQA, this strategy alone improves over previously reported results by a minimum of 1.6 points in P@1. Using hard negatives with a Transformer encoder provides a further improvement of 2.3 points. Further, we propose to use quadruplet loss for answer triggering, with the aim of producing globally meaningful similarity scores. We show that quadruplet loss function coupled with the selection of hard negatives enables bag-of-words models to improve F1 score by 2.3 points over previous baselines, on SelQA answer triggering dataset. Our results provide key insights into answer selection and answer triggering tasks. © 2019 Association for Computational Linguistics

Item Type: Conference Paper
Publication: EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference
Publisher: Association for Computational Linguistics
Additional Information: The copyright of the article belongs to Association for Computational Linguistics.
Department/Centre: Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited: 02 Nov 2020 06:17
Last Modified: 08 Dec 2022 10:19
URI: https://eprints.iisc.ac.in/id/eprint/65523

Actions (login required)

View Item View Item