Corpus-Based Translation Induction in Indian Languages Using Auxiliary Language Corpora from Wikipedia

Tholpadi, Goutham and Bhattacharyya, Chiranjib and Shevade, Shirish (2017) Corpus-Based Translation Induction in Indian Languages Using Auxiliary Language Corpora from Wikipedia. In: ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING, 16 (3).

PDF Acm_Tra_Asi_Low_Res_Lan_Inf_Pro_16-3_20_2017.pdf - Published Version Restricted to Registered users only Download (3MB) | Request a copy

Abstract

Identifying translations from comparable corpora is a well-known problem with several applications. Existing methods rely on linguistic tools or high-quality corpora. Absence of such resources, especially in Indian languages, makes this problem hard; for example, state-of-the-art techniques achieve a mean reciprocal rank of 0.66 for English-Italian, and a mere 0.187 for Telugu-Kannada. In this work, we address the problem of comparable corpora-based translation correspondence induction (CC-TCI) when the only resources available are small noisy comparable corpora extracted from Wikipedia. We observe that translations in the source and target languages have many topically related words in common in other ``auxiliary'' languages. To model this, we define the notion of a translingual theme, a set of topically related words from auxiliary language corpora, and present a probabilistic framework for CC-TCI. Extensive experiments on 35 comparable corpora showed dramatic improvements in performance. We extend these ideas to propose a method for measuring cross-lingual semantic relatedness (CLSR) between words. To stimulate further research in this area, we make publicly available two new high-quality human-annotated datasets for CLSR. Experiments on the CLSR datasets show more than 200% improvement in correlation on the CLSR task. We apply the method to the real-world problem of cross-lingual Wikipedia title suggestion and build the WikiTSu system. A user study on WikiTSu shows a 20% improvement in the quality of titles suggested.

Item Type:	Journal Article
Publication:	ACM TRANSACTIONS ON ASIAN AND LOW-RESOURCE LANGUAGE INFORMATION PROCESSING
Publisher:	ASSOC COMPUTING MACHINERY, 2 PENN PLAZA, STE 701, NEW YORK, NY 10121-0701 USA
Additional Information:	Copy right for this article belongs to the ASSOC COMPUTING MACHINERY, 2 PENN PLAZA, STE 701, NEW YORK, NY 10121-0701 USA
Department/Centre:	Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	20 May 2017 05:27
Last Modified:	20 May 2017 05:27
URI:	http://eprints.iisc.ac.in/id/eprint/56899

Actions (login required)

View Item