Fast and Accurate Learning of Knowledge Graph Embeddings at Scale

Gupta, U and Vadhiyar, S (2019) Fast and Accurate Learning of Knowledge Graph Embeddings at Scale. In: 26th Annual IEEE International Conference on High Performance Computing, HiPC 2019, 17-20. December 2019, Hyderabad, pp. 173-182.

PDF iee_int_con_hig_per_com_dat_ana_173-182_2019.pdf - Published Version Restricted to Registered users only Download (774kB) | Request a copy

Abstract

Knowledge Graph Embedding (KGE) is used to represent the entities and relations of a KG in a low dimensional vector space. KGE can then be used in a downstream task such as entity classification, link prediction and knowledge base completion. Training on large KG datasets takes a considerable amount of time. This work proposes three strategies which lead to faster training in distributed setting. The first strategy is a reduced communication approach which decreases the All-Gather size by sparsifying the Sparse Gradient Matrix (SGM). The second strategy is a variable margin approach that takes advantage of reduced communication for lower margins but retains the accuracy as obtained by the best fixed margin. The third strategy is called DistAdam which is a distributed version of the popular Adam optimization algorithm. Combining the three strategies results in reduction of training time for the FB250K dataset from twenty-seven hours on one processing node to under one hour on thirty-two nodes with each node consisting of twenty-four cores. © 2019 IEEE.

Item Type:	Conference Paper
Publication:	Proceedings - 26th IEEE International Conference on High Performance Computing, HiPC 2019
Publisher:	Institute of Electrical and Electronics Engineers Inc.
Additional Information:	cited By 0; Conference of 26th Annual IEEE International Conference on High Performance Computing, HiPC 2019 ; Conference Date: 17 December 2019 Through 20 December 2019; Conference Code:157722
Keywords:	Knowledge based systems; Large dataset; Vector spaces, Distributed learning; Horovod; Knowledge base; Knowledge graphs; Link prediction; Low dimensional; Optimization algorithms; Processing nodes, Embeddings
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	14 Aug 2020 10:22
Last Modified:	14 Aug 2020 10:22
URI:	http://eprints.iisc.ac.in/id/eprint/64832

Actions (login required)

View Item