Impact of message compression on the scalability of an atmospheric modeling application on clusters

Kumar, Santhosh V and Nanjundiah, R and Thazhuthaveetil, MJ and Govindarajan, R (2008) Impact of message compression on the scalability of an atmospheric modeling application on clusters. In: Parallel Computing, 34 (1). pp. 1-16.

PDF computing.pdf - Published Version Restricted to Registered users only Download (216kB) | Request a copy

Abstract

In this paper, we study the scalability of an atmospheric modeling application on a cluster with commercially available off-the-shelf interconnects. It is found that interconnects with large latency and low bandwidth are major bottlenecks for performance scalability. Response curves for latency shows that for large message sizes latency is extremely sensitive to the size of the message. Thus, decreasing the message size could reduce the latency and hence improve the scalability.We propose both lossless and lossy (i.e., with loss of some information) compression schemes to reduce message sizes.These compression techniques are investigated for the Community Atmospheric Model (CAM), which is a large scale parallel application used for global climate simulation, on a IBM Power 5 Cluster with Gigabit interconnect. This is a floating point intensive application which involves both point-to-point and collective all-to-all communication of large messages (>128 KB). Floating point data which constitute the messages in CAM application results in 14.8% compression when lossless compression is employed and the speedup improves by about 18% on 32 processors. We further evaluate three lossy compression schemes with very low overheads (0.15%). We study the acceptability criteria for information loss in the lossy compression schemes using a perturbation growth test procedure. The lossy compression schemes achieve a message size reduction of 66.2% and an execution time speedup of up to 20.78 on 32 processors. We also look at the criteria for acceptability of loss of information in lossy compression techniques.

Item Type:	Journal Article
Publication:	Parallel Computing
Publisher:	Elsevier
Additional Information:	Copyright of this articles belongs to Elsevier.
Keywords:	Cluster computing;Communication layer;Message compression;Parallel application;Atmospheric modeling.
Department/Centre:	Division of Interdisciplinary Sciences > Supercomputer Education & Research Centre Division of Mechanical Sciences > Centre for Atmospheric & Oceanic Sciences Division of Electrical Sciences > Computer Science & Automation
Date Deposited:	14 Oct 2008 11:27
Last Modified:	19 Sep 2010 04:50
URI:	http://eprints.iisc.ac.in/id/eprint/16075

Actions (login required)

View Item