ePrints@IIScePrints@IISc Home | About | Browse | Latest Additions | Advanced Search | Contact | Help

Latency Optimal Storage and Scheduling of Replicated Fragments for Memory Constrained Servers

Jinan, R and Badita, A and Sarvepalli, PK and Parag, P (2022) Latency Optimal Storage and Scheduling of Replicated Fragments for Memory Constrained Servers. In: IEEE Transactions on Information Theory, 68 (6). pp. 4135-4155.

IEEE_tra_inf_the_68-6_4135-4155_2022.pdf - Published Version

Download (2MB) | Preview
Official URL: https://doi.org/10.1109/TIT.2022.3152182


We consider the setting of a distributed storage system where a single file is subdivided into smaller fragments of same size which are then replicated with a common replication factor across servers of identical cache size. An incoming file download request is sent to all the servers, and the download is completed whenever the request gathers all the fragments. At each server, we are interested in determining the set of fragments to be stored, and the sequence in which fragments should be accessed, such that the mean file download time for a request is minimized. We model the fragment download time as an exponential random variable independent and identically distributed for all fragments across all servers, and show that the mean file download time can be lower bounded in terms of the expected number of useful servers summed over all distinct fragment downloads. We present deterministic storage schemes that attempt to maximize the number of useful servers. We show that finding the optimal sequence of accessing the fragments is a Markov decision problem, whose complexity grows exponentially with the number of fragments. We propose heuristic algorithms that determine the sequence of access to the fragments which are empirically shown to perform well.

Item Type: Journal Article
Publication: IEEE Transactions on Information Theory
Publisher: Institute of Electrical and Electronics Engineers Inc.
Additional Information: The copyright for this article belongs to the Authors.
Keywords: Heuristic algorithms; Markov processes; Multiprocessing systems; Optimization; Signal encoding, Code; Distributed storage system; Download time; Encodings; Government; Mean download time; Projective plane design; Projective planes; Replication factors; Replication storage code, Scheduling
Department/Centre: Division of Electrical Sciences > Electrical Communication Engineering
Division of Interdisciplinary Sciences > Robert Bosch Centre for Cyber Physical Systems
Date Deposited: 24 Jun 2022 10:12
Last Modified: 24 Jun 2022 10:12
URI: https://eprints.iisc.ac.in/id/eprint/73619

Actions (login required)

View Item View Item