Jinan, R and Badita, A and Sarvepalli, PK and Parag, P (2022) Latency Optimal Storage and Scheduling of Replicated Fragments for Memory Constrained Servers. In: IEEE Transactions on Information Theory, 68 (6). pp. 4135-4155.
|
PDF
IEEE_tra_inf_the_68-6_4135-4155_2022.pdf - Published Version Download (2MB) | Preview |
Abstract
We consider the setting of a distributed storage system where a single file is subdivided into smaller fragments of same size which are then replicated with a common replication factor across servers of identical cache size. An incoming file download request is sent to all the servers, and the download is completed whenever the request gathers all the fragments. At each server, we are interested in determining the set of fragments to be stored, and the sequence in which fragments should be accessed, such that the mean file download time for a request is minimized. We model the fragment download time as an exponential random variable independent and identically distributed for all fragments across all servers, and show that the mean file download time can be lower bounded in terms of the expected number of useful servers summed over all distinct fragment downloads. We present deterministic storage schemes that attempt to maximize the number of useful servers. We show that finding the optimal sequence of accessing the fragments is a Markov decision problem, whose complexity grows exponentially with the number of fragments. We propose heuristic algorithms that determine the sequence of access to the fragments which are empirically shown to perform well.
| Item Type: | Journal Article |
|---|---|
| Publication: | IEEE Transactions on Information Theory |
| Publisher: | Institute of Electrical and Electronics Engineers Inc. |
| Additional Information: | The copyright for this article belongs to the Authors. |
| Keywords: | Heuristic algorithms; Markov processes; Multiprocessing systems; Optimization; Signal encoding, Code; Distributed storage system; Download time; Encodings; Government; Mean download time; Projective plane design; Projective planes; Replication factors; Replication storage code, Scheduling |
| Department/Centre: | Division of Electrical Sciences > Electrical Communication Engineering Division of Interdisciplinary Sciences > Robert Bosch Centre for Cyber Physical Systems |
| Date Deposited: | 24 Jun 2022 10:12 |
| Last Modified: | 24 Jun 2022 10:12 |
| URI: | https://eprints.iisc.ac.in/id/eprint/73619 |
Actions (login required)
 |
View Item |
