TorqueDB: Distributed querying of time-series data from edge-local storage

Garg, D and Shirolkar, P and Shukla, A and Simmhan, Y (2020) TorqueDB: Distributed querying of time-series data from edge-local storage. In: 26th International European Conference on Parallel and Distributed Computing, Euro-Par 2020, 24-28 August 2020, Warsaw; Poland, pp. 281-295.

PDF lec_not_com_sci_12247_281-295_2020.pdf - Published Version Restricted to Registered users only Download (1MB) | Request a copy

Abstract

The rapid growth in edge computing devices as part of Internet of Things (IoT) allows real-time access to time-series data from 1000�s of sensors. Such observations are often queried to optimize the health of the infrastructure. Recently, edge storage systems allow us to retain data on the edge rather than moving them centrally to the cloud. However, such systems do not support flexible querying over the data spread across 10�100�s of devices. There is also a lack of distributed time-series databases that can run on the edge devices. Here, we propose TorqueDB, a distributed query engine over time-series data that operates on edge and fog resources. TorqueDB leverages our prior work on ElfStore, a distributed edge-local file store, and InfluxDB, a time-series database, to enable temporal queries to be decomposed and executed across multiple fog and edge devices. Interestingly, we move data into InfluxDB on-demand while retaining the durable data within ElfStore for use by other applications. We also design a cost model that maximizes parallel movement and execution of the queries across resources, and utilizes caching. Our experiments on a real edge, fog and cloud deployment show that TorqueDB performs comparable to InfluxDB on a cloud VM for a smart city query workload, but without the associated monetary costs. © Springer Nature Switzerland AG 2020.

Item Type:	Conference Paper
Publication:	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Publisher:	Springer
Additional Information:	The copyright of this article belongs to Springer
Keywords:	Digital storage; Fog; Internet of things; Query languages; Query processing; Search engines; Time series, Cloud deployments; Computing devices; Distributed query; Distributed time series; Flexible querying; Internet of Things (IOT); Real-time access; Time Series Database, Distributed database systems
Department/Centre:	Division of Interdisciplinary Sciences > Computational and Data Sciences
Date Deposited:	29 Sep 2020 10:19
Last Modified:	29 Sep 2020 10:19
URI:	http://eprints.iisc.ac.in/id/eprint/66542

Actions (login required)

View Item