Prashanthi, SK and Hegde, V and Patchava, K and Das, A and Simmhan, Y (2023) Performance Characterization of Containerized DNN Training and Inference on Edge Accelerators. In: 30th Annual IEEE International Conference on High Performance Computing, Data, and Analytics, HiPC 2023, 18 December 2023 through 21 December 2023, Goa, pp. 127-131.
|
PDF
pro_2023_iee_iee_30th_int_con_hig_percom_dat_ana_2023.pdf - Published Version Download (456kB) | Preview |
Abstract
Edge devices have typically been used for DNN in-ferencing. The increase in the compute power of accelerated edges is leading to their use in DNN training also. As privacy becomes a concern on multi-tenant edge devices, Docker containers provide a lightweight virtualization mechanism to sandbox models. But their overheads for edge devices are not yet explored. In this work, we study the impact of containerized DNN inference and training workloads on an NVIDIA AGX Orin edge device and contrast it against bare metal execution on running time, CPU, GPU and memory utilization, and energy consumption. Our analysis provides several interesting insights on these overheads. © 2023 IEEE.
| Item Type: | Conference Paper |
|---|---|
| Publication: | Proceedings - 2023 IEEE 30th International Conference on High Performance Computing, Data, and Analytics, HiPC 2023 |
| Publisher: | Institute of Electrical and Electronics Engineers Inc. |
| Additional Information: | The copyright for this article belongs to Authors. |
| Keywords: | Containers, Bare metals; DNN inference; DNN training; Docker; Edge accelerator; Multi tenants; Performance characterization; Power; Running time; Virtualizations, Energy utilization |
| Department/Centre: | Division of Interdisciplinary Sciences > Computational and Data Sciences |
| Date Deposited: | 02 Sep 2024 10:17 |
| Last Modified: | 02 Sep 2024 10:17 |
| URI: | http://eprints.iisc.ac.in/id/eprint/84954 |
Actions (login required)
 |
View Item |
