Holistic Measures for Evaluating Prediction Models in Smart Grids

Aman, Saima and Simmhan, Yogesh and Prasanna, Viktor K (2014) Holistic Measures for Evaluating Prediction Models in Smart Grids. In: IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 27 (2). pp. 475-488.

PDF iee_tra_kno_dat_eng_27-2_475_2014.pdf - Published Version Restricted to Registered users only Download (2MB) | Request a copy

Abstract

The performance of prediction models is often based on ``abstract metrics'' that estimate the model's ability to limit residual errors between the observed and predicted values. However, meaningful evaluation and selection of prediction models for end-user domains requires holistic and application-sensitive performance measures. Inspired by energy consumption prediction models used in the emerging ``big data'' domain of Smart Power Grids, we propose a suite of performance measures to rationally compare models along the dimensions of scale independence, reliability, volatility and cost. We include both application independent and dependent measures, the latter parameterized to allow customization by domain experts to fit their scenario. While our measures are generalizable to other domains, we offer an empirical analysis using real energy use data for three Smart Grid applications: planning, customer education and demand response, which are relevant for energy sustainability. Our results underscore the value of the proposed measures to offer a deeper insight into models' behavior and their impact on real applications, which benefit both data mining researchers and practitioners.

Item Type:	Journal Article
Publication:	IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING
Publisher:	IEEE COMPUTER SOC
Additional Information:	Copy right for this article belongs to the IEEE COMPUTER SOC, 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1314 USA
Keywords:	Consumption prediction; performance measures; time series forecasting; regression tree learning; smart grids; energy sustainability
Department/Centre:	Division of Interdisciplinary Sciences > Supercomputer Education & Research Centre
Date Deposited:	06 Feb 2015 15:06
Last Modified:	06 Feb 2015 15:06
URI:	http://eprints.iisc.ac.in/id/eprint/50791

Actions (login required)

View Item