On The Evaluation of District Heating Load Predictions

Abstract: District Heating is a technology with the potential to enable a fossil-free society. However, to realize this potential, some improvements need to be made in order to improve District Heating operation at large, decrease losses in the systems, and thus increase the competitiveness of District Heating as a technology. Several have shown that a possible solution, to increase the efficiency of District Heating, is to utilize data-based methods such as Machine Learning. Questions have been raised regarding studies on Machine Learning implemented on District Heating data, stating that the research does not reflect the actual needs of District Heating utilities. Therefore, the aim of this report was to investigate how Machine Learning models, implemented to predict heat load, could be evaluated in a way that aligns with how they are used in District Heating operation. Heat load predictions have a number of use cases, for example as a way to plan and optimize heat production. The report was divided into three different investigations. A survey study for District Heating utilities to give insight into how they currently use heat load prediction models as well as how they evaluate these models. Based on theory regarding District Heating, Machine Learning, and statistical error measures, it was investigated how heat load predictive models could be evaluated in a suitable way. The investigation resulted in the proposal of an evaluation framework. Lastly, the heat load prediction model Energy Predict, developed by Utilifeed, a software provider to District Heating utilities, was evaluated against this framework. Some conclusions could be drawn from the studies. According to the results from the survey, heat load predictions are used in District Heating utilities for planning production and sales, dimensioning equipment/infrastructure/production, and as a step in fault detection. However, it seemed as if the accuracy of these models is generally not evaluated in District Heating utilities today. From the theory section, it was concluded that research on heat load predictions has been limited to short-term load forecasts, used for planning production, and as a step in detecting faults. As a result, two evaluation frameworks were proposed, evaluating the predictive performance of heat load prediction models used for dimensioning and sales planning. It was concluded that the two purposes, dimensioning and sales planning, require different kinds of accuracy and thus also different kinds of evaluation frameworks. For dimensioning, it was considered valuable to predict peaks in heat load, and for sales planning, it was considered valuable to predict accumulated sums of heat loads. The insights gathered regarding evaluation metrics and periods were used when proposing the two evaluation frameworks. The frameworks utilized the error measures Coefficient of Variance of Root Mean Squared Error and Normalized Mean Bias Error, as well as different sectionings of the validation data, assessing the common flaws of heat load prediction models based on Machine Learning. The two evaluation frameworks were showcased by evaluating the predictive performance of four different load prediction models: Energy Predict, heat load signature, Supported Vector Regressor, and XGBoost. Energy Predict showed the best performance of all four models on both frameworks. These frameworks, and the discussion provided with them, are developed for District Heating utilities and other users and developers of heat load prediction models. As different models are compared with each other, or when measures of accuracy need to be quantified, these frameworks may be found valuable.