Abstract
The article deals with the problem of forecasting the power consumption at mining enterprises, which is characterized by a high nonstationarity and stochasticity levels. The data collected by the authors for four years on the mining enterprise of Yakutia, working in the field of coal mining and processing, are used. At the same time, a separate analysis was carried out for various objects of the enterprise: a coal mine and processing plants, which have fundamentally different technological processes, and, consequently, power consumption schedules. A study of two classes of machine learning methods was carried out: processing of retrospective data on the power consumption at an enterprise as a time series using recurrent neural networks; and extraction of the most significant features in order to apply to them ensemble models based on decision trees: a random forest, adaptive boosting and extreme gradient boosting. Since tuning hyper-parameters is very important for the specified machine learning models, for the correct comparison of the results, the procedure for optimizing the hyper-parameters of all models was carried out. The computational experiments have shown that recurrent multilayer neural networks are able to use time series for forecasting without preliminary processing, learning to recognize significant signs from the dynamics of changes in the electrical consumption schedule. To apply ensembles of regression decision trees, preliminary data analysis is required to extract the most significant features from a time series. Using the example of the considered enterprise, it is shown that the use of such an approach when working with ensemble models gives an accuracy close to that of recurrent neural networks. In this case, ensemble models are trained 1–2 orders of magnitude faster, and the disadvantage is a great tendency to overfitting.
Keywords
Opencast mining, power consumption forecasting, machine learning, recurrent neural network, feature selection, ensemble models.
1. Park S., Ruy S., Choi Y., Kim J., Kim H. Data-driven baseline estimation of residential buildings for demand response. Energies, 2015, vol. 8. pp. 10239-10259. doi: 10.3390/en80910239
2. Hahn H., Meyer-Nieberg S., Pickl S. Electric load forecasting methods: Tools for decision making. European Journal of Operational Research, 2009, vol. 199, no. 3, pp. 902-907. doi: 10.1016/j.ejor.2009.01.062
3. Deb C., Zhang F., Yang J., Lee S., Shah K. A review on time series forecasting techniques for building energy consumption. Renewable and Sustainable Energy Reviews, 2017, vol. 74 (C), pp. 902-924.
4. Taylor J.W. Short-term electricity demand forecasting using double seasonal exponential smoothing. The Journal of the Operational Research Society, 2003, vol. 54, no. 8, pp. 799–805.
5. Potapov V., Khamitov R., Makarov V., Gritsay A., Chervenchuk I., Tyunkov D. Short-Term Forecast of Electricity Load for LLC "Omsk Energy Retail Company" Using Neural Network. 2018 Dynamics of Systems, Mechanisms and Machines (Dynamics). Omsk, 2018, pp. 1-5. 10.1109/Dynamics.2018.8601430
6. Cho M.Y., Hwang J.C., Chen C.S. Customer short term load forecasting by using ARIMA transfer function model. International Conference on Energy Management and Power Delivery. Singapore, 1995, vol. 1, pp. 317-322. doi: 10.1109/EMPD.1995.500746
7. Hagan M.T., Behr S.M. The time series approach to short term load forecasting. IEEE Trans. Power Syst, 1987, vol. 2, pp. 785-791. doi: 10.1109/TPWRS.1987.4335210
8. Khujasaidov J.Kh., Rusina A.G., Matrenin P.V., Dmitriev S.A., Safaraliev M.Kh. Forecasting model development for electricity consumption based on time. Elektrotekhnicheskie sistemy i kompleksy [Electrotechnical Systems and Complexes], 2020, no. 3 (48), pp. 23-27. (In Russian). doi: 10.18503/2311-8318-2020-3(48)-23-27
9. Song K.B., Baek Y.S., Hong D.H., Jang G. Short-term load forecasting for the holidays using fuzzy linear regression method. IEEE Trans. Power Syst, 2005, vol. 20, no. 1, pp. 96-101. doi: 10.1109/PES.2005.1489152
10. Antonenkov D.V., Solovev D.B. Mathematic simulation of mining company’s power demand forecast (by example of "Neryungri" coal strip mine). IOP Conf. Series: Earth and Environmental Science, 2017, vol. 87, Art. no. 032003. doi: 10.1088/1755-1315/87/3/032003
11. Hong W.C. Electric load forecasting by seasonal recurrent SVR (support vector regression) with chaotic artificial bee colony algorithm. Energy. 2011, vol. 36, no. 9. pp. 5568-5578. doi: 10.1016/j.energy.2011.07.015
12. Li W., Yang X., Li H., Su L. Hybrid forecasting approach based on GRNN neural network and SVR machine for electricity demand forecasting. Energies, 2017, vol. 10, no. 1, Art. id 44. doi: 10.3390/en10010044
13. Huo J., Shi T., Chang J. Comparison of Random Forest and SVM for electrical short-term load forecast with different data sources. IEEE International Conference on Software Engineering and Service Science, Beijing. IEEE, 2016, pp. 1077-1080. doi: 10.1109/PESGM.2016.7741097
14. Zhang R., Xu Y., Dong Z.Y., Kong W., Wong, K.P. A composite k-nearest neighbor model for day-ahead load forecasting with limited temperature forecasts. 2016 IEEE Power and Energy Society General Meeting (PESGM). IEEE, 2016, pp. 1-5. doi: 10.1109/PESGM.2016.7741097
15. Mori H., Kosemura N. Optimal regression tree based rule discovery for short-term load forecasting. 2001 IEEE Power Engineering Society Winter Meeting. Conference Proceedings (Cat. No.01CH37194). IEEE, 2001, vol. 2, pp. 421-426. doi: 10.1109/PESW.2001.916878
16. Qiu X., Zhang L., Ren Y., Suganthan P., Amaratunga G. Ensemble deep learning for regression and time series forecasting. 2014 IEEE Symposium on Computational Intelligence in Ensemble Learning (CIEL). IEEE, 2014, pp. 21-26. doi: 10.1109/CIEL.2014.7015739
17. Deng Z., Wang B., Xu Y., Xu T., Liu C., Zhu Z. Multi-Scale Convolutional Neural Network with Time-Cognition for Multi-Step Short-Term Load Forecasting. IEEE Access, 2019, vol. 7, pp. 88058-88071. doi: 10.1109/ACCESS.2019.2926137
18. Wu, L.; Kong, C.; Hao, X.; Chen, W. A Short-Term Load Forecasting Method Based on GRU-CNN Hybrid Neural Network Model. Mathematical Problems in Engineering, 2020, Art. no. 1428104. doi: 10.1155/2020/1428104
19. Ryu S., Noh J., Kim H. Deep neural network based demand side short term load forecasting. Energies, 2016, vol. 10, no. 1, Art. id 3, pp. 1-20. doi: 10.3390/en10010003
20. Chen K., Wang Q., He Z., Hu J., He J. Short-Term Load Forecasting with Deep Residual Networks. IEEE Trans. Smart Grid, 2019, vol. 10, no. 4, pp. 3943-3952. doi: 10.1109/TSG.2018.2844307
21. Cai M., Pipattanasomporn M., Rahman S. Day-ahead building-level load forecasts using deep learning vs. traditional time-series techniques. Applied Energy, 2019, vol. 236, pp. 1078-1088. doi: 10.1016/j.apenergy.2018.12.042
22. Kong W., Dong Z., Jia Y., Hill D., Xu Y., Zhang Y. Short-term residential load forecasting based on LSTM recurrent neural network. IEEE Trans. on Smart Grid, 2019, vol. 10, no. 1, pp. 841-851. doi: 10.1109/TSG.2017.2753802
23. Matrenin P.V., Manusov V.Z., Khalyasmaa A.I., Antonenkov D.V., Eroshenko S.A., Butusov D. Improving accuracy and generalization performance of small-size recurrent neural networks applied to short-term load forecasting. Mathematics, 2020, vol. 8, no. 12, Art. id 2169. doi: 10.3390/math8122169
24. Hochreiter S., Schmidhuber J. Long short-term memory. Neural Computation, 1997, vol. 9, no. 8, pp. 1735-1780. doi: 10.1162/neco.1997.9.8.1735
25. Cho K., van Merriënboer B., Gulcehre C., Bahdanau D, Bougares F., Schwenk H., Bengio Y. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. Available at: https://arxiv.org/abs/1406.1078
26. LeCun Y., Bengio Y., Hinton G. Nature, 2015, vol. 521, pp. 436-444. doi: 10.1038/nature14539
27. Dahl G.E., Sainath T.N., Hinton G.E. Improving deep neural networks for LVCSR using rectified linear units and dropout. 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE, 2013, pp. 8609-8613. doi: 10.1109/ICASSP.2013.6639346
28. Ng A.Y. Feature selection, L1 vs. L2 regularization, and rotational invariance. Proceedings of the twenty-first international conference on Machine learning (ICML '04). Association for Computing Machinery, 2004, pp. 1-8. doi: 10.1145/1015330.1015435
29. Kingma D.P., Ba J.L. Adam: A method for stochastic optimization. Available at: https://arxiv.org/abs/1412.6980
30. Breiman L. Random Forests. Machine Learning. 2001. Vol. 4. Pp. 5-32. doi: 10.1023/A:1010933404324.
31. Machine learning in Python. Available at: https://scikit-learn.org
32. Drucker H. Improving Regressors using Boosting Techniques. Available at: https://www.researchgate.net/publication/2424244
33. Chen T., Guestrin C. XGBoost: A Scalable Tree Boosting System. Available at: https://arxiv.org/abs/1603.02754
34. Khalyasmaa A.I., Eroshenko S.A., Tashchilin V.A., Ramachandran H. Chakravarthi T.P., Butusov D.N. Industry Experience of Developing Day-Ahead Photovoltaic Plant Forecasting System Based on Machine Learning. Remote Sensing, 2020, vol. 12 (20), Art. id 3420. doi: 10.3390/rs12203420
35. Khalyasmaa A.I., Senyuk M.D., Eroshenko S.A. Analysis of the state of high-voltage current transformers based on gradient boosting on decision trees. IEEE Transactions on Power Delivery, 2020, vol. 36, no. 4, pp. 2154-2163. doi: 10.1109/TPWRD.2020.3021702
36. Asanova S.M., Ahyeev D.S. Dmitriev S.A., Matrenin P.V., Safaraliev M.Kh. Development of models for power consumption medium-term forecasting in isolated power systems based on ensemble methods of machine learning. Izvestiya NTTS Edinoy energeticheskoy sistemy [STC of Unified Power System Proceedings], 2021, no. 1(84), pp. 32-39. (In Russian)
37. Bergstra J., Bengio Y. Random search for hyper-parameter optimization. Journal of Machine Learning Research, 2012, vol. 13, pp. 281-305.
38. Wolpert D.H., Macready W.G. No Free Lunch Theorems for Optimization. IEEE Transactions on Evolutionary Computation, 1997, vol. 1, no. 1, pp. 67-82. doi: 10.1109/4235.585893