WEATHER-CONDITIONED ATTENTION CNN-LSTM FOR DAY-AHEAD LOAD FORECASTING: A LEAKAGE-FREE COMPARISON WITH FEED-FORWARD BENCHMARKS
-
Abdulla Mohannad Habbib(1*)
Northern Technical University
-
Omar Gheyath Ibrahim(2)
Northern Technical University
-
Mohammed Tariq Majeed(3)
Northern Technical University
(*) Corresponding Author
Abstract
Attention-based deep networks are increasingly proposed for day-ahead load forecasting in smart grids. This study develops a CNN-LSTM forecaster whose temporal attention query is conditioned on the calendar type and hourly temperature of the target day, and it evaluates the model under a protocol designed to avoid three common biases: target leakage across data splits, unequal inputs for benchmarks, and comparison of seed ensembles with single runs. Hourly PJM East load and temperature data for 2013–2018 were split chronologically with purging, and the model was compared with persistence, ridge regression, a multilayer perceptron (MLP), an LSTM and a CNN-LSTM, each stochastic model being trained with three random seeds. With temperature inputs, the proposed model reached a test MAPE of 2.21%, was the most accurate recurrent model, and its attention layer reduced the error of the CNN-LSTM by 20.8% (p < 0.001). However, an MLP given the same information was equally accurate in single runs, significantly more accurate as a seed ensemble (2.13%, p = 0.018) and more robust to temperature-forecast errors, and an exploratory combination of both models reached 2.02%. Deep hybrid forecasters should therefore be judged against well-configured feed-forward benchmarks under symmetric, leakage-free protocols.
Downloads
References
Y. Eren and İ. Küçükdemiral, "A comprehensive review on deep learning approaches for short-term load forecasting," Renewable and Sustainable Energy Reviews, vol. 189, Art. no. 114031, 2024, doi: https://doi.org/10.1016/j.rser.2023.114031.
H. Hou, C. Liu, Q. Wang, X. Wu, J. Tang, Y. Shi, and C. Xie, "Review of load forecasting based on artificial intelligence methodologies, models, and challenges," Electric Power Systems Research, vol. 210, Art. no. 108067, 2022, doi: https://doi.org/10.1016/j.epsr.2022.108067.
F. Dewangan, A. Y. Abdelaziz, and M. Biswal, "Load forecasting models in smart grid using smart meter information: A review," Energies, vol. 16, no. 3, Art. no. 1404, 2023, doi: https://doi.org/10.3390/en16031404.
R. Wazirali, E. Yaghoubi, M. S. S. Abujazar, R. Ahmad, and A. H. Vakili, "State-of-the-art review on energy and load forecasting in microgrids using artificial neural networks, machine learning, and deep learning techniques," Electric Power Systems Research, vol. 225, Art. no. 109792, 2023, doi: https://doi.org/10.1016/j.epsr.2023.109792.
A. Candela Esclapez, M. López García, S. Valero Verdú, and C. Senabre Blanes, "Automatic selection of temperature variables for short-term load forecasting," Sustainability, vol. 14, no. 20, Art. no. 13339, 2022, doi: https://doi.org/10.3390/su142013339.
N. Bashiri Behmiri, C. Fezzi, and F. Ravazzolo, "Incorporating air temperature into mid-term electricity load forecasting models using time-series regressions and neural networks," Energy, vol. 278, Art. no. 127831, 2023, doi: https://doi.org/10.1016/j.energy.2023.127831.
J. S. Caicedo-Vivas and W. Alfonso-Morales, "Short-term load forecasting using an LSTM neural network for a grid operator," Energies, vol. 16, no. 23, Art. no. 7878, 2023, doi: https://doi.org/10.3390/en16237878.
S. Zhang, J. Liu, and J. Wang, "High-resolution load forecasting on multiple time scales using long short-term memory and support vector machine," Energies, vol. 16, no. 4, Art. no. 1806, 2023, doi: https://doi.org/10.3390/en16041806.
F. Saeed, A. Paul, and H. Seo, "A hybrid channel-communication-enabled CNN-LSTM model for electricity load forecasting," Energies, vol. 15, no. 6, Art. no. 2263, 2022, doi: https://doi.org/10.3390/en15062263.
A. Wan, Q. Chang, K. AL-Bukhaiti, and J. He, "Short-term power load forecasting for combined heat and power using CNN-LSTM enhanced by attention mechanism," Energy, vol. 282, Art. no. 128274, 2023, doi: https://doi.org/10.1016/j.energy.2023.128274.
J. Lin, J. Ma, J. Zhu, and Y. Cui, "Short-term load forecasting based on LSTM networks considering attention mechanism," International Journal of Electrical Power & Energy Systems, vol. 137, Art. no. 107818, 2022, doi: https://doi.org/10.1016/j.ijepes.2021.107818.
A. L'Heureux, K. Grolinger, and M. A. M. Capretz, "Transformer-based model for electrical load forecasting," Energies, vol. 15, no. 14, Art. no. 4993, 2022, doi: https://doi.org/10.3390/en15144993.
C. Wang, Y. Wang, Z. Ding, T. Zheng, J. Hu, and K. Zhang, "A transformer-based method of multienergy load forecasting in integrated energy system," IEEE Transactions on Smart Grid, vol. 13, no. 4, pp. 2703–2714, 2022, doi: https://doi.org/10.1109/TSG.2022.3166600.
W. Choi and S. Lee, "Interpretable deep learning model for load and temperature forecasting: Depending on encoding length, models may be cheating on wrong answers," Energy and Buildings, vol. 297, Art. no. 113410, 2023, doi: https://doi.org/10.1016/j.enbuild.2023.113410.
A. Zeng, M. Chen, L. Zhang, and Q. Xu, "Are transformers effective for time series forecasting?," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 9, pp. 11121–11128, 2023, doi: https://doi.org/10.1609/aaai.v37i9.26317.
C. Challu, K. G. Olivares, B. N. Oreshkin, F. Garza Ramirez, M. Mergenthaler Canseco, and A. Dubrawski, "NHITS: Neural hierarchical interpolation for time series forecasting," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 6, pp. 6989–6997, 2023, doi: https://doi.org/10.1609/aaai.v37i6.25854.
V. Ekambaram, A. Jati, N. Nguyen, P. Sinthong, and J. Kalagnanam, "TSMixer: Lightweight MLP-Mixer model for multivariate time series forecasting," in Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 459–469, 2023, doi: https://doi.org/10.1145/3580305.3599533.
Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam, "A time series is worth 64 words: Long-term forecasting with transformers," in Proceedings of the International Conference on Learning Representations (ICLR), 2023. Available: https://openreview.net/forum?id=Jbdc0vTOcol.
F. Petropoulos et al., "Forecasting: theory and practice," International Journal of Forecasting, vol. 38, no. 3, pp. 705–871, 2022, doi: https://doi.org/10.1016/j.ijforecast.2021.11.001.
H. Hewamalage, K. Ackermann, and C. Bergmeir, "Forecast evaluation for data scientists: Common pitfalls and best practices," Data Mining and Knowledge Discovery, vol. 37, no. 2, pp. 788–832, 2023, doi: https://doi.org/10.1007/s10618-022-00894-5.
S. Kapoor and A. Narayanan, "Leakage and the reproducibility crisis in machine-learning-based science," Patterns, vol. 4, no. 9, Art. no. 100804, 2023, doi: https://doi.org/10.1016/j.patter.2023.100804.
R. Gao, L. Du, P. N. Suganthan, Q. Zhou, and K. F. Yuen, "Random vector functional link neural network based ensemble deep learning for short-term load forecasting," Expert Systems with Applications, vol. 206, Art. no. 117784, 2022, doi: https://doi.org/10.1016/j.eswa.2022.117784.
X. Wang, R. J. Hyndman, F. Li, and Y. Kang, "Forecast combinations: An over 50-year review," International Journal of Forecasting, vol. 39, no. 4, pp. 1518–1547, 2023, doi: https://doi.org/10.1016/j.ijforecast.2022.11.005.
R. Mulla, "Hourly energy consumption (PJM)," Kaggle dataset, 2018. Accessed: Sep. 29, 2026. Available: https://www.kaggle.com/datasets/robikscube/hourly-energy-consumption.
K. Xie, A. Giacomoni, X. Deng, and Y. Wu, "Bias calibration and error propagation adjustment for ML-based time series forecasting: A systematic study for PJM's electricity load forecast amid Virginia's data center surge," Energy, vol. 336, Art. no. 138411, 2025, doi: https://doi.org/10.1016/j.energy.2025.138411.
P. Zippenfenig, "Open-Meteo.com Weather API," Zenodo, 2024, doi: https://doi.org/10.5281/zenodo.7970649.
S. Bani Melhem, M. Golec, S. Alrabaee, M. Alshaikh, and M. Uyar, "EdgeAI-powered hybrid ESN-GRU model for high-accuracy and efficient short-term load forecasting in smart grids," IEEE Access, vol. 13, pp. 198241–198251, 2025, doi: https://doi.org/10.1109/ACCESS.2025.3631672.
Y. Yang, Q. Xing, K. Wang, C. Li, J. Wang, and X. Huang, "A novel combined probabilistic load forecasting system integrating hybrid quantile regression and knee improved multi-objective optimization strategy," Applied Energy, vol. 356, Art. no. 122341, 2024, doi: https://doi.org/10.1016/j.apenergy.2023.122341.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
This work is licensed under CC BY-SA 4.0
