research-article

FAMC-Net: Frequency Domain Parity Correction Attention and Multi-Scale Dilated Convolution for Time Series Forecasting

Authors:

Fan ZhangAuthors Info & Claims

CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

Pages 2554 - 2563

https://doi.org/10.1145/3583780.3614876

Published: 21 October 2023 Publication History

Abstract

In recent years, time series forecasting models based on the Transformer framework have shown great potential, but they suffer from the inherent drawback of high computational complexity and only focus on global modeling. Inspired by trend-seasonality decomposition, we propose a method that combines global modeling with local feature extraction within the seasonal cycle. It aims at capturing the global view while fully exploring the potential features within each seasonal cycle and better expressing the long-term and periodic characteristics of time series. We introduce a frequency domain parity correction block to compute global attention and utilize multi-scale dilated convolution to extract local correlations within each cycle. Additionally, we adopt a dual-branch structure to separately model the seasonality and trend based on their intrinsic features, improving prediction performance and enhancing model interpretability. This model is implemented on a completely single-layer decoder architecture, breaking through the traditional encoder-decoder architecture paradigm and reducing computational complexity to a certain extent. We conducted sufficient experimental validation on eight benchmark datasets, and the results demonstrate its superior performance compared to existing methods in both univariate and multivariate forecasting.

References

[1]

Changlu Chen, Yanbin Liu, Ling Chen, and Chengqi Zhang. 2022. Bidirectional Spatial-Temporal Adaptive Transformer for Urban Traffic Flow Forecasting. IEEE Transactions on Neural Networks and Learning Systems (June 2022), 1--13. https://doi.org/10.1109/TNNLS.2022.3183903

[2]

Mu-Yen Chen and Bo-Tsuen Chen. 2014. Online fuzzy time series analysis based on entropy discretization and a Fast Fourier Transform. Applied Soft Computing, Vol. 14 (January 2014), 156--166. https://doi.org/10.1016/j.asoc.2013.07.024

Digital Library

[3]

James W Cooley and John W Tukey. 1965. An algorithm for the machine calculation of complex Fourier series. Mathematics of computation, Vol. 19, 90 (May 1965), 297--301. https://doi.org/10.2307/2003354

[4]

Marzieh Fathi, Mostafa Haghi Kashani, Seyed Mahdi Jameii, and Ebrahim Mahdipour. 2022. Big data analytics in weather forecasting: A systematic review. Archives of Computational Methods in Engineering, Vol. 29, 2 (March 2022), 1247--1275. https://doi.org/10.1007/s11831-021-09616--4

[5]

Philip Hans Franses. 1991. Seasonality, non-stationarity and the forecasting of monthly time series. International Journal of forecasting, Vol. 7, 2 (August 1991), 199--208. https://doi.org/10.1016/0169--2070(91)90054-y

[6]

Xiulin Geng, Xiaoyu He, Lingyu Xu, and Jie Yu. 2022. Graph correlated attention recurrent neural network for multivariate time series forecasting. Information Sciences, Vol. 606 (August 2022), 126--142. https://doi.org/10.1016/j.ins.2022.04.045

Digital Library

[7]

Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics (Proceedings of Machine Learning Research, Vol. 9). JMLR Workshop and Conference Proceedings, PMLR, Chia Laguna Resort, Sardinia, Italy, 249--256. https://proceedings.mlr.press/v9/glorot10a.html

[8]

Clive WJ Granger. 1969. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, Vol. 37 (August 1969), 424--438. https://doi.org/10.5555/781840.781842

Digital Library

[9]

Meng-Hao Guo, Tian-Xing Xu, Jiang-Jiang Liu, Zheng-Ning Liu, Peng-Tao Jiang, Tai-Jiang Mu, Song-Hai Zhang, Ralph R Martin, Ming-Ming Cheng, and Shi-Min Hu. 2022. Attention mechanisms in computer vision: A survey. Computational Visual Media, Vol. 8, 3 (March 2022), 331--368. https://doi.org/10.1007/s41095-022-0271-y

[10]

Pradeep Hewage, Ardhendu Behera, Marcello Trovati, Ella Pereira, Morteza Ghahremani, Francesco Palmieri, and Yonghuai Liu. 2020. Temporal convolutional neural (TCN) network for an effective weather forecasting using time-series data from the local weather station. Soft Computing, Vol. 24 (April 2020), 16453--16482. https://doi.org/10.1007/s00500-020-04954-0

Digital Library

[11]

Diksha Khurana, Aditya Koli, Kiran Khatter, and Sukhdev Singh. 2023. Natural language processing: State of the art, current trends and challenges. Multimedia tools and applications, Vol. 82, 3 (July 2023), 3713--3744. https://doi.org/10.1007/s11042-022--13428--4

[12]

Nikita Kitaev, Łukasz Kaiser, and Anselm Levskaya. 2020. Reformer: The efficient transformer. In Proceedings of the 8th international conference on learning representations, Vol. 9. ICLR, Washington DC, 26--30. https://doi.org/10.48550/arXiv.2001.04451

[13]

Mahinda Mailagaha Kumbure, Christoph Lohrmann, Pasi Luukka, and Jari Porras. 2022. Machine learning techniques and data for stock market forecasting: A literature review. Expert Systems with Applications, Vol. 197 (July 2022), 116659. https://doi.org/10.1016/j.eswa.2022.116659

Digital Library

[14]

Ryno Laubscher. 2019. Time-series forecasting of coal-fired power plant reheater metal temperatures using encoder-decoder recurrent neural networks. Energy, Vol. 189 (December 2019), 116187. https://doi.org/10.1016/j.energy.2019.116187

[15]

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE, Vol. 86, 11 (November 1998), 2278--2324. https://doi.org/10.1109/5.726791

[16]

Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. 2019. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. In Advances in neural information processing systems, Vol. 32. Curran Associates, Inc., Red Hook, NY. https://doi.org/10.48550/arXiv.1907.00235

[17]

Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dustdar. 2021. Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting. In International conference on learning representations. ICLR, Washington DC. https://openreview.net/forum?id=0EXmFzUn5I

[18]

Xin Liu, Junhong Guo, Hua Wang, and Fan Zhang. 2022. Prediction of stock market index based on ISSA-BP neural network. Expert Systems with Applications, Vol. 204 (October 2022), 117604. https://doi.org/10.1016/j.eswa.2022.117604

Digital Library

[19]

Yisheng Lv, Yanjie Duan, Wenwen Kang, Zhengxi Li, and Fei-Yue Wang. 2014. Traffic flow prediction with big data: A deep learning approach. IEEE Transactions on Intelligent Transportation Systems, Vol. 16, 2 (April 2014), 865--873. https://doi.org/10.1109/TITS.2014.2345663

Digital Library

[20]

KK Minu, MC Lineesh, and C Jessy John. 2010. Wavelet neural networks for nonlinear time series analysis. Applied Mathematical Sciences, Vol. 4, 50 (July 2010), 2485--2495. https://doi.org/10.1007/978--3--540--28648--6_68

[21]

Boris N Oreshkin, Grzegorz Dudek, Paweł Pełka, and Ekaterina Turkina. 2021. N-BEATS neural network for mid-term electricity load forecasting. Applied Energy, Vol. 293 (July 2021), 116918. https://doi.org/10.1016/j.apenergy.2021.116918

[22]

Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Noam Shazeer, Alexander Ku, and Dustin Tran. 2018. Image transformer. In International conference on machine learning, Vol. 80. PMLR, PMLR, New York, 4055--4064. https://doi.org/10.48550/arXiv.1802.05751

[23]

Syama Sundar Rangapuram, Matthias W Seeger, Jan Gasthaus, Lorenzo Stella, Yuyang Wang, and Tim Januschowski. 2018. Deep State Space Models for Time Series Forecasting. In Advances in Neural Information Processing Systems, Vol. 31. Curran Associates, Inc., Red Hook, NY. https://doi.org/10.5555/3327757.3327876

Digital Library

[24]

Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. 2015. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Advances in Neural Information Processing Systems, Vol. 28. Curran Associates, Inc., Red Hook, NY. https://doi.org/10.1609/aaai.v33i01.33015409

Digital Library

[25]

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, Vol. 27. Curran Associates, Inc., Red Hook, NY. https://doi.org/10.5555/2969033.2969173

Digital Library

[26]

Pengfei Tang, Peijun Du, Junshi Xia, Peng Zhang, and Wei Zhang. 2022a. Channel attention-based temporal convolutional network for satellite image time series classification. IEEE Geoscience and Remote Sensing Letters, Vol. 19 (July 2022), 1--5. https://doi.org/10.1109/LGRS.2021.3095505

[27]

Yuqing Tang, Fusheng Yu, Witold Pedrycz, Xiyang Yang, Jiayin Wang, and Shihu Liu. 2022b. Building trend fuzzy granulation-based LSTM recurrent neural network for long-term time-series forecasting. IEEE transactions on fuzzy systems, Vol. 30, 6 (March 2022), 1599--1613. https://doi.org/10.1109/TFUZZ.2021.3062723

[28]

James W Taylor. 2008. An evaluation of methods for very short-term load forecasting using minute-by-minute British data. International journal of forecasting, Vol. 24, 4 (October 2008), 645--658. https://doi.org/10.1016/j.ijforecast.2008.07.007

[29]

Alexander Toshev and Christian Szegedy. 2014. Deeppose: Human pose estimation via deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, New York, 1653--1660. https://doi.org/10.1109/cvpr.2014.214

Digital Library

[30]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc., Red Hook, NY. https://doi.org/10.48550/arXiv.1706.03762

[31]

Fan Wang, Guangshun Li, Yilei Wang, Wajid Rafique, Mohammad R Khosravi, Guanfeng Liu, Yuwen Liu, and Lianyong Qi*. 2022. Privacy-aware traffic flow prediction based on multi-party sensor data with zero trust in smart city. ACM Transactions on Internet Technology (TOIT) (March 2022). https://doi.org/10.1145/3511904

Digital Library

[32]

Qingsong Wen, Jingkun Gao, Xiaomin Song, Liang Sun, Huan Xu, and Shenghuo Zhu. 2019. RobustSTL: A robust seasonal-trend decomposition algorithm for long time series. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. AAAI, Menlo Park, CA, 5409--5416. https://doi.org/10.1609/aaai.v33i01.33015409

Digital Library

[33]

Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. 2021. Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Advances in Neural Information Processing Systems, Vol. 34. Curran Associates, Inc., Red Hook, NY, 22419--22430. https://doi.org/10.48550/arXiv.2106.13008

[34]

Kun Yi, Qi Zhang, Shoujin Wang, Hui He, Guodong Long, and Zhendong Niu. 2023. Neural Time Series Analysis with Fourier Transform: A Survey. arXiv preprint arXiv:2302.02173 (2023). https://doi.org/10.48550/arXiv.2302.02173

[35]

Fan Zhang, Gongguan Chen, Hua Wang, Jinjiang Li, and Caiming Zhang. 2023 a. Multi-scale Video Super-resolution Transformer with Polynomial Approximation. IEEE Transactions on Circuits and Systems for Video Technology (May 2023), 1--1. https://doi.org/10.1109/TCSVT.2023.3278131

Digital Library

[36]

Fan Zhang, Tiantian Guo, and Hua Wang. 2023 b. DFNet: Decomposition fusion model for long sequence time-series forecasting. Knowledge-Based Systems, Vol. 277 (October 2023), 110794 pages. https://doi.org/10.1016/j.knosys.2023.110794

Digital Library

[37]

Yunhao Zhang and Junchi Yan. 2023. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The Eleventh International Conference on Learning Representations. ICLR, Washington DC. https://openreview.net/forum?id=vSVLM2j9eie

[38]

Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. AAAI, Palo Alto, 11106--11115. https://doi.org/10.1609/aaai.v35i12.17325

[39]

Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. 2022. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. In International Conference on Machine Learning, Vol. 162. PMLR, PMLR, New York, 27268--27286. https://doi.org/10.48550/arXiv.2201.12740

Cited By

Tian XHuang SXiao JWang HLiu Y(2025)SDVS-Net: A spatial dilated convolution and variable self-attention network for multivariate long-term time series forecastingNeurocomputing10.1016/j.neucom.2024.129148619(129148)Online publication date: Feb-2025
https://doi.org/10.1016/j.neucom.2024.129148
Yuan LWang HZhang F(2025)ADMNet: An adaptive downsampling multi-frequency multi-channel network for long-term time series forecastingExpert Systems with Applications10.1016/j.eswa.2024.125588262(125588)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125588
Zhou XWang WBuntine WQu SSriramulu ATan WBergmeir CSerra ESpezzano F(2024)Scalable Transformer for High Dimensional Multivariate Time Series ForecastingProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679757(3515-3526)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679757
Show More Cited By

Index Terms

FAMC-Net: Frequency Domain Parity Correction Attention and Multi-Scale Dilated Convolution for Time Series Forecasting
1. Computing methodologies
  1. Artificial intelligence

Recommendations

Combining seasonal ARIMA models with computational intelligence techniques for time series forecasting

Seasonal autoregressive integrated moving average (SARIMA) models form one of the most popular and widely used seasonal time series models over the past three decades. However, in several researches it has been argued that they have two basic ...
A Novel Hybridization of ARIMA, ANN, and K-Means for Time Series Forecasting

This article aims to propose a novel hybrid forecasting model involving autoregressive integrated moving average ARIMA, artificial neural networks ANNs and k-means clustering. The single models and k-means clustering are used to build the hybrid ...
A hybrid evolutionary decomposition system for time series forecasting

In the field of time series forecasting, the combination of linear and nonlinear models has been explored to improve the accuracy of predictions when compared with the performance of individual models. Traditional forecasting methods such as the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management

October 2023

5508 pages

ISBN:9798400701245

DOI:10.1145/3583780

General Chairs:
Ingo Frommholz
University of Wolverhampton, UK
,
Frank Hopfgartner
University of Koblenz, Germany
,
Mark Lee
University of Birmingham, UK
,
Michael Oakes
University of Birmingham, UK
,
Program Chairs:
Mounia Lalmas
Spotify, UK
,
Min Zhang
Tsinghua University, China
,
Rodrygo Santos
Federal University of Minas Gerais, Brazil

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Special Funds of Taishan Scholars Project
Youth Innovation Technology Project of Higher School in Shandong Province
National Natural Science Foundation of China

Conference

CIKM '23

Sponsor:

CIKM '23: The 32nd ACM International Conference on Information and Knowledge Management

October 21 - 25, 2023

Birmingham, United Kingdom

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
248
Total Downloads

Downloads (Last 12 months)168
Downloads (Last 6 weeks)16

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tian XHuang SXiao JWang HLiu Y(2025)SDVS-Net: A spatial dilated convolution and variable self-attention network for multivariate long-term time series forecastingNeurocomputing10.1016/j.neucom.2024.129148619(129148)Online publication date: Feb-2025
https://doi.org/10.1016/j.neucom.2024.129148
Yuan LWang HZhang F(2025)ADMNet: An adaptive downsampling multi-frequency multi-channel network for long-term time series forecastingExpert Systems with Applications10.1016/j.eswa.2024.125588262(125588)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125588
Zhou XWang WBuntine WQu SSriramulu ATan WBergmeir CSerra ESpezzano F(2024)Scalable Transformer for High Dimensional Multivariate Time Series ForecastingProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679757(3515-3526)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679757
Li LWang HLiu YZhang F(2024) Probabilistic interval prediction method based on shape‐adaptive quantile regression Expert Systems10.1111/exsy.1358541:8Online publication date: 21-Mar-2024
https://doi.org/10.1111/exsy.13585
Jiang LZhang F(2024)Using piecewise polynomial activation functions and relevance attention for long-term time-series prediction2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10650022(1-10)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10650022
Zhang FWang MZhang WWang H(2024)THATSN: Temporal Hierarchical Aggregation Tree Structure Network for Long-Term Time-Series ForecastingInformation Sciences10.1016/j.ins.2024.121659(121659)Online publication date: Nov-2024
https://doi.org/10.1016/j.ins.2024.121659
Zhang WWang HZhang F(2024)Spatio-temporal Fourier enhanced heterogeneous graph learning for traffic forecastingExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122766241:COnline publication date: 25-Jun-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122766

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents