Article

Automatic construction of nonparametric relational regression models for multiple time series

Authors:

Yunseong Hwang,

Jaesik ChoiAuthors Info & Claims

ICML'16: Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48

Pages 3030 - 3039

Published: 19 June 2016 Publication History

Abstract

Gaussian Processes (GPs) provide a general and analytically tractable way of modeling complex time-varying, nonparametric functions. The Automatic Bayesian Covariance Discovery (ABCD) system constructs natural-language description of time-series data by treating unknown time-series data nonparametrically using GP with a composite covariance kernel function. Unfortunately, learning a composite covariance kernel with a single time-series data set often results in less informative kernel that may not give qualitative, distinctive descriptions of data. We address this challenge by proposing two relational kernel learning methods which can model multiple time-series data sets by finding common, shared causes of changes. We show that the relational kernel learning methods find more accurate models for regression problems on several real-world data sets; US stock data, US house price index data and currency exchange rate data.

References

[1]

Álvarez, Mauricio A. and Lawrence, Neil D. Computationally efficient convolved multiple output gaussian processes. Journal of Machine Learning Research, 12: 1459-1500, 2011.

[2]

Bach, Francis R., Lanckriet, Gert R. G., and Jordan, Michael I. Multiple kernel learning, conic duality, and the SMO algorithm. In Proceedings of the Twenty-first International Conference on Machine Learning (ICML), 2004.

[3]

Belle, Vaishak, Passerini, Andrea, and Van den Broeck, Guy. Probabilistic inference in hybrid domains by weighted model integration. In Proceedings of 24th International Joint Conference on Artificial Intelligence (IJCAI), 2015.

[4]

Bing, Wu, Wen-qiong, Zhang, Ling, Chen, and Jia-hong, Liang. A gp-based kernel construction and optimization method for rvm. In Computer and Automation Engineering (ICCAE), volume 4, pp. 419-423, 2010.

[5]

Candela, Joaquin Quiñonero and Rasmussen, Carl Edward. A unifying view of sparse approximate gaussian process regression. Journal of Machine Learning Research, 6: 1939-1959, 2005.

[6]

Choi, Jaesik, Amir, Eyal, and Hill, David J. Lifted inference for relational continuous models. In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (UAI), pp. 126-134, 2010.

[7]

Choi, Jaesik, Amir, Eyal, Xu, Tianfang, and Valocchi, Albert J. Learning relational kalman filtering. In Proceedings of the Twenty-Ninth Conference on Artificial Intelligence (AAAI), pp. 2539-2546, 2015.

[8]

Chu, Wei, Sindhwani, Vikas, Ghahramani, Zoubin, and Keerthi, S. Sathiya. Relational learning with gaussian processes. In Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems (NIPS), pp. 289-296, 2006.

[9]

Diosan, Laura, Rogozan, Alexandrina, and Pécuchet, Jean-Pierre. Evolving kernel functions for svms by genetic programming. In The Sixth International Conference on Machine Learning and Applications (ICMLA), pp. 19- 24, 2007.

[10]

Duvenaud, David K., Lloyd, James Robert, Grosse, Roger B., Tenenbaum, Joshua B., and Ghahramani, Zoubin. Structure discovery in nonparametric regression through compositional kernel search. In Proceedings of the 30th International Conference on Machine Learning (ICML), pp. 1166-1174, 2013.

[11]

Getoor, Lise and Taskar, Ben. Introduction to statistical relational learning. MIT press, 2007.

[12]

Guarino, David and Blitzer, David. S&P/Case-Shiller home price tiered index levels. https://us:spindices:com/indices/realestate/sp-case-shiller-20-city-composite-home-price-index, 2015. Accessed: 2015-09-15.

[13]

Kimmig, Angelika, Mihalkova, Lilyana, and Getoor, Lise. Lifted graphical models: a survey. Machine Learning, 99(1):1-45, 2015.

[14]

Klenske, Edgar D., Zeilinger, Melanie Nicole, Schölkopf, Bernhard, and Hennig, Philipp. Nonparametric dynamics estimation for time periodic systems. In the 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 486-493, 2013.

[15]

Lloyd, James Robert. Gefcom2012 hierarchical load forecasting: Gradient boosting machines and gaussian processes. International Journal of Forecasting, 30(2):369-374, 2014.

[16]

Lloyd, James Robert, Duvenaud, David K., Grosse, Roger B., Tenenbaum, Joshua B., and Ghahramani, Zoubin. Automatic construction and natural-language description of nonparametric regression models. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI), pp. 1242-1250, 2014.

[17]

Pan, Jiazhu and Yao, Qiwei. Modelling multiple time series via common factors. Biometrika, 95(2):365-379, 2008.

[18]

Qiu, Huitong, Han, Fang, Liu, Han, and Caffo, Brian. Joint estimation of multiple graphical models from high dimensional time series. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 78(2):487- 504, 2016. ISSN 1467-9868.

[19]

Rasmussen, Carl Edward and Williams, Christopher K. I. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press, 2006.

[20]

Schwarz, Gideon. Estimating the dimension of a model. The annals of statistics, 6(2):461-464, 1978.

[21]

Silva, Ricardo, Chu, Wei, and Ghahramani, Zoubin. Hidden common cause relations in relational learning. In Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems (NIPS), pp. 1345-1352, 2008.

[22]

United States Census Bureau. Population estimates: Historical data. https://www:census:gov/popest/data/historical/index:html, 2014. Accessed: 2015-09-15.

[23]

von Alten, Tom. Top 100 market capitalization. http://fortboise:org/top100mktcap:html, 2001. Accessed: 2015-09-15.

[24]

Wang, Jue and Domingos, Pedro M. Hybrid markov logic networks. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI), pp. 1106-1111, 2008.

[25]

Wei, Xing, Sun, Jimeng, andWang, Xuerui. Dynamic mixture models for multiple time-series. In Proceedings of the 20nd International Joint Conference on Artificial Intelligence (IJCAI), pp. 2909-2914, 2007.

[26]

Wilson, Andrew Gordon and Adams, Ryan Prescott. Gaussian process kernels for pattern discovery and extrapolation. In Proceedings of the 30th International Conference on Machine Learning (ICML), pp. 1067-1075, 2013.

[27]

Wilson, Andrew Gordon and Ghahramani, Zoubin. Generalised wishart processes. In Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence (UAI), pp. 736-744, 2011.

[28]

Xu, Zhao, Kersting, Kristian, and Tresp, Volker. Multirelational learning with gaussian processes. In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI), pp. 1309-1314, 2009.

[29]

Yahoo Inc. Yahoo finance - business finance, stock market, quotes, news. http://finance:yahoo:com/, 2015. Accessed: 2016-01-20.

Cited By

Park YEom DSeo BChoi JBalch T(2020)Improved predictive deep temporal neural networks with trend filteringProceedings of the First ACM International Conference on AI in Finance10.1145/3383455.3422565(1-8)Online publication date: 15-Oct-2020
https://dl.acm.org/doi/10.1145/3383455.3422565

Automatic construction of nonparametric relational regression models for multiple time series
1. Computing methodologies

Recommendations

Time series forecasting by a seasonal support vector regression model

The support vector regression (SVR) model is a novel forecasting approach and has been successfully used to solve time series problems. However, the applications of SVR models in a seasonal time series forecasting has not been widely investigated. This ...
Combining seasonal ARIMA models with computational intelligence techniques for time series forecasting

Seasonal autoregressive integrated moving average (SARIMA) models form one of the most popular and widely used seasonal time series models over the past three decades. However, in several researches it has been argued that they have two basic ...
Nonparametric risk bounds for time-series forecasting

We derive generalization error bounds for traditional time-series forecasting models. Our results hold for many standard forecasting tools including autoregressive models, moving average models, and, more generally, linear state-space models. These ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

ICML'16: Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48

June 2016

3077 pages

Publisher

JMLR.org

Publication History

Published: 19 June 2016

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Park YEom DSeo BChoi JBalch T(2020)Improved predictive deep temporal neural networks with trend filteringProceedings of the First ACM International Conference on AI in Finance10.1145/3383455.3422565(1-8)Online publication date: 15-Oct-2020
https://dl.acm.org/doi/10.1145/3383455.3422565

View Options

View options

Media

Figures

Other

Tables

View Table of Contents