research-article

Supervised heterogeneous transfer learning using random forests

Authors:

Sanatan Sukhija,

Narayanan C Krishnan,

Deepak KumarAuthors Info & Claims

CODS-COMAD '18: Proceedings of the ACM India Joint International Conference on Data Science and Management of Data

Pages 157 - 166

https://doi.org/10.1145/3152494.3152510

Published: 11 January 2018 Publication History

Abstract

Supervised transfer learning algorithms utilize labeled data from auxiliary domains for learning in another domain where labeled data is scarce or absent. Given sufficient cross-domain corresponding instances, one can learn a robust transformation that maps the features across the domains by using any multi-output regression task. However, this cross-domain corresponding data is not available for real-world transfer tasks across heterogeneous feature spaces such as, cross-domain activity recognition and cross-lingual text/sentiment classification. In this paper, we present a shared label space driven algorithm that transfers labeled knowledge between heterogeneous feature spaces. The proposed algorithm treats the similar label distributions across the domains as pivots to generate cross-domain corresponding data. The shared label distributions and the corresponding data is obtained from the random forest models of the source and target domain. The experimental results on synthetic and real-world benchmark datasets having dissimilar modalities validate the performance of the proposed algorithm against state-of-the-art heterogeneous transfer learning approaches.

References

[1]

Massih Amini, Nicolas Usunier, and Cyril Goutte. 2009. Learning from multiple partially observed views-an application to multilingual text categorization. In Advances in neural information processing systems. 28--36.

Digital Library

[2]

John Blitzer, Ryan McDonald, and Fernando Pereira. 2006. Domain Adaptation with Structural Correspondence Learning. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. 120--128.

Digital Library

[3]

Hanen Borchani, Gherardo Varando, Concha Bielza, and Pedro Larrañaga. 2015. A survey on multi-output regression. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 5, 5 (2015), 216--233.

Digital Library

[4]

Leo Breiman. 2001. Random forests. Machine learning (2001), 5--32.

Digital Library

[5]

Rita Chattopadhyay, Qian Sun, Wei Fan, Ian Davidson, Sethuraman Panchanathan, and Jieping Ye. 2012. Multisource Domain Adaptation and Its Application to Early Detection of Fatigue. ACM Trans. Knowl. Discov. Data 6, 4, Article 18 (Dec. 2012), 26 pages.

Digital Library

[6]

Diane J. Cook, Aaron S. Crandall, Brian L. Thomas, and Narayanan C. Krishnan. 2013. CASAS: A Smart Home in a Box. Computer 46, 7 (2013), 62--69.

Digital Library

[7]

Diane J. Cook, Kyle Dillon Feuz, and Narayanan C. Krishnan. 2013. Transfer learning for activity recognition: a survey. Journal of Knowledge and Information Systems (2013), 537--556.

[8]

Lixin Duan, Dong Xu, and Ivor Tsang. 2012. Learning with augmented features for heterogeneous domain adaptation. arXiv preprint arXiv:1206.4660 (2012).

Digital Library

[9]

Bradley Efron, Trevor Hastie, Iain Johnstone, Robert Tibshirani, et al. 2004. Least angle regression. The Annals of statistics 32, 2 (2004), 407--499.

[10]

Kyle Dillon Feuz and Diane J. Cook. 2014. Heterogeneous transfer learning for activity recognition using heuristic search techniques. International Journal of Pervasive Computing and Communications (2014), 393--418.

[11]

Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach. In Proceedings of the 28th International Conference on Machine Learning (ICML-11). 513--520.

Digital Library

[12]

Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2001. The Elements of Statistical Learning.

[13]

Chih-Wei Hsu and Chih-Jen Lin. 2002. A comparison of methods for multiclass support vector machines. IEEE transactions on Neural Networks 13, 2 (2002), 415--425.

Digital Library

[14]

Derek Hao Hu and Qiang Yang. 2011. Transfer Learning for Activity Recognition via Sensor Mapping. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence. 1962--1967.

Digital Library

[15]

Yao-Hung Hubert Tsai, Yi-Ren Yeh, and Yu-Chiang Frank Wang. 2016. Learning cross-domain landmarks for heterogeneous domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5081--5090.

[16]

Narayanan C. Krishnan and Diane J. Cook. 2014. Activity Recognition on Streaming Sensor Data. Journal of Pervasive and Mobile Computing (2014), 138--154.

Digital Library

[17]

Ken Lang. 1995. Newsweeder: Learning to filter netnews. In Proceedings of the Twelfth International Conference on Machine Learning. 331--339.

Digital Library

[18]

M. Lichman. 2013. UCI Machine Learning Repository. (2013). http://archive.ics.uci.edu/ml

[19]

Jianhua Lin. 1991. Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory (1991), 145--151.

Digital Library

[20]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111--3119.

Digital Library

[21]

Sinno Jialin Pan, Xiaochuan Ni, Jian-Tao Sun, Qiang Yang, and Zheng Chen. 2010. Cross-domain sentiment classification via spectral feature alignment. In Proceedings of the 19th international conference on World wide web. ACM, 751--760.

Digital Library

[22]

Sinno Jialin Pan, Ivor W. Tsang, James T. Kwok, and Qiang Yang. 2009. Domain Adaptation via Transfer Component Analysis. In Proceedings of the 21st International Joint Conference on Artificial Intelligence. 1187--1192.

Digital Library

[23]

Sinno Jialin Pan and Qiang Yang. 2010. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering (2010), 1345--1359.

Digital Library

[24]

Sinno Jialin Pan, Vincent Wenchen Zheng, Qiang Yang, and Derek Hao Hu. 2008. Transfer learning for wifi-based indoor localization. In Association for the advancement of artificial intelligence (AAAI) workshop. 6.

[25]

Vishal M Patel, Raghuraman Gopalan, Ruonan Li, and Rama Chellappa. 2015. Visual domain adaptation: A survey of recent advances. Signal Processing Magazine, IEEE 32, 3 (2015), 53--69.

[26]

Peter Prettenhofer and Benno Stein. 2010. Cross-language Text Classification Using Structural Correspondence Learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL '10). 1118--1127.

Digital Library

[27]

Suju Rajan and Joydeep Ghosh. 2004. An Empirical Comparison of Hierarchical vs. Two-Level Approaches to Multiclass Problems. In Multiple Classifier Systems. 283--292.

[28]

Parisa Rashidi and Diane J. Cook. 2010. Multi home transfer learning for resident activity discovery and recognition. In Proceedings of the International Workshop on Knowledge Discovery from Sensor Data. 56--63.

[29]

Xiaoxiao Shi and Philip Yu. 2012. Dimensionality Reduction on Heterogeneous Feature Space. In Proceedings of the 12th IEEE International Conference on Data Mining. 635--644.

Digital Library

[30]

Xiangbo Shu, Guo-Jun Qi, Jinhui Tang, and Jingdong Wang. 2015. Weakly-shared deep transfer networks for heterogeneous-domain knowledge propagation. In Proceedings of the 23rd ACM international conference on Multimedia. ACM, 35--44.

Digital Library

[31]

Sanatan Sukhija, Narayanan C Krishnan, and Gurkanwal Singh. 2016. Supervised Heterogeneous Domain Adaptation via Random Forests. In Proceedings of the 25th International Joint Conference on Artificial Intelligence. 193--200.

Digital Library

[32]

Shiliang Sun, Honglei Shi, and Yuanbin Wu. 2015. A survey of multi-source domain adaptation. Information Fusion 24 (2015), 84--92.

Digital Library

[33]

T. L. M. van Kasteren, G. Englebienne, and B. J. A. Kröse. 2010. Transferring Knowledge of Activity Recognition Across Sensor Networks. In Proceedings of the 8th International Conference on Pervasive Computing. 283--300.

Digital Library

[34]

Chang Wang and Sridhar Mahadevan. 2009. A General Framework for Manifold Alignment. In AAAI Fall Symposium: Manifold Learning and Its Applications.

[35]

Chang Wang and Sridhar Mahadevan. 2011. Heterogeneous domain adaptation using manifold alignment. In IJCAI Proceedings-International Joint Conference on Artificial Intelligence (IJCAI), Vol. 22. 1541.

Digital Library

[36]

Chang Wang and Sridhar Mahadevan. 2013. Manifold Alignment Preserving Global Geometry. In Proceedings of the twenty-third International Joint Conference on Artifical Intelligence (IJCAI).

Digital Library

[37]

Ying Wei, Yin Zhu, Cane Wing-ki Leung, Yangqiu Song, and Qiang Yang. 2016. Instilling social to physical: Co-regularized heterogeneous transfer learning. In Thirtieth AAAI Conference on Artificial Intelligence.

Digital Library

[38]

Min Xiao and Yuhong Guo. 2015. Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2015, Porto, Portugal, September 7-11, 2015, Proceedings, Part II. Springer International Publishing, Chapter Semi-supervised Subspace Co-Projection for Multi-class Heterogeneous Domain Adaptation, 525--540.

[39]

Yasuhisa Yoshida, Tsutomu Hirao, Tomoharu Iwata, Masaaki Nagata, and Yuji Matsumoto. 2011. Transfer learning for multiple-domain sentiment analysis - identifying domain dependent/independent word polarity. In Twenty-Fifth AAAI Conference on Artificial Intelligence.

Digital Library

[40]

Guangyou Zhou, Tingting He, Wensheng Wu, and Xiaohua Tony Hu. 2015. Linking Heterogeneous Input Features with Pivots for Domain Adaptation. In Proceedings of the 24th International Conference on Artificial Intelligence (IJCAI'15). AAAI Press, 1419--1425.

Digital Library

[41]

Guangyou Zhou, Zhao Zeng, Jimmy Xiangji Huang, and Tingting He. 2016. Transfer learning for cross-lingual sentiment classification with weakly shared deep neural networks. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 245--254.

Digital Library

[42]

Joey Tianyi Zhou, Sinno Jialin Pan, Ivor W. Tsang, and Yan Yan. 2014. Hybrid Heterogeneous Transfer Learning Through Deep Learning. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence. AAAI Press, 2213--2219.

Digital Library

[43]

Joey Tianyi Zhou, Ivor W. Tsang, Sinno Jialin Pan, and Mingkui Tan. 2014. Heterogeneous Domain Adaptation for Multiple classes. In Proceedings of the 17th International Conference on Artificial Intelligence and Statistics. 1095--1103.

[44]

Yin Zhu, Yuqiang Chen, Zhongqi Lu, Sinno Jialin Pan, Gui-Rong Xue, Yong Yu, and Qiang Yang. 2011. Heterogeneous Transfer Learning for Image Classification. In AAAI.

Digital Library

Cited By

Pirbonyeh MShayegan MSotudeh GShamshirband S(2022)Heterogeneous domain adaptation by Features Normalization and Data Topology PreservingKnowledge-Based Systems10.1016/j.knosys.2022.109536257:COnline publication date: 5-Dec-2022
https://dl.acm.org/doi/10.1016/j.knosys.2022.109536
Ashmore RCalinescu RPaterson C(2021)Assuring the Machine Learning LifecycleACM Computing Surveys10.1145/345344454:5(1-39)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3453444
Sukhija SKrishnan N(2020)Shallow Domain AdaptationDomain Adaptation in Computer Vision with Deep Learning10.1007/978-3-030-45529-3_2(23-40)Online publication date: 19-Aug-2020
https://doi.org/10.1007/978-3-030-45529-3_2
Show More Cited By

Recommendations

Label space driven feature space remapping
CODS-COMAD '18: Proceedings of the ACM India Joint International Conference on Data Science and Management of Data

The elicitation of labeled training data from physical sources is a primary bottleneck that limits the applicability of traditional supervised learning algorithms. Transfer learning algorithms overcome the limitation for a target domain where training ...
A robust semi-supervised classification method for transfer learning
CIKM '10: Proceedings of the 19th ACM international conference on Information and knowledge management

The transfer learning problem of designing good classifiers with a high generalization ability by using labeled samples whose distribution is different from that of test samples is an important and challenging research issue in the fields of machine ...
Spectral domain-transfer learning
KDD '08: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining

Traditional spectral classification has been proved to be effective in dealing with both labeled and unlabeled data when these data are from the same domain. In many real world applications, however, we wish to make use of the labeled data from one ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CODS-COMAD '18: Proceedings of the ACM India Joint International Conference on Data Science and Management of Data

January 2018

379 pages

ISBN:9781450363419

DOI:10.1145/3152494

Conference Chair:
Sayan Ranu
IIT Delhi
,
General Chairs:
Niloy Ganguly
IIT Kharagpur
,
Raghu Ramakrishnan
Microsoft
,
Program Chairs:
Sunita Sarawagi
IIT Bombay
,
Shourya Roy
American Express Big Data Labs

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 January 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Department of Science and Technology, India

Conference

CoDS-COMAD '18

CoDS-COMAD '18: The ACM India Joint International Conference on Data Science & Management of Data

January 11 - 13, 2018

Goa, India

Acceptance Rates

CODS-COMAD '18 Paper Acceptance Rate 50 of 150 submissions, 33%;

Overall Acceptance Rate 197 of 680 submissions, 29%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
414
Total Downloads

Downloads (Last 12 months)31
Downloads (Last 6 weeks)3

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Pirbonyeh MShayegan MSotudeh GShamshirband S(2022)Heterogeneous domain adaptation by Features Normalization and Data Topology PreservingKnowledge-Based Systems10.1016/j.knosys.2022.109536257:COnline publication date: 5-Dec-2022
https://dl.acm.org/doi/10.1016/j.knosys.2022.109536
Ashmore RCalinescu RPaterson C(2021)Assuring the Machine Learning LifecycleACM Computing Surveys10.1145/345344454:5(1-39)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3453444
Sukhija SKrishnan N(2020)Shallow Domain AdaptationDomain Adaptation in Computer Vision with Deep Learning10.1007/978-3-030-45529-3_2(23-40)Online publication date: 19-Aug-2020
https://doi.org/10.1007/978-3-030-45529-3_2
Venkateswara HPanchanathan S(2020)Introduction to Domain AdaptationDomain Adaptation in Computer Vision with Deep Learning10.1007/978-3-030-45529-3_1(3-21)Online publication date: 19-Aug-2020
https://doi.org/10.1007/978-3-030-45529-3_1
Sukhija SKrishnan N(2019)Web-Induced Heterogeneous Transfer Learning with Sample SelectionMachine Learning and Knowledge Discovery in Databases10.1007/978-3-030-10928-8_46(777-793)Online publication date: 23-Jan-2019
https://doi.org/10.1007/978-3-030-10928-8_46
Sukhija SRanu SGanguly NRamakrishnan RSarawagi SRoy S(2018)Label space driven feature space remappingProceedings of the ACM India Joint International Conference on Data Science and Management of Data10.1145/3152494.3167977(310-313)Online publication date: 11-Jan-2018
https://dl.acm.org/doi/10.1145/3152494.3167977

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents