article

Double-bootstrapping source data selection for instance-based transfer learning

Authors:

Jian ZhangAuthors Info & Claims

Pattern Recognition Letters, Volume 34, Issue 11

Pages 1279 - 1285

https://doi.org/10.1016/j.patrec.2013.04.012

Published: 01 August 2013 Publication History

Abstract

Instance-based transfer is an important paradigm for transfer learning, where data from related tasks (source data) are combined with the data for the current learning task (target data) to train a learner for the current (target) task. However, in most application scenarios, the benefit of the source data is unclear. The source may contain both helpful and harmful instances to the target learning. Simply combining the source with the target data may result in performance deterioration (negative transfer). Selecting the instances from the source data that will benefit the target task is a key step for instance-based transfer learning. Most existing instance-based transfer methods lack such selection or mix source selection with the training for the target task. This leads to problems as the training may use source data harmful to the target. We propose a simple yet effective method for instance-based transfer learning in environments where the usefulness of the sources are unclear. The method employs a double-selection process, based on bootstrapping, to reduce the impact of irrelevant/harmful data in the source. Experiment results show that in most cases, our method produces more improvements through transfer than TrBagg (Kamishima et al., 2009) and TrAdaBoost (Dai et al., 2009). Our method can also deal with a wider range of transfer learning scenarios.

References

[1]

Kamishima, T., Hamasaki, M., Akaho, S., 2009. Trbagg: A simple transfer learning method and its application to personalization in collaborative tagging. In: Proc. of IEEE International Conference on Data Mining, pp. 219-228.

[2]

Dai, W., Yang, Q., Xue, G.-R., Yu, Y., 2009. Boosting for transfer learning. In: Proc. of International Conference on Machine Learning, pp. 193-200.

[3]

Thrun, S., 1995. Is learning the nth thing any easier than learning the first?. In: Proc. of Ann. Conf. Neural Information Processing Systems, pp. 640-646.

[4]

Multitask learning. Machine Learning. v28 i1. 41-75.

[5]

Jiang, J., Zhai, C., 2007. Instance weighting for domain adaptation in nlp. In: Proc. of Ann. Conf. for the Assoc. Computational Linguistics, pp. 264-271.

[6]

Huang, J., Smola, A., Gretton, A., Borgwardt, K., 2007. Correcting sample selection bias by unlabeled data. In: Proc. Ann. Conf. Neural Information Processing Systems, pp. 601-608.

[7]

A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences. v55 i1. 119-139.

[8]

A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering. v22 i10. 1345-1359.

[9]

Raina, R., Battle, A., Lee, H., Packer, B., Ng, A.Y., 2007. Self-taught learning:transfer learning from unlabeled data. In: Proc. International Conference on, Machine Learning, pp. 759-766.

[10]

Ando, R., Zhang, T., 2005. A high-performance semi-supervised learning method for text chunking, In: Proc. Ann. Meeting on Assoc for Computational Linguistics, pp. 1-9.

[11]

Blitzer, J., McDonald, R., Pereira, F., 2006. Domain adaptation with structural correspondence learning. In: Proc. Conf. Empirical Methods in Natural, Language, pp. 120-128.

[12]

Lawrence, N., Platt, J., 2004. Learning to learn with the informative vector machine. In: Proc. International Conference on Machine Learning, pp. 65.

[13]

Bonilla, E., Chai, K., Williams, C., 2008. Multi-task gaussian process prediction. In: Proc. Ann. Conf. Neural Information Processing Systems, pp. 145-154.

[14]

Schwaighofer, A., Tresp, V., Yu, K., 2005. Learning gaussian process kernels via hierarchical bayes, In: Proc. Ann. Conf. Neural Information Processing Systems, 2005, pp. 1209-1216.

[15]

Evgeniou, T., Pontil, M., 2004. Regularized multi-task learning. In: Proc ACM SIGKDD Int'l Conf Knowledge Discovery and Data Mining, pp. 109-117.

[16]

Mihalkova, L., Huynh, T., Mooney, R., 2007. Mapping and revising markov logic networks for transfer learning. In: Proc. Assoc. for the Advancement of Artificial Intelligence (AAAI) Conf, pp. 608-614.

[17]

Mihalkova, L., Mooney, R., 2008. Transfer learning by mapping with minimal target data. In: Proc. Assoc. for the Advancement of Artificial Intelligence, Workshop Transfer Learning for Complex Tasks, pp. 1163-1168.

[18]

UCI. <http://archive.ics.uci.edu/ml/>

[19]

WEKA. <http://www.cs.waikato.ac.nz/ml/weka/>

Cited By

Wu CChen JFang QHe KZhao ZRen HXu GLiu YXiang Y(2024)Rethinking Membership Inference Attacks Against Transfer LearningIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.341359219(6441-6454)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3413592
Luo WFeng PZhang JYu DWu Z(2022)A novel deep convolution generative adversarial transfer learning model for data-driven assembly quality diagnosisJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-21186042:3(2727-2741)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.3233/JIFS-211860
Chen SXu ZWang XZhang C(2022)Ambient air pollutants concentration prediction during the COVID-19Knowledge-Based Systems10.1016/j.knosys.2022.109996258:COnline publication date: 22-Dec-2022
https://dl.acm.org/doi/10.1016/j.knosys.2022.109996
Show More Cited By

Double-bootstrapping source data selection for instance-based transfer learning
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning

Recommendations

Learning to Select Instance: Simultaneous Transfer Learning and Clustering
SIGIR '21: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

Transfer learning leverages knowledge from a source domain with rich data to a target domain with sparse data. However, the difference between the source and target data distribution weakens the transferability. To bridge this gap, we focus on selecting ...
A Minimax Game for Instance based Selective Transfer Learning
KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Deep neural network based transfer learning has been widely used to leverage information from the domain with rich data to help domain with insufficient data. When the source data distribution is different from the target data, transferring knowledge ...
Multi-source transfer learning with multi-view adaboost
ICONIP'12: Proceedings of the 19th international conference on Neural Information Processing - Volume Part III

Transfer learning, which is one of the most important research directions in machine learning, has been studied in various fields in recent years. In this paper, we combine the theories of multi-source and multi-view learning into transfer learning and ...

Comments

Information & Contributors

Information

Published In

cover image Pattern Recognition Letters

Pattern Recognition Letters Volume 34, Issue 11

August, 2013

109 pages

ISSN:0167-8655

Issue’s Table of Contents

Copyright © Elsevier B.V. © 2013.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 August 2013

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wu CChen JFang QHe KZhao ZRen HXu GLiu YXiang Y(2024)Rethinking Membership Inference Attacks Against Transfer LearningIEEE Transactions on Information Forensics and Security10.1109/TIFS.2024.341359219(6441-6454)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIFS.2024.3413592
Luo WFeng PZhang JYu DWu Z(2022)A novel deep convolution generative adversarial transfer learning model for data-driven assembly quality diagnosisJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-21186042:3(2727-2741)Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.3233/JIFS-211860
Chen SXu ZWang XZhang C(2022)Ambient air pollutants concentration prediction during the COVID-19Knowledge-Based Systems10.1016/j.knosys.2022.109996258:COnline publication date: 22-Dec-2022
https://dl.acm.org/doi/10.1016/j.knosys.2022.109996
Deng CTang JMiao JZhao YChen XLu S(2022)Efficient stability prediction of milling process with arbitrary tool-holder combinations based on transfer learningJournal of Intelligent Manufacturing10.1007/s10845-022-01912-534:5(2263-2279)Online publication date: 14-Feb-2022
https://dl.acm.org/doi/10.1007/s10845-022-01912-5
Qin XChen YWang JYu C(2020)Cross-Dataset Activity Recognition via Adaptive Spatial-Temporal Transfer LearningProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33698183:4(1-25)Online publication date: 14-Sep-2020
https://dl.acm.org/doi/10.1145/3369818
Zhang YXu JMiao GChen YZhang Y(2018)Addressing Domain Adaptation for Chinese Word Segmentation with Instances-Based Transfer LearningChinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data10.1007/978-3-030-01716-3_3(24-36)Online publication date: 19-Oct-2018
https://dl.acm.org/doi/10.1007/978-3-030-01716-3_3
Zhou SSmirnov ESchoenmakers GDriessens KPeeters R(2017)Testing exchangeability for transfer decisionPattern Recognition Letters10.1016/j.patrec.2016.12.02188:C(64-71)Online publication date: 1-Mar-2017
https://dl.acm.org/doi/10.1016/j.patrec.2016.12.021
Zhou SSmirnov ESchoenmakers GPeeters R(2017)Conformal decision-tree approach to instance transferAnnals of Mathematics and Artificial Intelligence10.1007/s10472-017-9554-x81:1-2(85-104)Online publication date: 1-Oct-2017
https://dl.acm.org/doi/10.1007/s10472-017-9554-x
Inoue SPan XHara TShigeno H(2016)Supervised and Unsupervised Transfer Learning for Activity Recognition from Simple In-home SensorsProceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services10.1145/2994374.2994400(20-27)Online publication date: 28-Nov-2016
https://dl.acm.org/doi/10.1145/2994374.2994400
Zhao SCao QChen JZhang YTang JDuan Z(2016)A multi-ATL method for transfer learning across multiple domains with arbitrarily different distributionKnowledge-Based Systems10.1016/j.knosys.2015.10.03294:C(60-69)Online publication date: 15-Feb-2016
https://dl.acm.org/doi/10.1016/j.knosys.2015.10.032
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents