skip to main content
research-article

Roulette wheel-based level learning evolutionary algorithm for feature selection of high-dimensional data

Published: 01 September 2024 Publication History

Abstract

Feature selection in high-dimensional data is a large-scale sparse and discrete optimization problem. Most evolutionary algorithms are designed to tackle continuous optimization problems. However, when dealing with high-dimensional feature selection tasks, they often suffer from poor population diversity and are computationally expensive. To address these challenges, this work introduces a roulette wheel-based level learning evolutionary algorithm (RWLLEA). RWLLEA integrates two key components. Firstly, it employs a leveled population mode. Individuals from higher levels provide guidance to those at lower levels during the evolutionary process, thereby exploring the potential combinatorial effects among features. Secondly, recognizing the characteristics associated with the high-dimensional feature selection task, a roulette wheel-based update method is devised to dynamically reduce search space and harmonize the algorithm's exploitation and exploration capacities across different stages. The performance of the proposed method was evaluated by comparing it with six other feature selection techniques across a range of fifteen diverse datasets. The experimental findings demonstrate that the proposed method can achieve a reduced feature set with a shorter runtime and exhibit superior classification accuracy.

Highlights

A roulette wheel-based level learning evolutionary algorithm is introduced for the high-dimensional feature selection.
The population is divided into levels, with individuals at the higher levels guiding the lower levels for updating.
A roulette wheel-based update method is designed for the high-dimensional feature selection task.

References

[1]
B. Tang, M. Shepherd, E. Milios, M.I. Heywood, Comparing and combining dimension reduction techniques for efficient text clustering (2005) 17–26.
[2]
S. Ng, Principal component analysis to reduce dimension on digital image, Procedia Comput. Sci. 111 (2017) 113–119.
[3]
C.C. Aggarwal, P.S. Yu, Finding generalized projected clusters in high dimensional spaces (2000) 70–81.
[4]
S. Ayesha, M.K. Hanif, R. Talib, Overview and comparative study of dimensionality reduction techniques for high dimensional data, Inf. Fusion 59 (2020) 44–58.
[5]
X.-F. Song, Y. Zhang, D.-W. Gong, X.-Z. Gao, A fast hybrid feature selection based on correlation-guided clustering and particle swarm optimization for high-dimensional data, IEEE Trans. Cybern. 52 (2021) 9573–9586.
[6]
I. Guyon, A. Elisseeff, An introduction to variable and feature selection, J. Mach. Learn. Res. 3 (2003) 1157–1182.
[7]
G. Chandrashekar, F. Sahin, A survey on feature selection methods, Comput. Electr. Eng. 40 (2014) 16–28.
[8]
Y. Saeys, I. Inza, P. Larranaga, A review of feature selection techniques in bioinformatics, Bioinformatics 23 (2007) 2507–2517.
[9]
M. Robnik-Šikonja, I. Kononenko, Theoretical and empirical analysis of ReliefF and RReliefF, Mach. Learn. 53 (2003) 23–69.
[10]
C.E. Shannon, A mathematical theory of communication, Bell Syst. Tech. J. 27 (1948) 379–423.
[11]
I. Guyon, J. Weston, S. Barnhill, V. Vapnik, Gene selection for cancer classification using support vector machines, Mach. Learn. 46 (2002) 389–422.
[12]
M. Sandri, P. Zuccolotto, Variable selection using random forests, Data Analysis, Classification and the Forward Search, Springer, 2006, pp. 263–270.
[13]
A. Got, A. Moussaoui, D. Zouache, Hybrid filter-wrapper feature selection using whale optimization algorithm: a multi-objective approach, Expert Syst. Appl. 183 (2021).
[14]
L. Ke, M. Li, L. Wang, S. Deng, J. Ye, X. Yu, Improved swarm-optimization-based filter-wrapper gene selection from microarray data for gene expression tumor classification, Pattern Anal. Appl. 26 (2023) 455–472.
[15]
H. Pan, S. Chen, H. Xiong, A high-dimensional feature selection method based on modified Gray Wolf Optimization, Appl. Soft Comput. 135 (2023).
[16]
H.-H. Hsu, C.-W. Hsieh, M.-D. Lu, Hybrid feature selection by combining filters and wrappers, Expert Syst. Appl. 38 (2011) 8144–8150.
[17]
C.-M. Lai, Multi-objective simplified swarm optimization with weighting scheme for gene selection, Appl. Soft Comput. 65 (2018) 58–68.
[18]
M. Li, L. Ke, L. Wang, S. Deng, X. Yu, A novel hybrid gene selection for tumor identification by combining multifilter integration and a recursive flower pollination search algorithm, Knowl. - Based Syst. 262 (2023).
[19]
H. Lu, J. Chen, K. Yan, Q. Jin, Y. Xue, Z. Gao, A hybrid feature selection algorithm for gene expression data classification, Neurocomputing 256 (2017) 56–62.
[20]
M. Li, Y. Zhao, R. Cao, J. Wang, D. Wu, A recursive framework for improving the performance of multi-objective differential evolution algorithms for gene selection, Swarm Evolut. Comput. 87 (2024).
[21]
M. Li, Y. Zhao, M. Lou, S. Deng, L. Wang, A multitasking multi-objective differential evolution gene selection algorithm enhanced with new elite and guidance strategies for tumor identification, Expert Syst. Appl. (2023).
[22]
M. Li, R. Cao, Y. Zhao, Y. Li, S. Deng, Population characteristic exploitation-based multi-orientation multi-objective gene selection for microarray data classification, Comput. Biol. Med. (2024).
[23]
G. Hu, B. Du, X. Wang, G. Wei, An enhanced black widow optimization algorithm for feature selection, Knowl. -Based Syst. 235 (2022).
[24]
B. Nouri-Moghaddam, M. Ghazanfari, M. Fathian, A novel multi-objective forest optimization algorithm for wrapper feature selection, Expert Syst. Appl. 175 (2021).
[25]
I. Chatterjee, M. Ghosh, P.K. Singh, R. Sarkar, M. Nasipuri, A clustering-based feature selection framework for handwritten Indic script classification, Expert Syst. 36 (2019).
[26]
Y. Jin, Surrogate-assisted evolutionary computation: recent advances and future challenges, Swarm Evolut. Comput. 1 (2011) 61–70.
[27]
K. Deb, Multi-objective optimization, Search Method. Search Method. 2014 (2014) 403–449.
[28]
S. Liu, H. Wang, W. Peng, W. Yao, A surrogate-assisted evolutionary feature selection algorithm with parallel random grouping for high-dimensional classification, IEEE Trans. Evolut. Comput. 26 (2022) 1087–1101.
[29]
B. Xue, M. Zhang, W.N. Browne, Particle swarm optimization for feature selection in classification: a multi-objective approach, IEEE Trans. Cybern. 43 (2012) 1656–1671.
[30]
K. Deb, A. Pratap, S. Agarwal, T. Meyarivan, A fast and elitist multiobjective genetic algorithm: NSGA-II, IEEE Trans. Evolut. Comput. 6 (2002) 182–197.
[31]
D. Karaboga, B. Basturk, On the performance of artificial bee colony (ABC) algorithm, Appl. Soft Comput. 8 (2008) 687–697.
[32]
J. Kennedy, R. Eberhart, Particle swarm optimization, in: Particle swarm optimization, IEEE, 1995, pp. 1942–1948.
[33]
A.-D. Li, B. Xue, M. Zhang, Improved binary particle swarm optimization for feature selection with new initialization and search space reduction strategies, Appl. Soft Comput. 106 (2021).
[34]
J. Luo, D. Zhou, L. Jiang, H. Ma, A particle swarm optimization based multiobjective memetic algorithm for high-dimensional feature selection, Memetic Comput. 14 (2022) 77–93.
[35]
J.H. Holland, Genetic algorithms, Sci. Am. 267 (1992) 66–73.
[36]
I.-S. Oh, J.-S. Lee, B.-R. Moon, Hybrid genetic algorithms for feature selection, IEEE Trans. Pattern Anal. Mach. Intell. 26 (2004) 1424–1437.
[37]
F. Tan, X. Fu, Y. Zhang, A.G. Bourgeois, A genetic algorithm-based method for feature subset selection, Soft Comput. 12 (2008) 111–120.
[38]
S. Sayed, M. Nassef, A. Badr, I. Farag, A nested genetic algorithm for feature selection in high-dimensional cancer microarray datasets, Expert Syst. Appl. 121 (2019) 233–243.
[39]
R. Storn, K. Price, Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces, J. Glob. Optim. 11 (1997) 341–359.
[40]
B. Turkoglu, S.A. Uymaz, E. Kaya, Binary artificial algae algorithm for feature selection, Appl. Soft Comput. 120 (2022).
[41]
J.-S. Pan, N. Liu, S.-C. Chu, A competitive mechanism based multi-objective differential evolution algorithm and its application in feature selection, Knowl. - Based Syst. 245 (2022).
[42]
M. Li, H. Ma, S. Lv, L. Wang, S. Deng, Enhanced NSGA-II-based feature selection method for high-dimensional classification, Inf. Sci. (2024).
[43]
T. Li, J. Shi, W. Deng, Z. Hu, Pyramid particle swarm optimization with novel strategies of competition and cooperation, Appl. Soft Comput. 121 (2022).
[44]
Q. Yang, W.-N. Chen, J.Da Deng, Y. Li, T. Gu, J. Zhang, A level-based learning swarm optimizer for large-scale optimization, IEEE Trans. Evolut. Comput. 22 (2017) 578–594.
[45]
B. Xue, M. Zhang, W.N. Browne, Multi-Object. Part. swarm Optim. (PSO) Feature Sel. (2012) 81–88.
[46]
P. Wang, B. Xue, M. Zhang, J. Liang, A grid-dominance based multi-objective algorithm for feature selection in classification, IEEE, 2021, pp. 2053–2060.
[47]
P. Wang, B. Xue, J. Liang, M. Zhang, Differential evolution-based feature selection: a niching-based multiobjective approach, IEEE Trans. Evolut. Comput. 27 (2022) 296–310.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Applied Soft Computing
Applied Soft Computing  Volume 163, Issue C
Sep 2024
1292 pages

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 September 2024

Author Tags

  1. Feature selection
  2. Evolutionary algorithm
  3. High-dimensional
  4. Classification

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 29 Jan 2025

Other Metrics

Citations

View Options

View options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media