research-article

Hybrid Genetic Algorithms for Feature Selection

Authors:

Byung-Ro MoonAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 26, Issue 11

Pages 1424 - 1437

https://doi.org/10.1109/TPAMI.2004.105

Published: 01 November 2004 Publication History

Abstract

This paper proposes a novel hybrid genetic algorithm for feature selection. Local search operations are devised and embedded in hybrid GAs to fine-tune the search. The operations are parameterized in terms of their fine-tuning power, and their effectiveness and timing requirements are analyzed and compared. The hybridization technique produces two desirable effects: a significant improvement in the final performance and the acquisition of subset-size control. The hybrid GAs showed better convergence properties compared to the classical GAs. A method of performing rigorous timing analysis was developed, in order to compare the timing requirement of the conventional and the proposed algorithms. Experiments performed with various standard data sets revealed that the proposed hybrid GA is superior to both a simple GA and sequential search algorithms.

References

[1]

F.Z. Brill D.E. Brown and W.N. Martin, “Fast Genetic Selection of Features for Neural Network Classifiers,” IEEE Trans. Neural Networks, vol. 3, no. 2, pp. 324-328, Mar. 1992.

Digital Library

[2]

T.N. Bui and B.R. Moon, “Genetic Algorithm and Graph Partitioning,” IEEE Trans. Computers, vol. 45, no. 7, pp. 841-855, July 1996.

Digital Library

[3]

M. Dash and H. Liu, “Feature Selection for Classification,” Intelligent Data Analysis, vol. 1, no. 3, pp. 131-156, 1997.

[4]

L. Davis, Handbook of Genetic Algorithms. Van Nostrand Reinhold, 1991.

[5]

F.J. Ferri P. Pudil M. Hatef and J. Kittler, “Comparative Study of Techniques for Large-Scale Feature Selection,” Pattern Recognition in Practice IV, E.S. Gelsema and L.N. Kanal, eds., pp. 403-413, 1994.

[6]

J. Holland, Adaptation in Nature and Artificial Systems. MIT Press, 1992.

Digital Library

[7]

C.-C. Hung A. Fahsi W. Tadesse and T. Coleman, “A Comparative Study of Remotely Sensed Data Classification Using Principal Components Analysis and Divergence,” Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, pp. 2444-2449, 1997.

[8]

H. Ishikawa, “Multiscale Feature Selection in Stereo,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 132-137, 1999.

[9]

A. Jain and D. Zongker, “Feature Selection: Evaluation, Application, and Small Sample Performance,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 2, pp. 153-158, Feb. 1997.

Digital Library

[10]

P. Jog J. Suh and D. Gucht, “The Effect of Population Size, Heuristic Crossover and Local Improvement on a Genetic Algorithm for the Traveling Salesman Problem,” Proc. Int'l Conf. Genetic Algorithms, pp. 110-115, 1989.

Digital Library

[11]

J. Kittler, “Feature Selection and Extraction,” Handbook of Pattern Recognition and Image Processing, T.Y. Young and K.S. Fu, eds., pp.nbsp59-83, 1986.

[12]

M. Kudo and J. Sklansky, “Comparison of Algorithms that Select Features for Pattern Recognition,” Pattern Recognition, vol. 33,no. 1, pp. 25-41, 2000.

[13]

L.I. Kuncheva and L.C. Jain, “Nearest Neighbor Classifier: Simultaneous Editing and Feature Selection,” Pattern Recognition Letters, vol. 20, pp. 1149-1156, 1999.

Digital Library

[14]

P. Langley, “Selection of Relevant Features in Machine Learning,” Proc. AAAI Fall Symp. Relevance, pp. 1-5, 1994.

[15]

Principles of Visual Information Retrieval. M.S. Lew, ed., Springer, 2001.

[16]

Y. Liu and F. Dellaert, “A Classification Based Similarity Metric for 3D Image Retrieval,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 800-805, 1998.

Digital Library

[17]

M.J. Martin-Bautista and M.-A. Vila, “A Survey of Genetic Feature Selection in Mining Issues,” Proc. 1999 Congress on Evolutionary Computation (CEC '99), pp. 1314-1321, July 1999.

[18]

K. Messer and J. Kittler, “Using Feature Selection to Aid an Iconic Search through an Image Database,” Proc. EEE Int'l Conf. Acoustics, Speech, and Signal processing (ICASSP), vol. 4, pp. 2605-2608, 1997.

Digital Library

[19]

P.M. Murphy and D.W. Aha, “UCI Repository for Machine Learning Databases,” technical report, Dept. of Information and Computer Science, Univ. of California, Irvine, Calif., 1994, http://www.ics.uci.edu/mlearn/MLRepository.html.

[20]

P.M. Narendra and K. Fukunaga, “A Branch and Bound Algorithm for Feature Subset Selection,” IEEE Trans. Computers, vol. 26, no. 9, pp. 917-922, Sept. 1977.

Digital Library

[21]

I.-S. Oh and C.Y. Suen, “Distance Features for Neural Network-Based Recognition of Handwritten Characters,” Int'l J. Document Analysis and Recognition, vol. 1, no. 2, pp. 73-88, 1998.

[22]

I.-S. Oh J.-S. Lee and C.Y. Suen, “Analysis of Class Separation and Combination of Class-Dependent Features for Handwriting Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 10, pp. 1089-1094, Oct. 1999.

Digital Library

[23]

I.-S. Oh J.-S. Lee and B.-R. Moon, “Local Search-Embedded Genetic Algorithms for Feature Selection,” Proc. Int'l Conf. Pattern Recognition, 2002.

[24]

S. Piramuthu, “Evaluating Feature Selection Methods for Learning in Data Mining Applications,” Proc. 31st Ann. Hawaii Int'l Conf. System Science, pp. 294-301, 1998.

Digital Library

[25]

P. Pudil J. Novovicova and J. Kittler, “Floating Search Methods in Feature Selection,” Pattern Recognition Letters, vol. 15, pp. 1119-1125, 1994.

Digital Library

[26]

S. Puuronen A. Tsymbal and I. Skrypnik, “Advanced Local Feature Selection in Medical Diagnostics,” Proc. 13th IEEE Symp. Computer-Based Medical Systems, pp. 25-30, 2000.

Digital Library

[27]

M.L. Raymer W.F. Punch E.D. Goodman L.A. Kuhn and A.K. Jain, “Dimensionality Reduction Using Genetic Algorithms,” IEEE Trans. Evolutionary Computation, vol. 4, no. 2, pp. 164-171, July 2000.

Digital Library

[28]

W. Siedlecki and J. Sklansky, “On Automatic Feature Selection,” Int'l J. Pattern Recognition and Artificial Intelligence, vol. 2, no. 2, pp.nbsp197-220, 1988.

[29]

W. Siedlecki and J. Sklansky, “A Note on Genetic Algorithms for Large-Scale Feature Selection,” Pattern Recognition Letters, vol. 10, pp. 335-347, 1989.

Digital Library

[30]

J.H. Yang and V. Honavar, “Feature Subset Selection Using a Genetic Algorithm,” IEEE Intelligent Systems, vol. 13, no. 2, pp. 44-49, 1998.

Digital Library

[31]

B. Yu and B. Yuan, “A More Efficient Branch and Bound Algorithm for Feature Selection,” Pattern Recognition, vol. 26,no. 6, pp. 883-889, 1993.

[32]

X. Zheng B.A. Julstrom and W. Cheng, “Design of Vector Quantization Codebooks Using a Genetic Algorithm,” Proc. IEEE Int'l Conf. Evolutionary Computation, pp. 525-529, 1997.

Cited By

Liu ZWang ASun GLi JBao HLiu Y(2024)Evolutionary feature selection based on hybrid bald eagle search and particle swarm optimizationIntelligent Data Analysis10.3233/IDA-22722228:1(121-159)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.3233/IDA-227222
Saadatmand HAkbarzadeh-T M(2024)Many-Objective Jaccard-Based Evolutionary Feature Selection for High-Dimensional Imbalanced Data ClassificationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.341619646:12(8820-8835)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TPAMI.2024.3416196
Wang XShangguan HHuang FWu SJia W(2024)MEL: Efficient Multi-Task Evolutionary Learning for High-Dimensional Feature SelectionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.336633336:8(4020-4033)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TKDE.2024.3366333
Show More Cited By

Index Terms

Hybrid Genetic Algorithms for Feature Selection
1. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies
      1. Heuristic function construction
  2. Machine learning
    1. Machine learning algorithms
      1. Feature selection
2. Theory of computation
  1. Design and analysis of algorithms
    1. Data structures design and analysis
      1. Sorting and searching

Recommendations

Classifier ensemble selection using hybrid genetic algorithms

This paper proposes a hybrid genetic algorithm for classifier ensemble selection. In this paper, two local search operations used to improve offspring prior to replacement are proposed. The operations are parameterized in order to control the ...
A hybrid genetic algorithm for feature subset selection in rough set theory

Rough set theory has been proven to be an effective tool to feature subset selection. Current research usually employ hill-climbing as search strategy to select feature subset. However, they are inadequate to find the optimal feature subset since no ...
A new local search based hybrid genetic algorithm for feature selection

This paper presents a new hybrid genetic algorithm (HGA) for feature selection (FS), called as HGAFS. The vital aspect of this algorithm is the selection of salient feature subset within a reduced size. HGAFS incorporates a new local search operation ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 26, Issue 11

November 2004

144 pages

ISSN:0162-8828

Issue’s Table of Contents

Copyright © 2004.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 November 2004

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

203
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu ZWang ASun GLi JBao HLiu Y(2024)Evolutionary feature selection based on hybrid bald eagle search and particle swarm optimizationIntelligent Data Analysis10.3233/IDA-22722228:1(121-159)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.3233/IDA-227222
Saadatmand HAkbarzadeh-T M(2024)Many-Objective Jaccard-Based Evolutionary Feature Selection for High-Dimensional Imbalanced Data ClassificationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.341619646:12(8820-8835)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TPAMI.2024.3416196
Wang XShangguan HHuang FWu SJia W(2024)MEL: Efficient Multi-Task Evolutionary Learning for High-Dimensional Feature SelectionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2024.336633336:8(4020-4033)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TKDE.2024.3366333
Nguyen BXue BZhang M(2024)A Constrained Competitive Swarm Optimizer With an SVM-Based Surrogate Model for Feature SelectionIEEE Transactions on Evolutionary Computation10.1109/TEVC.2022.319742728:1(2-16)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1109/TEVC.2022.3197427
Cheng FZhang RHuang ZQiu JXia MZhang L(2024)An Objective Space Constraint-Based Evolutionary Method for High-Dimensional Feature Selection [Research Frontier]IEEE Computational Intelligence Magazine10.1109/MCI.2024.336442919:2(113-128)Online publication date: 8-Apr-2024
https://dl.acm.org/doi/10.1109/MCI.2024.3364429
Chen YDing WJu HHuang JYin T(2024)A distributed attribute reduction based on neighborhood evidential conflict with Apache SparkInformation Sciences: an International Journal10.1016/j.ins.2024.120521668:COnline publication date: 9-Jul-2024
https://dl.acm.org/doi/10.1016/j.ins.2024.120521
Moslemi AAhmadian A(2024)Subspace learning for feature selection via rank revealing QR factorizationExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.124919256:COnline publication date: 5-Dec-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.124919
Moslemi A(2024)A tutorial-based survey on feature selectionEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107136126:PDOnline publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107136
Al-Eiadeh MAbdallah M(2024) GeniGraphComputers and Security10.1016/j.cose.2024.103927144:COnline publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1016/j.cose.2024.103927
Ma HLi MLv SWang LDeng S(2024)Roulette wheel-based level learning evolutionary algorithm for feature selection of high-dimensional dataApplied Soft Computing10.1016/j.asoc.2024.111948163:COnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.asoc.2024.111948
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents