article

Generalized Discriminant Analysis Using a Kernel Approach

Authors:

F. AnouarAuthors Info & Claims

Neural Computation, Volume 12, Issue 10

Pages 2385 - 2404

https://doi.org/10.1162/089976600300014980

Published: 01 October 2000 Publication History

Abstract

We present a new method that we call generalized discriminant analysis (GDA) to deal with nonlinear discriminant analysis using kernel function operator. The underlying theory is close to the support vector machines (SVM) insofar as the GDA method provides a mapping of the input vectors into high-dimensional feature space. In the transformed space, linear properties make it easy to extend and generalize the classical linear discriminant analysis (LDA) to nonlinear discriminant analysis. The formulation is expressed as an eigenvalue problem resolution. Using a different kernel, one can cover a wide class of nonlinearities. For both simulated data and alternate kernels, we give classification results, as well as the shape of the decision function. The results are confirmed using real data to perform seed classification.

References

[1]

Aizerman, M. A., Braverman, E. M., & Rozonoér, L. I. (1964). Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control, 25, 821-837.]]

[2]

Anouar, F., Badran, F., & Thiria, S. (1998). Probabilistic self organizing map and radial basis function. Journal Neurocomputing, 20, 83-96.]]

[3]

Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In D. Haussler (Ed.), Fifth Annual ACM Workshop on COLT (pp. 144-152). Pittsburgh, PA: ACM Press.]]

Digital Library

[4]

Burges, C. J. C. (1998). A tutorial on support vector machine for pattern recognition. Support vector web page available online at: http://svm.first.gmd.de.]]

[5]

Burges, C. J. C., & Schölkopf, B. (1997). Improving the accuracy and speed of support vector machines. In M. Mozer, M. Jordan, & T. Petsche (Eds.), Neural information processing systems, 9. Cambridge, MA: MIT Press.]]

[6]

Drucker, H., Burges, C. J. C., Kaufman, L., Smola, A., & Vapnik, V. (1997). Support vector regression machines. In M. Mozer, M. Jordan, & T. Petsche (Eds.), Neural information processing systems, 9. Cambridge, MA: MIT Press.]]

[7]

Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annual Eugenics, 7, 179-188.]]

[8]

Fukunaga, K. (1990). Introduction to statistical pattern recognition (2nd ed.). Orlando, FL: Academic Press.]]

Digital Library

[9]

Gabrijel, I., & Dobnikar, A. (1997). Adaptive RBF neural network. In Proceedings of SOCO '97 Conference (pp. 164-170).]]

[10]

Gunn, S. R. (1997). Support vector machines for classification and regression (Tech. rep.). Image Speech and Intelligent Systems Research Group, University of Southampton. Available online at: http://www.isis.ecs.soton.ac.uk/ resource/svminfo/.]]

[11]

Hastie, T., Tibshirani, R., & Buja, A. (1993). Flexible discriminant analysis by optimal scoring (Res. Rep.). AT&T Bell Labs.]]

[12]

Kohonen, T. (1994). Self-organizing maps. New York: Springer-Verlag.]]

Digital Library

[13]

Musavi, M. T., Kalantri, K., Ahmed, W., & Chan, K. H. (1993). A minimum error neural network (MNN). Neural Networks, 6, 397-407.]]

Digital Library

[14]

Poggio, T. (1975). On optimal nonlinear associative recall. Biological Cybernetics, 19, 201-209.]]

[15]

Saporta, G. (1990). Probabilites, analyse des données et statistique. Paris: Editions Technip.]]

[16]

Schölkopf, B. (1997). Support vector learning. Munich: R. Oldenbourg Verlag.]]

[17]

Schölkopf, B., Smola, A., & Müller, K. R. (1996). Nonlinear component analysis as a kernel eigenvalue problem (Tech. Rep. No. 44). MPI fur biologische kybernetik.]]

[18]

Schölkopf, B., Smola, A., & Müller, K. R. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10, 1299-1319.]]

Digital Library

[19]

Specht, D. F. (1990). Probabilistic neural networks. Neural Networks, 3(1), 109- 118.]]

Digital Library

[20]

Vapnik, V. (1995). The nature of statistical learning theory. New York: Springer-Verlag.]]

Digital Library

[21]

Vapnik, V., Golowich, S. E., & Smola, A. (1997). Support vector method for function approximation, regression estimation, and signal processing. In M. Mozer, M. Jordan, & T. Petsche (Eds.), Neural information processing systems, 9. Cambridge, MA: MIT Press.]]

[22]

Wilkinson, J. H., & Reinsch, C. (1971). Handbook for automatic computation, Vol. 2: Linear algebra. New York: Springer-Verlag.]]

Digital Library

[23]

Jaakkola, T. S., & Haussler, D. (1999). Exploiting generative models in discriminative classifiers. In M. S. Kearns, S. A. Solla, & D. A. Cohn (Eds.), Advances in neural information processing systems, 11. Cambridge, MA: MIT Press.]]

Digital Library

[24]

Mika, S., Rätsch, G., Weston, J., Schölkopf, B., & Müller, K. R. (1999). Fisher discriminant analysis with kernels. In Proc. IEEE Neural Networks for Signal Processing Workshop, NNSP.]]

Cited By

Xie MTan HDu JYang SYan GLi WFeng J(2024)Eigenspectrum regularisation reverse neighbourhood discriminative learningIET Computer Vision10.1049/cvi2.1228418:6(842-858)Online publication date: 14-May-2024
https://dl.acm.org/doi/10.1049/cvi2.12284
Togban EZiou D(2024)Hierarchical mixture of discriminative Generalized Dirichlet classifiersPattern Recognition10.1016/j.patcog.2024.110789156:COnline publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1016/j.patcog.2024.110789
Wang GHuang XYang YTiwari PZhang J(2024)Euclidean-Distance-Preserved Feature Reduction for efficient person re-identificationNeural Networks10.1016/j.neunet.2024.106572180:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106572
Show More Cited By

Generalized Discriminant Analysis Using a Kernel Approach
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning

Recommendations

Nonlinear Discriminant Analysis Using Kernel Functions and the Generalized Singular Value Decomposition

Linear discriminant analysis (LDA) has been widely used for linear dimension reduction. However, LDA has limitations in that one of the scatter matrices is required to be nonsingular and the nonlinearly clustered structure is not easily captured. In ...
Feature extraction via generalized uncorrelated linear discriminant analysis
ICML '04: Proceedings of the twenty-first international conference on Machine learning

Feature extraction is important in many applications, such as text and image retrieval, because of high dimensionality. Uncorrelated Linear Discriminant Analysis (ULDA) was recently proposed for feature extraction. The extracted features via ULDA were ...
A comparison of generalized linear discriminant analysis algorithms

Linear discriminant analysis (LDA) is a dimension reduction method which finds an optimal linear transformation that maximizes the class separability. However, in undersampled problems where the number of data samples is smaller than the dimension of ...

Comments

Information & Contributors

Information

Published In

cover image Neural Computation

Neural Computation Volume 12, Issue 10

October 2000

242 pages

ISSN:0899-7667

Issue’s Table of Contents

Publisher

MIT Press

Cambridge, MA, United States

Publication History

Published: 01 October 2000

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

337
Total Citations
View Citations
93
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xie MTan HDu JYang SYan GLi WFeng J(2024)Eigenspectrum regularisation reverse neighbourhood discriminative learningIET Computer Vision10.1049/cvi2.1228418:6(842-858)Online publication date: 14-May-2024
https://dl.acm.org/doi/10.1049/cvi2.12284
Togban EZiou D(2024)Hierarchical mixture of discriminative Generalized Dirichlet classifiersPattern Recognition10.1016/j.patcog.2024.110789156:COnline publication date: 18-Nov-2024
https://dl.acm.org/doi/10.1016/j.patcog.2024.110789
Wang GHuang XYang YTiwari PZhang J(2024)Euclidean-Distance-Preserved Feature Reduction for efficient person re-identificationNeural Networks10.1016/j.neunet.2024.106572180:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106572
Zeng XGuo JWei YZhuo Y(2024)Deep hybrid manifold for image set classificationImage and Vision Computing10.1016/j.imavis.2024.104935143:COnline publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1016/j.imavis.2024.104935
Alfeo ACimino MGagliardi G(2024)Recognizing Bearings’ Degradation Stage Using Multimodal Autoencoder to Learn Features from Different Time SeriesSN Computer Science10.1007/s42979-024-02635-55:4Online publication date: 29-Mar-2024
https://dl.acm.org/doi/10.1007/s42979-024-02635-5
Ran RWang TLi ZFang B(2024)Polynomial linear discriminant analysisThe Journal of Supercomputing10.1007/s11227-023-05485-980:1(413-434)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s11227-023-05485-9
Bahar Talukder BFerdaus FRahman M(2023)A Noninvasive Technique to Detect Authentic/Counterfeit SRAM ChipsACM Journal on Emerging Technologies in Computing Systems10.1145/359702419:2(1-25)Online publication date: 30-May-2023
https://dl.acm.org/doi/10.1145/3597024
Nagarajan VMandarapu DKulkarni MGallivan KNikolopoulos DBeivide RGallopoulos E(2023)RT-kNNS Unbound: Using RT Cores to Accelerate Unrestricted Neighbor SearchProceedings of the 37th International Conference on Supercomputing10.1145/3577193.3593738(289-300)Online publication date: 21-Jun-2023
https://dl.acm.org/doi/10.1145/3577193.3593738
Latif SRana RKhalifa SJurdak RQadir JSchuller B(2023)Survey of Deep Representation Learning for Speech Emotion RecognitionIEEE Transactions on Affective Computing10.1109/TAFFC.2021.311436514:2(1634-1654)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1109/TAFFC.2021.3114365
Cai YHao X(2023)ISL-GKFDAInformation Sciences: an International Journal10.1016/j.ins.2023.119449647:COnline publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1016/j.ins.2023.119449
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents