skip to main content
10.1145/3591156.3591164acmotherconferencesArticle/Chapter ViewAbstractPublication PagesivspConference Proceedingsconference-collections
research-article

Transformer for Hyperspectral Image Classification based on Multi-feature Learning

Published: 16 June 2023 Publication History

Abstract

The CNN-based HSI classification methods are difficulty to capture the long-range spectral dependencies that resulting to lose the intrinsic and potentially spectral information. To address it, this letter proposes a model that combine transformer and portion of Inception V4 to learn multi-feature for the HSI classification, where multi-features are incorporated by multi-bands to address the problems from data features to redundant bands. The band clustering and selection strategy is applied to obtain multi-bank samples at first, subsequently, we utilize the first reduction block in Inception V4 network to extract the multi spatial features, then transformer is utilized to learn the long-range sequential spectra data, multilayer perceptron(MLP) is conducted the final classification task. The multi-head self-attention within the transformer is adopted to integrate information with different band cluster and emphasize the contributive spectral features. The proposed method is tested on two widely used hyperspectral datasets and shows the well results.

References

[1]
Tu B, Zhou C, Peng J, Feature extraction via joint adaptive structure density for hyperspectral imagery classification[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-16.
[2]
Jiang G, Sun Y, Liu B. A fully convolutional network with channel and spatial attention for hyperspectral image classification[J]. Remote Sensing Letters, 2021, 12(12): 1238-1249.
[3]
Szegedy C, Ioffe S, Vanhoucke V, Inception-v4, inception-resnet and the impact of residual connections on learning[C]//Thirty-first AAAI conference on artificial intelligence. 2017.
[4]
Seydgar, M.; Alizadeh Naeini, A.; Zhang, M.; Li, W. 3-D convolution-recurrent networks for spectral-spatial classification of hyperspectral images. Remote Sens. 2019, 11(7): 883.
[5]
He X, Chen Y. Optimized input for CNN-based hyperspectral image classification using spatial transformer network[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(12): 1884-1888.
[6]
Qing Y, Liu W, Feng L, Improved transformer net for hyperspectral image classification[J]. Remote Sensing, 2021, 13(11): 2216.
[7]
Chen C F R, Fan Q, Panda R. Crossvit: Cross-attention multi-scale vision transformer for image classification[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2021: 357-366.
[8]
Dosovitskiy An image is worth 16×16 words: Transformers for image recognition at scale,” 2020, arXiv:2010.11929
[9]
He W, Huang W, Liao S, CSiT: A Multiscale Vision Transformer for Hyperspectral Image Classification[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 9266-9277
[10]
Haut, J. M., M. E. Paoletti, J. Plaza, A. Plaza, and J. Li. Hyperspectral Image Classification Using Random Occlusion Data Augmentation. IEEE Geoscience and Remote Sensing Letters, 2019,16 (11):1751–1755.
[11]
V Slavkovikj, S Verstockt, W De Neve, S .Van Hoecke,and R.Van de Walle, Hyperspectral image classification with convolutional neural networks, in the Proceedings of the 23rd ACM international conference on Multimedia,2015:1159-1162.
[12]
Y. Chen, X. Zhao and X. jia, Spectral-spatial classification of hyperspectral data based on deep belief network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol.8, no. 6,2015:2381-2392.

Index Terms

  1. Transformer for Hyperspectral Image Classification based on Multi-feature Learning

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    IVSP '23: Proceedings of the 2023 5th International Conference on Image, Video and Signal Processing
    March 2023
    207 pages
    ISBN:9781450398381
    DOI:10.1145/3591156
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 16 June 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Hyperspectral Image classification
    2. multi-feature
    3. multi-head self-attention
    4. transformer

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    IVSP 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 39
      Total Downloads
    • Downloads (Last 12 months)16
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 28 Jan 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media