research-article

The Convolution Neural Network with Transformed Exponential Linear Unit Activation Function for Image Classification

Authors:

Peter T. Yamak,

Ting ZhangAuthors Info & Claims

IVSP '19: Proceedings of the 2019 International Conference on Image, Video and Signal Processing

Pages 55 - 62

https://doi.org/10.1145/3317640.3317649

Published: 25 February 2019 Publication History

Abstract

Activation functions play an important role in deep learning and its choice has a significant effect on the training and performance of a model. In this study, a new variant of Exponential Linear Unit (ELU) activation called Transformed Exponential Linear Unit (TELU) is proposed. An empirical evaluation is done to determine the effectiveness of the new activation function using state-of-the-art deep learning architectures. From the experiments, TELU activation function tends to work better than the conventional activations functions on deep models across a number of benchmarking datasets. TELU achieves superior classification accuracy on Cifar-10, SVHN and Caltech-101 dataset on state-of-the-art deep learning models. Additionally, it shows superior AUROC, MCC, and F1-score on the STL-10 dataset. This proves that TELU can be successfully applied in deep learning for image classification.

References

[1]

G. Huang, K. Q. Weinberger, and L. Van Der Maaten, "Densely Connected Convolutional Networks."

[2]

K. Zhang, M. Sun, T. X. Han, X. Yuan, L. Guo, and T. Liu, "Residual Networks of Residual Networks: Multilevel Residual Networks," Arxiv, vol. 14, no. 8, pp. 1--10, 2016.

[3]

C. Szegedy, V. Vanhoucke, and J. Shlens, "Rethinking the Inception Architecture for Computer Vision," 2014.

[4]

F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, "SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size," pp. 1--13, 2016.

[5]

R. Arora, A. Basu, P. Mianjy, and A. Mukherjee, "Understanding Deep Neural Networks with Rectified Linear Units," pp. 1--21, 2016.

[6]

D.-A. Clevert, T. Unterthiner, and S. Hochreiter, "Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)," pp. 1--14, 2015.

[7]

G. Klambauer, T. Unterthiner, A. Mayr, and S. Hochreiter, "Self-Normalizing Neural Networks," 2017.

[8]

K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," Arxiv.Org, vol. 7, no. 3, pp. 171--180, 2015.

[9]

K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," Proc. IEEE Int. Conf. Comput. Vis., vol. 2015 Inter, pp. 1026--1034, 2015.

Digital Library

[10]

Y. Netzer and T. Wang, "Reading digits in natural images with unsupervised feature learning," Nips, pp. 1--9, 2011.

[11]

A. Coates, A. Arbor, and A. Y. Ng, "coatesAnalysis.pdf."

[12]

L. N. Smith, "Cyclical Learning Rates for Training Neural Networks," no. April, 2015.

[13]

Krizhevsky, A. Learning multiple layers of features from tiny images, 2009.

[14]

Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. L. Fei-Fei, R. Fergus, and P. Perona. CVPR 2004, Workshop on Generative-Model Based Vision. 2004

Digital Library

[15]

Chollet, F. Keras. https://github.com/ fchollet/keras, 2016.

Cited By

Matthew UKazaure JOnyebuchi ADaniel OMuhammed IOkafor N(2021)Artificial Intelligence Autonomous Unmanned Aerial Vehicle (UAV) System for Remote Sensing in Security Surveillance2020 IEEE 2nd International Conference on Cyberspac (CYBER NIGERIA)10.1109/CYBERNIGERIA51635.2021.9428862(1-10)Online publication date: 23-Feb-2021
https://doi.org/10.1109/CYBERNIGERIA51635.2021.9428862
Hao XYang GYe QLin D(2019)Rare Animal Image Recognition Based on Convolutional Neural Networks2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI)10.1109/CISP-BMEI48845.2019.8965748(1-5)Online publication date: Oct-2019
https://doi.org/10.1109/CISP-BMEI48845.2019.8965748

Index Terms

The Convolution Neural Network with Transformed Exponential Linear Unit Activation Function for Image Classification
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Performance analysis of nonlinear activation function in convolution neural network for image classification

Deep learning architectures which are exceptionally deep have exhibited to be incredibly powerful models for image processing. As the architectures become deep, it introduces challenges and difficulties in the training process such as overfitting, ...
ErfReLU: adaptive activation function for deep neural network
Abstract
Recent research has found that the activation function (AF) plays a significant role in introducing non-linearity to enhance the performance of deep learning networks. Researchers recently started developing activation functions that can be ...
Improving deep neural network with Multiple Parametric Exponential Linear Units

Activation function is crucial to the recent successes of deep neural networks. In this paper, we first propose a new activation function, Multiple Parametric Exponential Linear Units (MPELU), aiming to generalize and unify the rectified and exponential ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

IVSP '19: Proceedings of the 2019 International Conference on Image, Video and Signal Processing

February 2019

140 pages

ISBN:9781450361750

DOI:10.1145/3317640

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Wuhan Univ.: Wuhan University, China
City University of Hong Kong: City University of Hong Kong

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 February 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

IVSP 2019

IVSP 2019: 2019 International Conference on Image, Video and Signal Processing

February 25 - 28, 2019

Shanghai, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
141
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Matthew UKazaure JOnyebuchi ADaniel OMuhammed IOkafor N(2021)Artificial Intelligence Autonomous Unmanned Aerial Vehicle (UAV) System for Remote Sensing in Security Surveillance2020 IEEE 2nd International Conference on Cyberspac (CYBER NIGERIA)10.1109/CYBERNIGERIA51635.2021.9428862(1-10)Online publication date: 23-Feb-2021
https://doi.org/10.1109/CYBERNIGERIA51635.2021.9428862
Hao XYang GYe QLin D(2019)Rare Animal Image Recognition Based on Convolutional Neural Networks2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI)10.1109/CISP-BMEI48845.2019.8965748(1-5)Online publication date: Oct-2019
https://doi.org/10.1109/CISP-BMEI48845.2019.8965748

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten