Deep Neural Networks are characterized by the weight, bias, and activation function. The activation functions decide whether a neuron should be activated or not by computing weighted sums and biases. In this book, I represent an experimental review on the eight different activation functions for the Convolutional layers in Neural Networks.
For my experiment, I selected eight activation functions for three different datasets. The activation functions are – Sigmoid, Softmax, tanh, Softplus, Softsign, ReLU, ELU, and SELU. I also experimented on the networks where I did not use any activation function for the convolutional layers. After analyzing the results I see that the models with three different activation functions achieved higher performance for the three datasets. The interesting thing is, the best average performance for the three datasets is achieved by using the Softmax activation function for the Convolutional layer. At present time ReLu and ELU are the most used activation functions, but I see that tanh, Softplus, and Softsign achieved better average performance for the three datasets.
In this book, I just focus only on the activation function for the Convolutional layer to test the performance of the Convolutional Neural Networks.
Write your review for this book (optional)
Review Deleted
Your review has been deleted and won’t appear on the book anymore.
Al Mahmud Al Mamun, Prof. Dr. Ahsan-Ul-Ambia
Al Mahmud Al Mamun
He is a Researcher, Editor, and Author. He is the Editor-in-Chief at Circuit Cellar Magazine, United States. He is a Writer at TechnologyAdvice, United States. Although, his Bachelor's and Master's in Computer Science and Engineering, he also attained thirty diploma courses and a hundred certificate courses. According to Publons (Web of Science Core Collections), he is one of the World's top peer-reviewers on Convolutional Neural Networks. His expertise and research interests include Convolutional Neural Networks (CNN), Artificial Neural Networks (ANN), Cloud Computing, Artificial Intelligence (AI), Intelligent Transportation Systems, Information Technology (IT), System Development Life Cycle (SDLC), Computer Vision, Psychology, and Astronomy.
Prof. Dr. Ahsan-Ul-Ambia
He is one of the most senior Professors in the Department of Computer Science and Engineering at Islamic University, Bangladesh. He has done his Doctor of Philosophy (Ph.D.) from Shizuoka University, Japan. He achieved a rich research profile in several areas. His research skills and expertise includes - Electric Impedance, Image Reconstruction, Biomedical Imaging, Tomography, Electronic Instrumentation, Medical Imaging, Medical and Biomedical Image Processing, Medical Imaging Physics, Image Processing, Imaging, Tomographic Reconstruction, and Computed Tomography.
