Dimensionality Reduction using Principal Component Analysis and Autoencoder based on Convolutional Neural Network
Kamran Ullah, Institute of Computer Sciences and Information Technology (ICS/IT), The University of Agriculture Peshawar, Pakistan.
Kamran Ullah, Institute of Computer Sciences and Information Technology (ICS/IT), The University of Agriculture Peshawar, Pakistan.
Iqtidar Ali, Institute of Computer Sciences and Information Technology (ICS/IT), The University of Agriculture Peshawar, Pakistan.
Saleem Zahid, Department of Computer Science, University of Engineering and Technology, Taxila, Pakistan.
Gulzar Ahmed, Department of Computer Science, University of Engineering and Technology, Taxila, Pakistan.
Corresponding Author:
Kamran Ullah (kamranullah@aup.edu.pk)
Abstract:
Dimensionality reduction is an important technique in machine learning and computer vision to reduce the complexity and computational cost of high-dimensional data. In this research paper, we propose a methodology for dimensionality reduction using Principal Component Analysis (PCA) and Autoencoder with Convolutional Neural Networks (CNN) for the popular MNIST dataset, which consists of grayscale images of handwritten digits. We compare the performance of PCA and Autoencoder in reducing the dimensionality of the MNIST dataset, and evaluate the impact on accuracy and loss in a CNN-based image classification task. The experimental results show that the proposed methodology achieves competitive performance in terms of accuracy and loss. Accuracy of CNN with PCA is 98.94% and CNN with Autoencoder is 98.68% while the loss of CNN with PCA is 0.0326 and CNN with Autoencoder is 0.0437. Furthermore, the execution time of CNN is 37 milliseconds on average for each epoch while execution time is five milliseconds on average for each epoch for CNN with PCA and Autoencoder. Therefore, dimensionality reduction for both PCA and Autoencoder performs much better in term of execution without compromising the accuracy and loss.
Keywords:
Dimensionality Reduction; Principal Component Analysis; Autoencoder; Convolutional Neural Networks; MNIST