Abstract:
Deep neural networks (DNNs) have become an important tool in solving various problems in numerous disciplines. However, DNNs are also known for their high resource requirement, weight redundancy, and large-scale parameters. As a result, the use of DNNs is restricted for devices that lack the necessary resources required to execute, especially resource-constrained devices such as mobile phones, wearable devices, and other edge devices. In recent years, pruning has emerged as an essential technique to reduce insignificant parameters and accelerate the model performance. However, finding the optimal number of parameters that can be pruned without significantly affecting the model performance is a time-consuming, tedious task and require a lot of manual tuning. This paper represent pruning as an optimization problem with the goal of improving DNN run-time inference performance by pruning low impacting parameters (filters) and their corresponding feature maps. To do this, we present a Bayesian optimization-based method for automatically determining the appropriate number of filters for each convolutional layer. Also, we proposed an objective function incorporating distinct model performance and resource-specific constraints. The proposed method is applied to two different kinds of convolutional network architectures (i.e., VGG16 and deeper network ResNet34) on CIFAR10, CIFAR100, and ImageNet datasets. The large-scale ImageNet experimental findings showed that the floating-point operations of the ResNet34 and VGG16 could be reduced by 35.46 percent and 84.97 percent, respectively, with negligible loss of accuracy.
Publication: Future Generation Computer Systems
Publisher: Science Direct
Authors: Tejalal Choudhary, Vipul Mishra, Anurag Goswami and Jagannathan Sarangapani
Keywords: Neural network, pruning, deep neural network, convolutional neural network
Meet one of the Author:
Dr Tejalal Choudhary received his Ph.D. in Computer Science Engineering from Bennett University in Greater Noida, India. He has expertise in computer vision, machine learning, deep learning, and model compression. His exceptional ability to think creatively, tackle problems from multiple angles, and generate innovative solutions sets him apart from others.
Affiliations:
Bennett University, Greater Noida, India – Bennett University was established in the year 2016 by the Times Group which is India’s largest media conglomerate, to provide Ivy League quality education to undergraduate and postgraduate students. The six schools with 30+ programs and 70+ leading specialisations in Engineering, Management, Media, Law and Liberal Arts have positioned it as one of the top universities in India.
Missouri University of Science and Technology – Missouri University of Science and Technology (Missouri S&T) is a world-class technological research university. Founded in 1870 as one of the first technological schools west of the Mississippi, Missouri S&T offers a broad array of degrees in engineering, the sciences, business, information technology, the humanities and liberal arts, and education — all in an environment that emphasizes technological literacy.
Stay In the Know
Get Latest updates and industry insights every month.