Preserving the essential features in CNNs: pruning and analysis

Abstract

The exceptional performance of Convolutional Neural Networks (CNNs) entails increasing requirements in computing power and storage. While several efficient compression methods have been developed, there is no consideration on which features are removed or preserved, which can affect pruning. In this paper, we propose a novel filter pruning strategy, named Layer Factor Analysis one to one (LFA1-1), that, relying on explainability, selects the filters that best retain the essential features underlying convolutional layers. We provide insights about the relevance of preserving these features and verify its relationship with compressed network’s performance. The explanatory analysis carried out allows us to justify pruning efficiency and detect problematic parts. Experiments with VGG-16 on CIFAR-10 are conducted in order to validate our approach. Quantitative and qualitative comparisons with methods in the literature uncover pruning properties and prove the effectiveness of our proposal, which reaches a 89.1% parameters and 83.8% FLOPs reduction with the lowest accuracy drop.