Robust SVM Classification with l0-Norm Feature Selection

Abstract

In this article, we introduce a robust classification model designed for feature selection. Support vector machine (SVM) models continue to play a crucial role in binary classification, particularly with tabular data. Their robust variants are essential for developing classifiers that remain stable despite shifts in data distribution. Additionally, sparse classifiers are highly desirable, as they offer improved performance in classification tasks and help reduce overfitting, especially when the number of features exceeds the number of samples. In this context, penalty methods for feature selection are fundamental to the development of sparse optimization models. Despite its inherent nonlinearity and nonconvexity, the l0-norm remains a popular choice for this task and has been effectively applied in diverse fields, such as pattern recognition and signal processing. Our objective is to evaluate the performance of two classification approaches that integrate robust classification in SVM-type models with embedded feature selection. We propose a diagonal algorithm for numerically solving these optimization models, which we validate through numerical tests on benchmark datasets. The results are compared to those obtained using models based on l2-norm regularization.