New internal clustering validation measure for contiguous arbitrary shape clusters

Abstract

In this study a new internal clustering validation index is proposed. It is based on a measure of the uniformity of the data in clusters. It uses the local density of each cluster, in particular, the normalized variability of the density within the clusters to find the ideal partition. The new validity measure allows it to capture the spatial pattern of the data and obtain the right number of clusters in an automatic way. This new approach, unlike the traditional one that usually identifies well-separated compact clouds, works with arbitrary-shape clusters that may be contiguous or even overlapped. The proposed clustering measure has been evaluated on nine artificial data sets, with different cluster distributions and an increasing number of classes, on three highly nonlinear data sets, and on 17 real data sets. It has been compared with nine well-known clustering validation indices with very satisfactory results. This proves that including density in the definition of clustering validation indices may be useful to identify the right partition of arbitrary-shape and different-size clusters.