  Coreset selection seeks to choose a subset of crucial training samples for
efficient learning. It has gained traction in deep learning, particularly with
the surge in training dataset sizes. Sample selection hinges on two main
aspects: a sample's representation in enhancing performance and the role of
sample diversity in averting overfitting. Existing methods typically measure
both the representation and diversity of data based on similarity metrics, such
as L2-norm. They have capably tackled representation via distribution matching
guided by the similarities of features, gradients, or other information between
data. However, the results of effectively diverse sample selection are mired in
sub-optimality. This is because the similarity metrics usually simply aggregate
dimension similarities without acknowledging disparities among the dimensions
that significantly contribute to the final similarity. As a result, they fall
short of adequately capturing diversity. To address this, we propose a
feature-based diversity constraint, compelling the chosen subset to exhibit
maximum diversity. Our key lies in the introduction of a novel Contributing
Dimension Structure (CDS) metric. Different from similarity metrics that
measure the overall similarity of high-dimensional features, our CDS metric
considers not only the reduction of redundancy in feature dimensions, but also
the difference between dimensions that contribute significantly to the final
similarity. We reveal that existing methods tend to favor samples with similar
CDS, leading to a reduced variety of CDS types within the coreset and
subsequently hindering model performance. In response, we enhance the
performance of five classical selection methods by integrating the CDS
constraint. Our experiments on three datasets demonstrate the general
effectiveness of the proposed method in boosting existing methods.
