structural:scholar_edition
Bias in ImageNet

It is estimated that over 6% of labels in the ImageNet-1k validation set are wrong. It is also found that around 10% of ImageNet-1k contains ambiguous or erroneous labels, and that, when presented with a model's prediction and the original ImageNet label, human annotators prefer the prediction of a state of the art model in 2020 trained on the original ImageNet, suggesting that ImageNet-1k has been saturated.