Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Selection via Proxy: Efficient Data Selection for Deep Learning   Data selection methods, such as active learning and core-set selection, are
useful tools for machine learning on large datasets. However, they can be
prohibitively expensive to apply in deep learning because they depend on
feature representations that need to be learned. In this work, we show that we
can greatly improve the computational efficiency by using a small proxy model
to perform data selection (e.g., selecting data points to label for active
learning). By removing hidden layers from the target model, using smaller
architectures, and training for fewer epochs, we create proxies that are an
order of magnitude faster to train. Although these small proxy models have
higher error rates, we find that they empirically provide useful signals for
data selection. We evaluate this "selection via proxy" (SVP) approach on
several data selection tasks across five datasets: CIFAR10, CIFAR100, ImageNet,
Amazon Review Polarity, and Amazon Review Full. For active learning, applying
SVP can give an order of magnitude improvement in data selection runtime (i.e.,
the time it takes to repeatedly train and select points) without significantly
increasing the final error (often within 0.1%). For core-set selection on
CIFAR10, proxies that are over 10x faster to train than their larger, more
accurate targets can remove up to 50% of the data without harming the final
accuracy of the target, leading to a 1.6x end-to-end training time improvement.

30
The categories of this paper are: q-alg
2
The categories of this paper are: math
25
The categories of this paper are: patt-sol
15
The categories of this paper are: nucl-ex
4
The categories of this paper are: cond-mat
35
The categories of this paper are: mtrl-th
27
The categories of this paper are: comp-gas
21
The categories of this paper are: adap-org
7
The categories of this paper are: hep-th
23
The categories of this paper are: solv-int
9
The categories of this paper are: nlin
17
The categories of this paper are: q-fin
18
The categories of this paper are: econ
31
The categories of this paper are: ao-sci
10
The categories of this paper are: q-bio
32
The categories of this paper are: atom-ph