Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Detection of data drift and outliers affecting machine learning model
  performance over time   A trained ML model is deployed on another `test' dataset where target feature
values (labels) are unknown. Drift is distribution change between the training
and deployment data, which is concerning if model performance changes. For a
cat/dog image classifier, for instance, drift during deployment could be rabbit
images (new class) or cat/dog images with changed characteristics (change in
distribution). We wish to detect these changes but can't measure accuracy
without deployment data labels. We instead detect drift indirectly by
nonparametrically testing the distribution of model prediction confidence for
changes. This generalizes our method and sidesteps domain-specific feature
representation.
  We address important statistical issues, particularly Type-1 error control in
sequential testing, using Change Point Models (CPMs; see Adams and Ross 2012).
We also use nonparametric outlier methods to show the user suspicious
observations for model diagnosis, since the before/after change confidence
distributions overlap significantly. In experiments to demonstrate robustness,
we train on a subset of MNIST digit classes, then insert drift (e.g., unseen
digit class) in deployment data in various settings (gradual/sudden changes in
the drift proportion). A novel loss function is introduced to compare the
performance (detection delay, Type-1 and 2 errors) of a drift detector under
different levels of drift class contamination.

15
The categories of this paper are: nucl-ex
21
The categories of this paper are: adap-org
30
The categories of this paper are: q-alg
20
The categories of this paper are: acc-phys
29
The categories of this paper are: funct-an
34
The categories of this paper are: chem-ph
2
The categories of this paper are: math
35
The categories of this paper are: mtrl-th
36
The categories of this paper are: cmp-lg
13
The categories of this paper are: quant-ph
32
The categories of this paper are: atom-ph
12
The categories of this paper are: nucl-th
28
The categories of this paper are: alg-geom
4
The categories of this paper are: cond-mat
18
The categories of this paper are: econ
22
The categories of this paper are: chao-dyn