Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: AI Total: Analyzing Security ML Models with Imperfect Data in Production   Development of new machine learning models is typically done on manually
curated data sets, making them unsuitable for evaluating the models'
performance during operations, where the evaluation needs to be performed
automatically on incoming streams of new data. Unfortunately, pure reliance on
a fully automatic pipeline for monitoring model performance makes it difficult
to understand if any observed performance issues are due to model performance,
pipeline issues, emerging data distribution biases, or some combination of the
above. With this in mind, we developed a web-based visualization system that
allows the users to quickly gather headline performance numbers while
maintaining confidence that the underlying data pipeline is functioning
properly. It also enables the users to immediately observe the root cause of an
issue when something goes wrong. We introduce a novel way to analyze
performance under data issues using a data coverage equalizer. We describe the
various modifications and additional plots, filters, and drill-downs that we
added on top of the standard evaluation metrics typically tracked in machine
learning (ML) applications, and walk through some real world examples that
proved valuable for introspecting our models.

14
The categories of this paper are: math-ph
33
The categories of this paper are: plasm-ph
15
The categories of this paper are: nucl-ex
9
The categories of this paper are: nlin
17
The categories of this paper are: q-fin
16
The categories of this paper are: stat
13
The categories of this paper are: quant-ph
6
The categories of this paper are: astro-ph
28
The categories of this paper are: alg-geom
23
The categories of this paper are: solv-int
5
The categories of this paper are: gr-qc
0
The categories of this paper are: hep-ph
7
The categories of this paper are: hep-th
26
The categories of this paper are: bayes-an
27
The categories of this paper are: comp-gas
32
The categories of this paper are: atom-ph