Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: New CleverHans Feature: Better Adversarial Robustness Evaluations with
  Attack Bundling   This technical report describes a new feature of the CleverHans library
called "attack bundling". Many papers about adversarial examples present lists
of error rates corresponding to different attack algorithms. A common approach
is to take the maximum across this list and compare defenses against that error
rate. We argue that a better approach is to use attack bundling: the max should
be taken across many examples at the level of individual examples, then the
error rate should be calculated by averaging after this maximization operation.
Reporting the bundled attacker error rate provides a lower bound on the true
worst-case error rate. The traditional approach of reporting the maximum error
rate across attacks can underestimate the true worst-case error rate by an
amount approaching 100\% as the number of attacks approaches infinity. Attack
bundling can be used with different prioritization schemes to optimize
quantities such as error rate on adversarial examples, perturbation size needed
to cause misclassification, or failure rate when using a specific confidence
threshold.

21
The categories of this paper are: adap-org
30
The categories of this paper are: q-alg
24
The categories of this paper are: dg-ga
35
The categories of this paper are: mtrl-th
20
The categories of this paper are: acc-phys
33
The categories of this paper are: plasm-ph
15
The categories of this paper are: nucl-ex
28
The categories of this paper are: alg-geom
34
The categories of this paper are: chem-ph
22
The categories of this paper are: chao-dyn
9
The categories of this paper are: nlin
31
The categories of this paper are: ao-sci
16
The categories of this paper are: stat
32
The categories of this paper are: atom-ph
23
The categories of this paper are: solv-int
2
The categories of this paper are: math