Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Neural Network Surgery with Sets   The cost to train machine learning models has been increasing exponentially,
making exploration and research into the correct features and architecture a
costly or intractable endeavor at scale. However, using a technique named
"surgery" OpenAI Five was continuously trained to play the game DotA 2 over the
course of 10 months through 20 major changes in features and architecture.
Surgery transfers trained weights from one network to another after a selection
process to determine which sections of the model are unchanged and which must
be re-initialized. In the past, the selection process relied on heuristics,
manual labor, or pre-existing boundaries in the structure of the model,
limiting the ability to salvage experiments after modifications of the feature
set or input reorderings.
  We propose a solution to automatically determine which components of a neural
network model should be salvaged and which require retraining. We achieve this
by allowing the model to operate over discrete sets of features and use
set-based operations to determine the exact relationship between inputs and
outputs, and how they change across tweaks in model architecture. In this
paper, we introduce the methodology for enabling neural networks to operate on
sets, derive two methods for detecting feature-parameter interaction maps, and
show their equivalence. We empirically validate that we can surgery weights
across feature and architecture changes to the OpenAI Five model.

36
The categories of this paper are: cmp-lg
30
The categories of this paper are: q-alg
6
The categories of this paper are: astro-ph
10
The categories of this paper are: q-bio
35
The categories of this paper are: mtrl-th
8
The categories of this paper are: hep-ex
34
The categories of this paper are: chem-ph
32
The categories of this paper are: atom-ph
19
The categories of this paper are: eess
21
The categories of this paper are: adap-org
4
The categories of this paper are: cond-mat
31
The categories of this paper are: ao-sci
14
The categories of this paper are: math-ph
27
The categories of this paper are: comp-gas
2
The categories of this paper are: math
29
The categories of this paper are: funct-an