Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Adaptive Sampled Softmax with Kernel Based Sampling   Softmax is the most commonly used output function for multiclass problems and
is widely used in areas such as vision, natural language processing, and
recommendation. A softmax model has linear costs in the number of classes which
makes it too expensive for many real-world problems. A common approach to speed
up training involves sampling only some of the classes at each training step.
It is known that this method is biased and that the bias increases the more the
sampling distribution deviates from the output distribution. Nevertheless,
almost any recent work uses simple sampling distributions that require a large
sample size to mitigate the bias. In this work, we propose a new class of
kernel based sampling methods and develop an efficient sampling algorithm.
Kernel based sampling adapts to the model as it is trained, thus resulting in
low bias. Kernel based sampling can be easily applied to many models because it
relies only on the model's last hidden layer. We empirically study the
trade-off of bias, sampling distribution and sample size and show that kernel
based sampling results in low bias with few samples.

17
The categories of this paper are: q-fin
11
The categories of this paper are: hep-lat
25
The categories of this paper are: patt-sol
16
The categories of this paper are: stat
33
The categories of this paper are: plasm-ph
32
The categories of this paper are: atom-ph
34
The categories of this paper are: chem-ph
23
The categories of this paper are: solv-int
9
The categories of this paper are: nlin
3
The categories of this paper are: physics
6
The categories of this paper are: astro-ph
13
The categories of this paper are: quant-ph
20
The categories of this paper are: acc-phys
4
The categories of this paper are: cond-mat
29
The categories of this paper are: funct-an
0
The categories of this paper are: hep-ph