Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: TraDE: Transformers for Density Estimation   We present TraDE, a self-attention-based architecture for auto-regressive
density estimation with continuous and discrete valued data. Our model is
trained using a penalized maximum likelihood objective, which ensures that
samples from the density estimate resemble the training data distribution. The
use of self-attention means that the model need not retain conditional
sufficient statistics during the auto-regressive process beyond what is needed
for each covariate. On standard tabular and image data benchmarks, TraDE
produces significantly better density estimates than existing approaches such
as normalizing flow estimators and recurrent auto-regressive models. However
log-likelihood on held-out data only partially reflects how useful these
estimates are in real-world applications. In order to systematically evaluate
density estimators, we present a suite of tasks such as regression using
generated samples, out-of-distribution detection, and robustness to noise in
the training data and demonstrate that TraDE works well in these scenarios.

31
The categories of this paper are: ao-sci
20
The categories of this paper are: acc-phys
9
The categories of this paper are: nlin
22
The categories of this paper are: chao-dyn
12
The categories of this paper are: nucl-th
21
The categories of this paper are: adap-org
2
The categories of this paper are: math
29
The categories of this paper are: funct-an
5
The categories of this paper are: gr-qc
36
The categories of this paper are: cmp-lg
19
The categories of this paper are: eess
4
The categories of this paper are: cond-mat
23
The categories of this paper are: solv-int
24
The categories of this paper are: dg-ga
15
The categories of this paper are: nucl-ex
14
The categories of this paper are: math-ph