Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for
  Code Understanding and Generation   Pre-trained models for Natural Languages (NL) like BERT and GPT have been
recently shown to transfer well to Programming Languages (PL) and largely
benefit a broad set of code-related tasks. Despite their success, most current
methods either rely on an encoder-only (or decoder-only) pre-training that is
suboptimal for generation (resp. understanding) tasks or process the code
snippet in the same way as NL, neglecting the special characteristics of PL
such as token types. We present CodeT5, a unified pre-trained encoder-decoder
Transformer model that better leverages the code semantics conveyed from the
developer-assigned identifiers. Our model employs a unified framework to
seamlessly support both code understanding and generation tasks and allows for
multi-task learning. Besides, we propose a novel identifier-aware pre-training
task that enables the model to distinguish which code tokens are identifiers
and to recover them when they are masked. Furthermore, we propose to exploit
the user-written code comments with a bimodal dual generation task for better
NL-PL alignment. Comprehensive experiments show that CodeT5 significantly
outperforms prior methods on understanding tasks such as code defect detection
and clone detection, and generation tasks across various directions including
PL-NL, NL-PL, and PL-PL. Further analysis reveals that our model can better
capture semantic information from code. Our code and pre-trained models are
released at https: //github.com/salesforce/CodeT5 .

9
The categories of this paper are: nlin
17
The categories of this paper are: q-fin
36
The categories of this paper are: cmp-lg
30
The categories of this paper are: q-alg
27
The categories of this paper are: comp-gas
3
The categories of this paper are: physics
18
The categories of this paper are: econ
25
The categories of this paper are: patt-sol
13
The categories of this paper are: quant-ph
22
The categories of this paper are: chao-dyn
24
The categories of this paper are: dg-ga
28
The categories of this paper are: alg-geom
8
The categories of this paper are: hep-ex
5
The categories of this paper are: gr-qc
6
The categories of this paper are: astro-ph
29
The categories of this paper are: funct-an