Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Automatic Code Generation using Pre-Trained Language Models   Recent advancements in natural language processing \cite{gpt2} \cite{BERT}
have led to near-human performance in multiple natural language tasks. In this
paper, we seek to understand whether similar techniques can be applied to a
highly structured environment with strict syntax rules. Specifically, we
propose an end-to-end machine learning model for code generation in the Python
language built on-top of pre-trained language models. We demonstrate that a
fine-tuned model can perform well in code generation tasks, achieving a BLEU
score of 0.22, an improvement of 46\% over a reasonable sequence-to-sequence
baseline. All results and related code used for training and data processing
are available on GitHub.

12
The categories of this paper are: nucl-th
19
The categories of this paper are: eess
7
The categories of this paper are: hep-th
18
The categories of this paper are: econ
17
The categories of this paper are: q-fin
21
The categories of this paper are: adap-org
2
The categories of this paper are: math
33
The categories of this paper are: plasm-ph
30
The categories of this paper are: q-alg
28
The categories of this paper are: alg-geom
9
The categories of this paper are: nlin
23
The categories of this paper are: solv-int
36
The categories of this paper are: cmp-lg
3
The categories of this paper are: physics
32
The categories of this paper are: atom-ph
24
The categories of this paper are: dg-ga