Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Automated Source Code Generation and Auto-completion Using Deep
  Learning: Comparing and Discussing Current Language-Model-Related Approaches   In recent years, the use of deep learning in language models gained much
attention. Some research projects claim that they can generate text that can be
interpreted as human-writing, enabling new possibilities in many application
areas. Among the different areas related to language processing, one of the
most notable in applying this type of modeling is programming languages. For
years, the Machine Learning community has been researching this software
engineering area, pursuing goals like applying different approaches to
auto-complete, generate, fix, or evaluate code programmed by humans.
Considering the increasing popularity of the Deep-Learning-enabled language
models approach, we detected a lack of empirical papers that compare different
deep learning architectures to create and use language models based on
programming code. This paper compares different neural network architectures
like AWD-LSTMs, AWD-QRNNs, and Transformer while using transfer learning and
different tokenizations to see how they behave in building language models
using a Python dataset for code generation and filling mask tasks. Considering
the results, we discuss each approach's different strengths and weaknesses and
what gaps we find to evaluate the language models or apply them in a real
programming context.

25
The categories of this paper are: patt-sol
30
The categories of this paper are: q-alg
2
The categories of this paper are: math
12
The categories of this paper are: nucl-th
0
The categories of this paper are: hep-ph
17
The categories of this paper are: q-fin
33
The categories of this paper are: plasm-ph
36
The categories of this paper are: cmp-lg
13
The categories of this paper are: quant-ph
26
The categories of this paper are: bayes-an
14
The categories of this paper are: math-ph
22
The categories of this paper are: chao-dyn
5
The categories of this paper are: gr-qc
21
The categories of this paper are: adap-org
18
The categories of this paper are: econ
15
The categories of this paper are: nucl-ex