Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: Multitask Prompted Training Enables Zero-Shot Task Generalization   Large language models have recently been shown to attain reasonable zero-shot
generalization on a diverse set of tasks (Brown et al., 2020). It has been
hypothesized that this is a consequence of implicit multitask learning in
language models' pretraining (Radford et al., 2019). Can zero-shot
generalization instead be directly induced by explicit multitask learning? To
test this question at scale, we develop a system for easily mapping any natural
language tasks into a human-readable prompted form. We convert a large set of
supervised datasets, each with multiple prompts with diverse wording. These
prompted datasets allow for benchmarking the ability of a model to perform
completely held-out tasks. We fine-tune a pretrained encoder-decoder model
(Raffel et al., 2020; Lester et al., 2021) on this multitask mixture covering a
wide variety of tasks. The model attains strong zero-shot performance on
several standard datasets, often outperforming models up to 16x its size.
Further, our approach attains strong performance on a subset of tasks from the
BIG-bench benchmark, outperforming models up to 6x its size. All trained models
are available at https://github.com/bigscience-workshop/t-zero and all prompts
are available at https://github.com/bigscience-workshop/promptsource.

2
The categories of this paper are: math
10
The categories of this paper are: q-bio
21
The categories of this paper are: adap-org
26
The categories of this paper are: bayes-an
17
The categories of this paper are: q-fin
25
The categories of this paper are: patt-sol
31
The categories of this paper are: ao-sci
7
The categories of this paper are: hep-th
11
The categories of this paper are: hep-lat
34
The categories of this paper are: chem-ph
35
The categories of this paper are: mtrl-th
16
The categories of this paper are: stat
9
The categories of this paper are: nlin
19
The categories of this paper are: eess
30
The categories of this paper are: q-alg
27
The categories of this paper are: comp-gas