Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: BAM! Born-Again Multi-Task Networks for Natural Language Understanding   It can be challenging to train multi-task neural networks that outperform or
even match their single-task counterparts. To help address this, we propose
using knowledge distillation where single-task models teach a multi-task model.
We enhance this training with teacher annealing, a novel method that gradually
transitions the model from distillation to supervised learning, helping the
multi-task model surpass its single-task teachers. We evaluate our approach by
multi-task fine-tuning BERT on the GLUE benchmark. Our method consistently
improves over standard single-task and multi-task training.

27
The categories of this paper are: comp-gas
20
The categories of this paper are: acc-phys
35
The categories of this paper are: mtrl-th
19
The categories of this paper are: eess
18
The categories of this paper are: econ
22
The categories of this paper are: chao-dyn
33
The categories of this paper are: plasm-ph
6
The categories of this paper are: astro-ph
32
The categories of this paper are: atom-ph
13
The categories of this paper are: quant-ph
4
The categories of this paper are: cond-mat
15
The categories of this paper are: nucl-ex
5
The categories of this paper are: gr-qc
11
The categories of this paper are: hep-lat
3
The categories of this paper are: physics
8
The categories of this paper are: hep-ex