Instruct: Identify the main and secondary category of Arxiv papers based on the titles and abstracts
Query: ZmBART: An Unsupervised Cross-lingual Transfer Framework for Language
  Generation   Despite the recent advancement in NLP research, cross-lingual transfer for
natural language generation is relatively understudied. In this work, we
transfer supervision from high resource language (HRL) to multiple low-resource
languages (LRLs) for natural language generation (NLG). We consider four NLG
tasks (text summarization, question generation, news headline generation, and
distractor generation) and three syntactically diverse languages, i.e.,
English, Hindi, and Japanese. We propose an unsupervised cross-lingual language
generation framework (called ZmBART) that does not use any parallel or
pseudo-parallel/back-translated data. In this framework, we further pre-train
mBART sequence-to-sequence denoising auto-encoder model with an auxiliary task
using monolingual data of three languages. The objective function of the
auxiliary task is close to the target tasks which enriches the multi-lingual
latent representation of mBART and provides good initialization for target
tasks. Then, this model is fine-tuned with task-specific supervised English
data and directly evaluated with low-resource languages in the Zero-shot
setting. To overcome catastrophic forgetting and spurious correlation issues,
we applied freezing model component and data argumentation approaches
respectively. This simple modeling approach gave us promising results.We
experimented with few-shot training (with 1000 supervised data points) which
boosted the model performance further. We performed several ablations and
cross-lingual transferability analyses to demonstrate the robustness of ZmBART.

4
The categories of this paper are: cond-mat
17
The categories of this paper are: q-fin
18
The categories of this paper are: econ
20
The categories of this paper are: acc-phys
24
The categories of this paper are: dg-ga
23
The categories of this paper are: solv-int
11
The categories of this paper are: hep-lat
32
The categories of this paper are: atom-ph
30
The categories of this paper are: q-alg
29
The categories of this paper are: funct-an
12
The categories of this paper are: nucl-th
5
The categories of this paper are: gr-qc
21
The categories of this paper are: adap-org
34
The categories of this paper are: chem-ph
25
The categories of this paper are: patt-sol
31
The categories of this paper are: ao-sci