[GEM] add WikiLingua cross-lingual abstractive summarization dataset
CONTRIBUTOR
Hey @yjernite. This is a very interesting dataset. Would love to work on adding it but I see that the link to the data is to a gdrive folder. Can I just confirm wether dlmanager can handle gdrive urls or would this have to be a manual dl?
2021-04-15T09:32:15Z
https://github.com/huggingface/datasets/issues/834#issuecomment-820279725
https://api.github.com/repos/huggingface/datasets/issues/834
MDEyOklzc3VlQ29tbWVudDgyMDI3OTcyNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/820279725/reactions
2021-04-15T09:32:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/820279725
https://avatars.githubusercontent.com/u/35491698?v=4
https://api.github.com/users/KMFODA/events{/privacy}
https://api.github.com/users/KMFODA/followers
https://api.github.com/users/KMFODA/following{/other_user}
https://api.github.com/users/KMFODA/gists{/gist_id}
https://github.com/KMFODA
KMFODA
MDQ6VXNlcjM1NDkxNjk4
https://api.github.com/users/KMFODA/orgs
https://api.github.com/users/KMFODA/received_events
https://api.github.com/users/KMFODA/repos
https://api.github.com/users/KMFODA/starred{/owner}{/repo}
https://api.github.com/users/KMFODA/subscriptions
User
https://api.github.com/users/KMFODA
public
MEMBER
Hi @KMFODA ! A version of WikiLingua is actually already accessible in the [GEM dataset](https://huggingface.co/datasets/gem)

You can use it for example to load the French to English translation with:
```python
from datasets import load_dataset
wikilingua = load_dataset("gem", "wiki_lingua_french_fr")
```

Closed by https://github.com/huggingface/datasets/pull/1807
2021-04-15T12:01:38Z
https://github.com/huggingface/datasets/issues/834#issuecomment-820368721
https://api.github.com/repos/huggingface/datasets/issues/834
MDEyOklzc3VlQ29tbWVudDgyMDM2ODcyMQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/820368721/reactions
2021-04-15T12:04:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/820368721
https://avatars.githubusercontent.com/u/10469459?v=4
https://api.github.com/users/yjernite/events{/privacy}
https://api.github.com/users/yjernite/followers
https://api.github.com/users/yjernite/following{/other_user}
https://api.github.com/users/yjernite/gists{/gist_id}
https://github.com/yjernite
yjernite
MDQ6VXNlcjEwNDY5NDU5
https://api.github.com/users/yjernite/orgs
https://api.github.com/users/yjernite/received_events
https://api.github.com/users/yjernite/repos
https://api.github.com/users/yjernite/starred{/owner}{/repo}
https://api.github.com/users/yjernite/subscriptions
User
https://api.github.com/users/yjernite
public
## Adding a Dataset
- **Name:** WikiLingua
- **Description:** The dataset includes ~770k article and summary pairs in 18 languages from WikiHow. The gold-standard article-summary alignments across languages were extracted by aligning the images that are used to describe each how-to step in an article.
- **Paper:** https://arxiv.org/pdf/2010.03093.pdf
- **Data:** https://github.com/esdurmus/Wikilingua
- **Motivation:** Included in the GEM shared task. Multilingual.

Instructions to add a new dataset can be found [here](https://huggingface.co/docs/datasets/share_dataset.html).
