Adding the Linguistic Code-switching Evaluation (LinCE) benchmark
CONTRIBUTOR
I am checking the details of the CI log for the failed test, but I don't see how the error relates to the code I added; the error is coming from a config builder different than the `LinceConfig`, and it crashes when `self.config.data_files` because is self.config is None. I would appreciate if someone could help me find out where I could have messed things up :)

Also, the real and dummy data tests passed before committing and pushing my changes.

Thanks a lot in advance!

```
=================================== FAILURES ===================================
____________________ AWSDatasetTest.test_load_dataset_text _____________________

self = <tests.test_dataset_common.AWSDatasetTest testMethod=test_load_dataset_text>
dataset_name = 'text'

    def test_load_dataset(self, dataset_name):
        configs = self.dataset_tester.load_all_configs(dataset_name)[:1]
>       self.dataset_tester.check_load_dataset(dataset_name, configs)

tests/test_dataset_common.py:243: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/test_dataset_common.py:137: in check_load_dataset
    try_from_hf_gcs=False,
../.local/lib/python3.6/site-packages/nlp/builder.py:432: in download_and_prepare
    dl_manager=dl_manager, verify_infos=verify_infos, **download_and_prepare_kwargs
../.local/lib/python3.6/site-packages/nlp/builder.py:466: in _download_and_prepare
    split_generators = self._split_generators(dl_manager, **split_generators_kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <nlp.datasets.text.bf5568367c6707640e5601a44ed0af98f40a8483db81a7db99b85fab6606fc8b.text.Text object at 0x7efa744ffb70>
dl_manager = <nlp.utils.mock_download_manager.MockDownloadManager object at 0x7efb304c52b0>

    def _split_generators(self, dl_manager):
        """ The `datafiles` kwarg in load_dataset() can be a str, List[str], Dict[str,str], or Dict[str,List[str]].
    
            If str or List[str], then the dataset returns only the 'train' split.
            If dict, then keys should be from the `nlp.Split` enum.
        """
        if isinstance(self.config.data_files, (str, list, tuple)):
            # Handle case with only one split
            files = self.config.data_files
            if isinstance(files, str):
                files = [files]
            return [nlp.SplitGenerator(name=nlp.Split.TRAIN, gen_kwargs={"files": files})]
        else:
            # Handle case with several splits and a dict mapping
            splits = []
            for split_name in [nlp.Split.TRAIN, nlp.Split.VALIDATION, nlp.Split.TEST]:
>               if split_name in self.config.data_files:
E               TypeError: argument of type 'NoneType' is not iterable

../.local/lib/python3.6/site-packages/nlp/datasets/text/bf5568367c6707640e5601a44ed0af98f40a8483db81a7db99b85fab6606fc8b/text.py:24: TypeError
=============================== warnings summary ===============================
... 
=========================== short test summary info ============================
FAILED tests/test_dataset_common.py::AWSDatasetTest::test_load_dataset_text
====== 1 failed, 963 passed, 532 skipped, 5 warnings in 166.33s (0:02:46) ======

Exited with code exit status 1
```
2020-07-12T00:38:09Z
https://github.com/huggingface/datasets/pull/383#issuecomment-657154578
https://api.github.com/repos/huggingface/datasets/issues/383
MDEyOklzc3VlQ29tbWVudDY1NzE1NDU3OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/657154578/reactions
2020-07-12T02:03:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/657154578
https://avatars.githubusercontent.com/u/5833357?v=4
https://api.github.com/users/gaguilar/events{/privacy}
https://api.github.com/users/gaguilar/followers
https://api.github.com/users/gaguilar/following{/other_user}
https://api.github.com/users/gaguilar/gists{/gist_id}
https://github.com/gaguilar
gaguilar
MDQ6VXNlcjU4MzMzNTc=
https://api.github.com/users/gaguilar/orgs
https://api.github.com/users/gaguilar/received_events
https://api.github.com/users/gaguilar/repos
https://api.github.com/users/gaguilar/starred{/owner}{/repo}
https://api.github.com/users/gaguilar/subscriptions
User
https://api.github.com/users/gaguilar
public
CONTRIBUTOR
@lhoestq Hi Quentin, I was wondering if you could give some feedback on this error from the `run_dataset_script_tests` script. It seems that's coming from a different config builder than the one I added, so I am not sure why this error would occur. Thanks in advance!
2020-07-14T21:04:14Z
https://github.com/huggingface/datasets/pull/383#issuecomment-658412676
https://api.github.com/repos/huggingface/datasets/issues/383
MDEyOklzc3VlQ29tbWVudDY1ODQxMjY3Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/658412676/reactions
2020-07-14T21:04:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/658412676
https://avatars.githubusercontent.com/u/5833357?v=4
https://api.github.com/users/gaguilar/events{/privacy}
https://api.github.com/users/gaguilar/followers
https://api.github.com/users/gaguilar/following{/other_user}
https://api.github.com/users/gaguilar/gists{/gist_id}
https://github.com/gaguilar
gaguilar
MDQ6VXNlcjU4MzMzNTc=
https://api.github.com/users/gaguilar/orgs
https://api.github.com/users/gaguilar/received_events
https://api.github.com/users/gaguilar/repos
https://api.github.com/users/gaguilar/starred{/owner}{/repo}
https://api.github.com/users/gaguilar/subscriptions
User
https://api.github.com/users/gaguilar
public
CONTRIBUTOR
Awesome! Thank you for all your comments! 👌  I will update the PR in a bit with all the required changes 🙂 

Let me just provide a bit of context for my changes:

I was referring to the GLUE, XTREME and WNUT_17 dataset scripts to build mine (not sure if the new documentation was available last week). This is where I took the naming convention for the citation and description variables. Also, these scripts didn't have the `BUILDER_CONFIG_CLASS = LinceConfig` line so I commented this out thinking I didn't need that; I tried this line in my attempts to make the real and dummy data tests pass but it was not helping. 

The problem I was facing was that the tests were passing a default `BuilderConfig` (i.e., `self.config.name` property was set to `'default'` and my custom properties were not available). This means, for example, that within the `def _info(...)`  method, I was not able to access the specific fields of my `LinceConfig` class (which is why I have now a global variable `_LINCE_CITATIONS`, to detach the individual citations from the corresponding LinceConfig objects, as well as I am constructing manually the feature infos). This default `BuilderConfig` is why I added the `if not isinstance(self.config, LinceConfig): return []` statement. Otherwise, accessing custom properties like `self.config.colnames` was failing the test because such properties did not exist in the default config (i.e., it was not a `LinceConfig`).

I will update the PR and see if these problems happen in the CI tests.

Thanks again for the follow-up! @lhoestq 
2020-07-15T14:18:51Z
https://github.com/huggingface/datasets/pull/383#issuecomment-658796386
https://api.github.com/repos/huggingface/datasets/issues/383
MDEyOklzc3VlQ29tbWVudDY1ODc5NjM4Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/658796386/reactions
2020-07-15T14:18:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/658796386
https://avatars.githubusercontent.com/u/5833357?v=4
https://api.github.com/users/gaguilar/events{/privacy}
https://api.github.com/users/gaguilar/followers
https://api.github.com/users/gaguilar/following{/other_user}
https://api.github.com/users/gaguilar/gists{/gist_id}
https://github.com/gaguilar
gaguilar
MDQ6VXNlcjU4MzMzNTc=
https://api.github.com/users/gaguilar/orgs
https://api.github.com/users/gaguilar/received_events
https://api.github.com/users/gaguilar/repos
https://api.github.com/users/gaguilar/starred{/owner}{/repo}
https://api.github.com/users/gaguilar/subscriptions
User
https://api.github.com/users/gaguilar
public
MEMBER
Ok I see !

To give you more details: the line `BUILDER_CONFIG_CLASS = LinceConfig` tells the tests how to instantiate a config for this dataset. Therefore if you have this line you should have all the fields of your config available.

To fix the errors you get you'll have to, first, have the `BUILDER_CONFIG_CLASS = LinceConfig` line, and second, add default values for the parameters of your config (or the tests functions will be unable to instantiate it by calling `LinceConfig()`.

An example of dataset with a custom config with additional filed like this one is [biomrc](https://github.com/huggingface/nlp/blob/master/datasets/biomrc/biomrc.py).
Feel free to give a look at it if you want.
2020-07-15T14:29:07Z
https://github.com/huggingface/datasets/pull/383#issuecomment-658802328
https://api.github.com/repos/huggingface/datasets/issues/383
MDEyOklzc3VlQ29tbWVudDY1ODgwMjMyOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/658802328/reactions
2020-07-15T14:29:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/658802328
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Thanks for the reference!

I just updated the PR with the suggested changes. It seems the CI failed on the same test you said we could ignore, so I guess it's okay :) 

Please let me know if there is something else I may need to change.
2020-07-15T17:05:46Z
https://github.com/huggingface/datasets/pull/383#issuecomment-658886825
https://api.github.com/repos/huggingface/datasets/issues/383
MDEyOklzc3VlQ29tbWVudDY1ODg4NjgyNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/658886825/reactions
2020-07-15T17:05:46Z
https://api.github.com/repos/huggingface/datasets/issues/comments/658886825
https://avatars.githubusercontent.com/u/5833357?v=4
https://api.github.com/users/gaguilar/events{/privacy}
https://api.github.com/users/gaguilar/followers
https://api.github.com/users/gaguilar/following{/other_user}
https://api.github.com/users/gaguilar/gists{/gist_id}
https://github.com/gaguilar
gaguilar
MDQ6VXNlcjU4MzMzNTc=
https://api.github.com/users/gaguilar/orgs
https://api.github.com/users/gaguilar/received_events
https://api.github.com/users/gaguilar/repos
https://api.github.com/users/gaguilar/starred{/owner}{/repo}
https://api.github.com/users/gaguilar/subscriptions
User
https://api.github.com/users/gaguilar
public
Hi,

First of all, this library is really cool! Thanks for putting all of this together! 

This PR contains the [Linguistic Code-switching Evaluation (LinCE) benchmark](https://ritual.uh.edu/lince). As described in the official website (FAQ):

> 1. Why do we need LinCE?
>LinCE brings 10 code-switching datasets together for 4 tasks and 4 language pairs with 5 leaderboards in a single evaluation platform. We examined each dataset and fixed major issues on the partitions (or even define official partitions) with a comprehensive stratification method (see our paper for more details).
>Besides, we believe that online benchmarks like LinCE bring steady research progress and allow to compare state-of-the-art models at the pace of the progress in NLP. We expect to benefit greatly the code-switching community with this benchmark.


The data comes from social media and here's the summary table of tasks per language pair:

| Language Pairs                         | LID | POS | NER | SA |
|----------------------------------------|-----|-----|-----|----|
| Spanish-English                        | ✅  | ✅  | ✅  | ✅ |
| Hindi-English                          | ✅  | ✅  | ✅  |    |
| Modern Standard Arabic-Egyptian Arabic | ✅  |     | ✅  |    |
| Nepali-English                         | ✅  |     |     |    |

The tasks are as follows:
* LID: token-level language identification
* POS: part-of-speech tagging
* NER: named entity recognition
* SA: sentiment analysis

With the exception of MSA-EA, the rest of the datasets contain token-level LID labels.

## Usage

For Spanish-English LID, we can load the data as follows:
```
import nlp

data = nlp.load_dataset('./datasets/lince/lince.py', 'lid_spaeng')

for split in data:
    print(data[split])
```

Here's the output:
```
Dataset(schema: {'idx': 'int32', 'tokens': 'list<item: string>', 'lid': 'list<item: string>'}, num_rows: 21030)
Dataset(schema: {'idx': 'int32', 'tokens': 'list<item: string>', 'lid': 'list<item: string>'}, num_rows: 3332)
Dataset(schema: {'idx': 'int32', 'tokens': 'list<item: string>', 'lid': 'list<item: string>'}, num_rows: 8289)
```

Here's the list of shortcut names for every dataset available in LinCE:
* `lid_spaeng`
* `lid_hineng`
* `lid_nepeng`
* `lid_msaea`
* `pos_spaeng`
* `pos_hineng`
* `ner_spaeng`
* `ner_hineng`
* `ner_msaea`
* `sa_spaeng`


All the numbers match with Table 3 in the LinCE [paper](https://www.aclweb.org/anthology/2020.lrec-1.223.pdf). Also, note that the MSA-EA datasets use the Persian script while the other datasets use the Roman script.


## Features

Here is how the features look in the case of language identification (LID) tasks:

| LID Feature          | Type          | Description                               |
|----------------------|---------------|-------------------------------------------|
| `idx`                | `int`       | Dataset index of current sentence         |
| `tokens`             | `list<str>` | List of tokens (string) of a sentence     |
| `lid`                | `list<str>` | List of LID labels (string) of a sentence |

For part-of-speech (POS) tagging:

| POS Feature          | Type          | Description                               |
|----------------------|---------------|-------------------------------------------|
| `idx`                | `int`       | Dataset index of current sentence         |
| `tokens`             | `list<str>` | List of tokens (string) of a sentence     |
| `lid`                | `list<str>` | List of LID labels (string) of a sentence |
| `pos`                | `list<str>` | List of POS tags (string) of a sentence   |

For named entity recognition (NER):

| NER Feature          | Type          | Description                               |
|----------------------|---------------|-------------------------------------------|
| `idx`                | `int`       | Dataset index of current sentence         |
| `tokens`             | `list<str>` | List of tokens (string) of a sentence     |
| `lid`                | `list<str>` | List of LID labels (string) of a sentence |
| `ner`                | `list<str>` | List of NER labels (string) of a sentence |

**NOTE**: the MSA-EA NER dataset does not contain the `lid` feature.

For sentiment analysis (SA):

| SA Feature          | Type        | Description                               |
|---------------------|-------------|-------------------------------------------|
| `idx`               | `int`       | Dataset index of current sentence         |
| `tokens`            | `list<str>` | List of tokens (string) of a sentence     |
| `lid`               | `list<str>` | List of LID labels (string) of a sentence |
| `sa`                | `str`       | Sentiment label (string) of a sentence    |




