Adding The Microsoft Terminology Collection dataset.
CONTRIBUTOR
I have to manually copy a dataset_infos.json file from other dataset and modify it since the `datasets-cli` isn't able to handle manually downloaded datasets yet (as far as I know).
2020-12-03T00:07:47Z
https://github.com/huggingface/datasets/pull/973#issuecomment-737571476
https://api.github.com/repos/huggingface/datasets/issues/973
MDEyOklzc3VlQ29tbWVudDczNzU3MTQ3Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/737571476/reactions
2020-12-03T00:07:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/737571476
https://avatars.githubusercontent.com/u/7915719?v=4
https://api.github.com/users/leoxzhao/events{/privacy}
https://api.github.com/users/leoxzhao/followers
https://api.github.com/users/leoxzhao/following{/other_user}
https://api.github.com/users/leoxzhao/gists{/gist_id}
https://github.com/leoxzhao
leoxzhao
MDQ6VXNlcjc5MTU3MTk=
https://api.github.com/users/leoxzhao/orgs
https://api.github.com/users/leoxzhao/received_events
https://api.github.com/users/leoxzhao/repos
https://api.github.com/users/leoxzhao/starred{/owner}{/repo}
https://api.github.com/users/leoxzhao/subscriptions
User
https://api.github.com/users/leoxzhao
public
MEMBER
you can generate the dataset_infos.json file even for dataset with manual data
To do so just specify `--data_dir <path/to/the/folder/containing/the/manual/data>`
2020-12-03T09:43:05Z
https://github.com/huggingface/datasets/pull/973#issuecomment-737793017
https://api.github.com/repos/huggingface/datasets/issues/973
MDEyOklzc3VlQ29tbWVudDczNzc5MzAxNw==
https://api.github.com/repos/huggingface/datasets/issues/comments/737793017/reactions
2020-12-03T09:43:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/737793017
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Also, dummy_data seems having difficulty to handle manually downloaded datasets. `python datasets-cli dummy_data datasets/ms_terms  --data_dir ...` reported `error: unrecognized arguments: --data_dir` error. Without `--data_dir`, it reported this error:
```
Dataset ms_terms with config BuilderConfig(name='ms_terms-full', version=1.0.0, data_dir=None, data_files=None, description='...\n') seems to already open files in the method `_split_generators(...)`. You might consider to instead only open files in the method `_generate_examples(...)` instead. If this is not possible the dummy data has to be created with less guidance. Make sure you create the file None.
Traceback (most recent call last):
  File "datasets-cli", line 36, in <module>
    service.run()
  File "/Users/lzhao/Downloads/huggingface/datasets/src/datasets/commands/dummy_data.py", line 326, in run
    dataset_builder=dataset_builder, mock_dl_manager=mock_dl_manager
  File "/Users/lzhao/Downloads/huggingface/datasets/src/datasets/commands/dummy_data.py", line 406, in _print_dummy_data_instructions
    for split in generator_splits:
UnboundLocalError: local variable 'generator_splits' referenced before assignment
```
2020-12-03T15:49:21Z
https://github.com/huggingface/datasets/pull/973#issuecomment-738092602
https://api.github.com/repos/huggingface/datasets/issues/973
MDEyOklzc3VlQ29tbWVudDczODA5MjYwMg==
https://api.github.com/repos/huggingface/datasets/issues/comments/738092602/reactions
2020-12-03T15:49:21Z
https://api.github.com/repos/huggingface/datasets/issues/comments/738092602
https://avatars.githubusercontent.com/u/7915719?v=4
https://api.github.com/users/leoxzhao/events{/privacy}
https://api.github.com/users/leoxzhao/followers
https://api.github.com/users/leoxzhao/following{/other_user}
https://api.github.com/users/leoxzhao/gists{/gist_id}
https://github.com/leoxzhao
leoxzhao
MDQ6VXNlcjc5MTU3MTk=
https://api.github.com/users/leoxzhao/orgs
https://api.github.com/users/leoxzhao/received_events
https://api.github.com/users/leoxzhao/repos
https://api.github.com/users/leoxzhao/starred{/owner}{/repo}
https://api.github.com/users/leoxzhao/subscriptions
User
https://api.github.com/users/leoxzhao
public
MEMBER
Oh yes `--data_dir` seems to only be supported for the `datasets_cli test` command. Sorry about that.

Can you try to build the dummy_data.zip file manually ?

It has to be inside `./datasets/ms_terms/dummy/ms_terms-full/1.0.0`.
Inside this folder, please create a folder `dummy_data` that contains a dummy file `MicrosoftTermCollection.tbx` (with just a few examples in it). Then you can zip the `dummy_data` folder to `dummy_data.zip`

Then you can check if it worked using the command
```
pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_ms_terms
```

Feel free to use some debugging print statements in your script if it doesn't work first try to see what `dl_manager.manual_dir` ends up being and also `path_to_manual_file`.

Feel free to ping me if you have other questions
2020-12-03T19:01:55Z
https://github.com/huggingface/datasets/pull/973#issuecomment-738224928
https://api.github.com/repos/huggingface/datasets/issues/973
MDEyOklzc3VlQ29tbWVudDczODIyNDkyOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/738224928/reactions
2020-12-03T19:01:55Z
https://api.github.com/repos/huggingface/datasets/issues/comments/738224928
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
`pytest tests/test_dataset_common.py::LocalDatasetTest::test_load_dataset_ms_terms` gave `1 passed, 4 warnings in 8.13s`. Existing datasets, like `wikihow`, and `newsroom`, also report 4 warnings. So, I guess that is not related to this dataset.
2020-12-04T01:29:44Z
https://github.com/huggingface/datasets/pull/973#issuecomment-738493602
https://api.github.com/repos/huggingface/datasets/issues/973
MDEyOklzc3VlQ29tbWVudDczODQ5MzYwMg==
https://api.github.com/repos/huggingface/datasets/issues/comments/738493602/reactions
2020-12-04T01:29:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/738493602
https://avatars.githubusercontent.com/u/7915719?v=4
https://api.github.com/users/leoxzhao/events{/privacy}
https://api.github.com/users/leoxzhao/followers
https://api.github.com/users/leoxzhao/following{/other_user}
https://api.github.com/users/leoxzhao/gists{/gist_id}
https://github.com/leoxzhao
leoxzhao
MDQ6VXNlcjc5MTU3MTk=
https://api.github.com/users/leoxzhao/orgs
https://api.github.com/users/leoxzhao/received_events
https://api.github.com/users/leoxzhao/repos
https://api.github.com/users/leoxzhao/starred{/owner}{/repo}
https://api.github.com/users/leoxzhao/subscriptions
User
https://api.github.com/users/leoxzhao
public
MEMBER
Could you run `make style` before we merge @leoxzhao ?
2020-12-04T11:17:19Z
https://github.com/huggingface/datasets/pull/973#issuecomment-738728250
https://api.github.com/repos/huggingface/datasets/issues/973
MDEyOklzc3VlQ29tbWVudDczODcyODI1MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/738728250/reactions
2020-12-04T11:17:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/738728250
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
the other errors are fixed on master so it's fine
2020-12-04T13:21:33Z
https://github.com/huggingface/datasets/pull/973#issuecomment-738780557
https://api.github.com/repos/huggingface/datasets/issues/973
MDEyOklzc3VlQ29tbWVudDczODc4MDU1Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/738780557/reactions
2020-12-04T13:21:33Z
https://api.github.com/repos/huggingface/datasets/issues/comments/738780557
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
> Could you run `make style` before we merge @leoxzhao ?

Sure thing. Done. Thanks Quentin. I have other datasets in mind. All of which requires manual download. This process is very helpful
2020-12-04T14:43:16Z
https://github.com/huggingface/datasets/pull/973#issuecomment-738821773
https://api.github.com/repos/huggingface/datasets/issues/973
MDEyOklzc3VlQ29tbWVudDczODgyMTc3Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/738821773/reactions
2020-12-04T14:43:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/738821773
https://avatars.githubusercontent.com/u/7915719?v=4
https://api.github.com/users/leoxzhao/events{/privacy}
https://api.github.com/users/leoxzhao/followers
https://api.github.com/users/leoxzhao/following{/other_user}
https://api.github.com/users/leoxzhao/gists{/gist_id}
https://github.com/leoxzhao
leoxzhao
MDQ6VXNlcjc5MTU3MTk=
https://api.github.com/users/leoxzhao/orgs
https://api.github.com/users/leoxzhao/received_events
https://api.github.com/users/leoxzhao/repos
https://api.github.com/users/leoxzhao/starred{/owner}{/repo}
https://api.github.com/users/leoxzhao/subscriptions
User
https://api.github.com/users/leoxzhao
public
MEMBER
Thank you :) 
2020-12-04T15:11:52Z
https://github.com/huggingface/datasets/pull/973#issuecomment-738837405
https://api.github.com/repos/huggingface/datasets/issues/973
MDEyOklzc3VlQ29tbWVudDczODgzNzQwNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/738837405/reactions
2020-12-04T15:11:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/738837405
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public