[Testing] Improved testing structure
MEMBER
Awesome!
Let's have this in the doc at the end :-)
2020-05-07T22:07:19Z
https://github.com/huggingface/datasets/pull/51#issuecomment-625521252
https://api.github.com/repos/huggingface/datasets/issues/51
MDEyOklzc3VlQ29tbWVudDYyNTUyMTI1Mg==
https://api.github.com/repos/huggingface/datasets/issues/comments/625521252/reactions
2020-05-07T22:07:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/625521252
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
This PR refactors the test design a bit and puts the mock download manager in the `utils` files as it is just a test helper class.

as @mariamabarham pointed out, creating a dummy folder structure can be quite hard to grasp.
This PR tries to change that to some extent.

It follows the following logic for the `dummy` folder structure now:
1.) The data bulider has no config -> the  `dummy` folder structure is:
`dummy/<version>/dummy_data.zip`
2) The data builder has >= 1 configs -> the `dummy` folder structure is: 
`dummy/<config_name_1>/<version>/dummy_data.zip`
`dummy/<config_name_2>/<version>/dummy_data.zip`

Now, the difficult part is how to create the `dummy_data.zip` file. There are two cases:
A) The `data_urs` parameter inserted into the `download_and_extract` fn is a **string**:
-> the `dummy_data.zip` file zips the folder: 
`dummy_data/<relative_path_of_folder_structure_of_url>`
B) The `data_urs` parameter inserted into the `download_and_extract` fn is a **dict**:
-> the `dummy_data.zip` file zips the folder: 
`dummy_data/<relative_path_of_folder_structure_of_url_behind _key_1>`
`dummy_data/<relative_path_of_folder_structure_of_url_behind _key_2>`

By relative folder structure I mean `url_path.split('./')[-1]`. As an example the dataset **xquad** by deepmind has the following url path behind the key `de`: `https://github.com/deepmind/xquad/blob/master/xquad.de.json` 
-> This means that the relative url path should be `xquad.de.json`.


@mariamabarham B) is a change from how is was before and I think is makes more sense. 
While before the `dummy_data.zip` file for xquad with config `de` looked like:
`dummy_data/de` it would now look like `dummy_data/xquad.de.json`. I think this is better and easier to understand. 

Therefore there are currently 6 tests that would have to have changed their dummy folder structure, but which can easily be done (30min). 

I also added a function: `print_dummy_data_folder_structure` that prints out the expected structures when testing which should be quite helpful.