Run CI tests against non-production server
NONE
Hey @albertvillanova could you provide more context, including extracts from the discussion we had ?

Let's ping @Pierrci @julien-c and @n1t0 for their opinion about that
2021-09-28T12:50:24Z
https://github.com/huggingface/datasets/issues/2978#issuecomment-929178592
https://api.github.com/repos/huggingface/datasets/issues/2978
IC_kwDODunzps43YiPg
https://api.github.com/repos/huggingface/datasets/issues/comments/929178592/reactions
2021-09-28T12:50:33Z
https://api.github.com/repos/huggingface/datasets/issues/comments/929178592
https://avatars.githubusercontent.com/u/76620?v=4
https://api.github.com/users/elishowk/events{/privacy}
https://api.github.com/users/elishowk/followers
https://api.github.com/users/elishowk/following{/other_user}
https://api.github.com/users/elishowk/gists{/gist_id}
https://github.com/elishowk
elishowk
MDQ6VXNlcjc2NjIw
https://api.github.com/users/elishowk/orgs
https://api.github.com/users/elishowk/received_events
https://api.github.com/users/elishowk/repos
https://api.github.com/users/elishowk/starred{/owner}{/repo}
https://api.github.com/users/elishowk/subscriptions
User
https://api.github.com/users/elishowk
public
NONE
@julien-c increased the huggingface.co production workers in order to see if it solve [the 502 you had this morning](https://app.circleci.com/pipelines/github/huggingface/datasets/7843/workflows/fc83fa32-18f5-4dc3-9e2f-ba277ae1af74)

For the decision process: be aware that moon-staging does not have persistent repos (they are deleted regularly). as a consequence, **if the moon-staging solution is validated**, you should consider a way to keep the repository that are loaded in tests. These are the ones I found : https://github.com/huggingface/datasets/blob/d488db2f64f312f88f72bbc57a09b7eddb329182/tests/test_load.py and https://github.com/huggingface/datasets/blob/40773111c3e7db8a992fa1c48af32d900a1018d6/tests/test_streaming_download_manager.
2021-09-28T13:21:03Z
https://github.com/huggingface/datasets/issues/2978#issuecomment-929204236
https://api.github.com/repos/huggingface/datasets/issues/2978
IC_kwDODunzps43YogM
https://api.github.com/repos/huggingface/datasets/issues/comments/929204236/reactions
2021-09-28T15:23:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/929204236
https://avatars.githubusercontent.com/u/76620?v=4
https://api.github.com/users/elishowk/events{/privacy}
https://api.github.com/users/elishowk/followers
https://api.github.com/users/elishowk/following{/other_user}
https://api.github.com/users/elishowk/gists{/gist_id}
https://github.com/elishowk
elishowk
MDQ6VXNlcjc2NjIw
https://api.github.com/users/elishowk/orgs
https://api.github.com/users/elishowk/received_events
https://api.github.com/users/elishowk/repos
https://api.github.com/users/elishowk/starred{/owner}{/repo}
https://api.github.com/users/elishowk/subscriptions
User
https://api.github.com/users/elishowk
public
Currently, the CI test suite performs requests to the HF production server.

As discussed with @elishowk, we should refactor our tests to use the HF staging server instead, like `huggingface_hub` and `transformers`.