Use max number of data files to infer module
MEMBER
Cool thanks :) Feel free to merge if it's all good for you
2021-12-14T10:30:01Z
https://github.com/huggingface/datasets/pull/3407#issuecomment-993398861
https://api.github.com/repos/huggingface/datasets/issues/3407
IC_kwDODunzps47NhBN
https://api.github.com/repos/huggingface/datasets/issues/comments/993398861/reactions
2021-12-14T10:30:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/993398861
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
When inferring the module for datasets without script, set a maximum number of iterations over data files.

This PR fixes the issue of taking too long when hundred of data files present.

Please, feel free to agree on both numbers:
```
# Datasets without script
DATA_FILES_MAX_NUMBER = 10
ARCHIVED_DATA_FILES_MAX_NUMBER = 5
```

Fix #3404.