Use tqdm from tqdm_utils
COLLABORATOR
The current CI failure is due to modifications in the dataset script.
2021-07-18T11:56:46Z
https://github.com/huggingface/datasets/pull/2667#issuecomment-882045316
https://api.github.com/repos/huggingface/datasets/issues/2667
IC_kwDODunzps40kvGE
https://api.github.com/repos/huggingface/datasets/issues/comments/882045316/reactions
2021-07-18T11:56:46Z
https://api.github.com/repos/huggingface/datasets/issues/comments/882045316
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
Merging since the CI is only failing because of dataset card issues, which is unrelated to this PR
2021-07-19T17:31:56Z
https://github.com/huggingface/datasets/pull/2667#issuecomment-882729682
https://api.github.com/repos/huggingface/datasets/issues/2667
IC_kwDODunzps40nWLS
https://api.github.com/repos/huggingface/datasets/issues/comments/882729682/reactions
2021-07-19T17:31:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/882729682
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
This PR replaces `tqdm` from the `tqdm` lib with `tqdm` from `datasets.utils.tqdm_utils`. With this change, it's possible to disable progress bars just by calling `disable_progress_bar`. Note this doesn't work on Windows when using multiprocessing due to how global variables are shared between processes. Currently, there is no easy way to disable progress bars in a multiprocess setting on Windows (patching logging with `datasets.utils.logging.get_verbosity = lambda: datasets.utils.logging.NOTSET` doesn't seem to work as well), so adding support for this is a future goal. Additionally, this PR adds a unit ("ba" for batches) to the bar printed by `Dataset.to_json` (this change is motivated by https://github.com/huggingface/datasets/issues/2657).