dataset loading logger level
MEMBER
These warnings are showed when there's a call to `.map` to say to the user that a dataset is reloaded from the cache instead of being recomputed.
They are warnings since we want to make sure the users know that it's not recomputed.
2021-02-25T19:47:48Z
https://github.com/huggingface/datasets/issues/1948#issuecomment-786156694
https://api.github.com/repos/huggingface/datasets/issues/1948
MDEyOklzc3VlQ29tbWVudDc4NjE1NjY5NA==
https://api.github.com/repos/huggingface/datasets/issues/comments/786156694/reactions
2021-02-25T19:47:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/786156694
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Thank you for explaining the intention, @lhoestq 

1. Could it be then made more human-friendly? Currently the hex gibberish tells me nothing of what's really going on. e.g. the following is instructive, IMHO:

```
WARNING: wmt16/ro-en/train dataset was loaded from cache instead of being recomputed
WARNING: wmt16/ro-en/validation dataset was loaded from cache instead of being recomputed
WARNING: wmt16/ro-en/test dataset was loaded from cache instead of being recomputed
```
note that it removes the not so useful hex info and tells the user instead which split it's referring to - but probably no harm in keeping the path if it helps the debug. But the key is that now the warning is telling me what it is it's warning me about.
```
Warning:Loading cache path
```
on the other hand isn't telling what it is warning about.

And I still suggest this is INFO level, otherwise you need to turn all 'using cache' statements to WARNING to be consistent. The user is most likely well aware the cache is used for models, etc. So this feels very similar.

2. Should there be a way for a user to void warranty by having a flag - `I know I'm expecting the cached version to load if it's available - please do not warn me about it=True`

To explain the need: Warnings are a problem, they constantly take attention away because they could be the harbinger of a problem. Therefore I prefer not to have any warnings in the log, and if I get any I usually try to deal with those so that my log is clean. 

It's less of an issue for somebody doing long runs. It's a huge issue for someone who does a new run every few minutes and on the lookout for any potential problems which is what I have been doing a lot of integrating DeepSpeed and other things. And since there are already problems to deal with during the integration it's nice to have a clean log to start with. 

I hope my need is not unreasonable and I was able to explain it adequately. 

Thank you.
2021-02-25T20:00:59Z
https://github.com/huggingface/datasets/issues/1948#issuecomment-786163943
https://api.github.com/repos/huggingface/datasets/issues/1948
MDEyOklzc3VlQ29tbWVudDc4NjE2Mzk0Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/786163943/reactions
2021-02-26T01:27:04Z
https://api.github.com/repos/huggingface/datasets/issues/comments/786163943
https://avatars.githubusercontent.com/u/10676103?v=4
https://api.github.com/users/stas00/events{/privacy}
https://api.github.com/users/stas00/followers
https://api.github.com/users/stas00/following{/other_user}
https://api.github.com/users/stas00/gists{/gist_id}
https://github.com/stas00
stas00
MDQ6VXNlcjEwNjc2MTAz
https://api.github.com/users/stas00/orgs
https://api.github.com/users/stas00/received_events
https://api.github.com/users/stas00/repos
https://api.github.com/users/stas00/starred{/owner}{/repo}
https://api.github.com/users/stas00/subscriptions
User
https://api.github.com/users/stas00
public
NONE
Hey, any news about the issue? So many warnings when I'm really ok with the dataset not being recomputed :)
2022-02-08T12:47:19Z
https://github.com/huggingface/datasets/issues/1948#issuecomment-1032571334
https://api.github.com/repos/huggingface/datasets/issues/1948
IC_kwDODunzps49i8nG
https://api.github.com/repos/huggingface/datasets/issues/comments/1032571334/reactions
2022-02-08T12:47:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1032571334
https://avatars.githubusercontent.com/u/57996478?v=4
https://api.github.com/users/yuvalkirstain/events{/privacy}
https://api.github.com/users/yuvalkirstain/followers
https://api.github.com/users/yuvalkirstain/following{/other_user}
https://api.github.com/users/yuvalkirstain/gists{/gist_id}
https://github.com/yuvalkirstain
yuvalkirstain
MDQ6VXNlcjU3OTk2NDc4
https://api.github.com/users/yuvalkirstain/orgs
https://api.github.com/users/yuvalkirstain/received_events
https://api.github.com/users/yuvalkirstain/repos
https://api.github.com/users/yuvalkirstain/starred{/owner}{/repo}
https://api.github.com/users/yuvalkirstain/subscriptions
User
https://api.github.com/users/yuvalkirstain
public
on master I get this with `--dataset_name wmt16 --dataset_config ro-en`:

```
WARNING:datasets.arrow_dataset:Loading cached processed dataset at /home/stas/.cache/huggingface/datasets/wmt16/ro-en/1.0.0/9dc00622c30446e99c4c63d12a484ea4fb653f2f37c867d6edcec839d7eae50f/cache-2e01bead8cf42e26.arrow
WARNING:datasets.arrow_dataset:Loading cached processed dataset at /home/stas/.cache/huggingface/datasets/wmt16/ro-en/1.0.0/9dc00622c30446e99c4c63d12a484ea4fb653f2f37c867d6edcec839d7eae50f/cache-ac3bebaf4f91f776.arrow
WARNING:datasets.arrow_dataset:Loading cached processed dataset at /home/stas/.cache/huggingface/datasets/wmt16/ro-en/1.0.0/9dc00622c30446e99c4c63d12a484ea4fb653f2f37c867d6edcec839d7eae50f/cache-810c3e61259d73a9.arrow
```

why are those WARNINGs? Should be INFO, no?

warnings should only be used when a user needs to pay attention to something, this is just informative - I'd even say it should be DEBUG, but definitely not WARNING.

Thank you.
