Avoid downloading the whole dataset when only README.me has been touched on hub.
COLLABORATOR
you're right, we're tackling this here: https://github.com/huggingface/dataset-viewer/issues/2757
2024-05-29T20:40:37Z
https://github.com/huggingface/datasets/issues/6929#issuecomment-2138227549
https://api.github.com/repos/huggingface/datasets/issues/6929
IC_kwDODunzps5_csNd
https://api.github.com/repos/huggingface/datasets/issues/comments/2138227549/reactions
2024-05-29T20:40:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2138227549
https://avatars.githubusercontent.com/u/1676121?v=4
https://api.github.com/users/severo/events{/privacy}
https://api.github.com/users/severo/followers
https://api.github.com/users/severo/following{/other_user}
https://api.github.com/users/severo/gists{/gist_id}
https://github.com/severo
severo
MDQ6VXNlcjE2NzYxMjE=
https://api.github.com/users/severo/orgs
https://api.github.com/users/severo/received_events
https://api.github.com/users/severo/repos
https://api.github.com/users/severo/starred{/owner}{/repo}
https://api.github.com/users/severo/subscriptions
User
https://api.github.com/users/severo
public
NONE
@severo : great !
2024-05-29T20:51:55Z
https://github.com/huggingface/datasets/issues/6929#issuecomment-2138242839
https://api.github.com/repos/huggingface/datasets/issues/6929
IC_kwDODunzps5_cv8X
https://api.github.com/repos/huggingface/datasets/issues/comments/2138242839/reactions
2024-05-29T20:51:55Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2138242839
https://avatars.githubusercontent.com/u/73740254?v=4
https://api.github.com/users/zinc75/events{/privacy}
https://api.github.com/users/zinc75/followers
https://api.github.com/users/zinc75/following{/other_user}
https://api.github.com/users/zinc75/gists{/gist_id}
https://github.com/zinc75
zinc75
MDQ6VXNlcjczNzQwMjU0
https://api.github.com/users/zinc75/orgs
https://api.github.com/users/zinc75/received_events
https://api.github.com/users/zinc75/repos
https://api.github.com/users/zinc75/starred{/owner}{/repo}
https://api.github.com/users/zinc75/subscriptions
User
https://api.github.com/users/zinc75
public
### Feature request

`datasets.load_dataset()` triggers a new download of the **whole dataset** when the README.md file has been touched on huggingface hub, even if data files / parquet files are the exact same.

I think the current behaviour of the load_dataset function is triggered whenever a change of the hash of latest commit on huggingface hub, but  is there a clever way to only download again the dataset **if and only if** data is modified ?  

### Motivation

The current behaviour is a waste of network bandwidth / disk space / research time.

### Your contribution

I don't have time to submit a PR, but I hope a simple solution will emerge from this issue ! 