Take namespace into account in caching
COLLABORATOR
We might have collisions if a username and a dataset_name are the same. Maybe instead serialize the dataset name by replacing `/` with some string, eg `__SLASH__`, that will hopefully never appear in a dataset or user name (it's what I did in https://github.com/huggingface/datasets-preview-backend/blob/master/benchmark/scripts/serialize.py. That way, all the datasets are one-level deep directories
2021-09-17T17:32:33Z
https://github.com/huggingface/datasets/pull/2938#issuecomment-921964930
https://api.github.com/repos/huggingface/datasets/issues/2938
IC_kwDODunzps429BGC
https://api.github.com/repos/huggingface/datasets/issues/comments/921964930/reactions
2021-09-17T17:33:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/921964930
https://avatars.githubusercontent.com/u/1676121?v=4
https://api.github.com/users/severo/events{/privacy}
https://api.github.com/users/severo/followers
https://api.github.com/users/severo/following{/other_user}
https://api.github.com/users/severo/gists{/gist_id}
https://github.com/severo
severo
MDQ6VXNlcjE2NzYxMjE=
https://api.github.com/users/severo/orgs
https://api.github.com/users/severo/received_events
https://api.github.com/users/severo/repos
https://api.github.com/users/severo/starred{/owner}{/repo}
https://api.github.com/users/severo/subscriptions
User
https://api.github.com/users/severo
public
MEMBER
IIRC we enforce that no repo id or username can contain `___` (exactly 3 underscores) specifically for this reason, so you can use that string (that we use in other projects)

cc @Pierrci 
2021-09-18T06:21:24Z
https://github.com/huggingface/datasets/pull/2938#issuecomment-922212168
https://api.github.com/repos/huggingface/datasets/issues/2938
IC_kwDODunzps4299dI
https://api.github.com/repos/huggingface/datasets/issues/comments/922212168/reactions
2021-09-18T06:21:30Z
https://api.github.com/repos/huggingface/datasets/issues/comments/922212168
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
COLLABORATOR
> IIRC we enforce that no repo id or username can contain ___ (exactly 3 underscores) specifically for this reason, so you can use that string (that we use in other projects)

out of curiosity: where is it enforced?
2021-09-20T07:27:53Z
https://github.com/huggingface/datasets/pull/2938#issuecomment-922691905
https://api.github.com/repos/huggingface/datasets/issues/2938
IC_kwDODunzps42_ylB
https://api.github.com/repos/huggingface/datasets/issues/comments/922691905/reactions
2021-09-20T07:27:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/922691905
https://avatars.githubusercontent.com/u/1676121?v=4
https://api.github.com/users/severo/events{/privacy}
https://api.github.com/users/severo/followers
https://api.github.com/users/severo/following{/other_user}
https://api.github.com/users/severo/gists{/gist_id}
https://github.com/severo
severo
MDQ6VXNlcjE2NzYxMjE=
https://api.github.com/users/severo/orgs
https://api.github.com/users/severo/received_events
https://api.github.com/users/severo/repos
https://api.github.com/users/severo/starred{/owner}{/repo}
https://api.github.com/users/severo/subscriptions
User
https://api.github.com/users/severo
public
MEMBER
> where is it enforced?

Nowhere yet but we should :) feel free to track in internal tracker and/or implement, as this will be useful in the future
2021-09-20T08:20:09Z
https://github.com/huggingface/datasets/pull/2938#issuecomment-922720356
https://api.github.com/repos/huggingface/datasets/issues/2938
IC_kwDODunzps42_5hk
https://api.github.com/repos/huggingface/datasets/issues/comments/922720356/reactions
2021-09-20T08:20:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/922720356
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
MEMBER
Thanks for the trick, I'm doing the change :)
We can use
`~/.cache/huggingface/datasets/username___dataset_name` for the data
`~/.cache/huggingface/modules/datasets_modules/datasets/username___dataset_name` for the python files
2021-09-21T15:42:04Z
https://github.com/huggingface/datasets/pull/2938#issuecomment-924112677
https://api.github.com/repos/huggingface/datasets/issues/2938
IC_kwDODunzps43FNcl
https://api.github.com/repos/huggingface/datasets/issues/comments/924112677/reactions
2021-09-21T15:42:04Z
https://api.github.com/repos/huggingface/datasets/issues/comments/924112677
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Merging, though it will have to be integrated again the refactor at #2986
2021-09-29T13:01:27Z
https://github.com/huggingface/datasets/pull/2938#issuecomment-930153176
https://api.github.com/repos/huggingface/datasets/issues/2938
IC_kwDODunzps43cQLY
https://api.github.com/repos/huggingface/datasets/issues/comments/930153176/reactions
2021-09-29T13:01:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/930153176
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
COLLABORATOR
@lhoestq we changed a bit the naming policy on the Hub, and the substring '--' is now forbidden, which makes it available for serializing the repo names (`namespace/repo` -> `namespace--repo`). See https://github.com/huggingface/moon-landing/pull/1657 and https://github.com/huggingface/huggingface_hub/pull/545
2021-12-17T10:52:17Z
https://github.com/huggingface/datasets/pull/2938#issuecomment-996624442
https://api.github.com/repos/huggingface/datasets/issues/2938
IC_kwDODunzps47Z0g6
https://api.github.com/repos/huggingface/datasets/issues/comments/996624442/reactions
2021-12-17T10:52:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/996624442
https://avatars.githubusercontent.com/u/1676121?v=4
https://api.github.com/users/severo/events{/privacy}
https://api.github.com/users/severo/followers
https://api.github.com/users/severo/following{/other_user}
https://api.github.com/users/severo/gists{/gist_id}
https://github.com/severo
severo
MDQ6VXNlcjE2NzYxMjE=
https://api.github.com/users/severo/orgs
https://api.github.com/users/severo/received_events
https://api.github.com/users/severo/repos
https://api.github.com/users/severo/starred{/owner}{/repo}
https://api.github.com/users/severo/subscriptions
User
https://api.github.com/users/severo
public
Loading a dataset "username/dataset_name" hosted by a user on the hub used to cache the dataset only taking into account the dataset name, and ignorign the username. Because of this, if a user later loads "dataset_name" without specifying the username, it would reload the dataset from the cache instead of failing.

I changed the dataset cache and module cache mechanism to include the username in the name of the cache directory that is used:
<s>
`~/.cache/huggingface/datasets/username/dataset_name`  for the data
`~/.cache/huggingface/modules/datasets_modules/datasets/username/dataset_name` for the python files
</s>
EDIT: actually using three underscores:
`~/.cache/huggingface/datasets/username___dataset_name` for the data
`~/.cache/huggingface/modules/datasets_modules/datasets/username___dataset_name` for the python files

This PR should fix the issue https://github.com/huggingface/datasets/issues/2842

cc @stas00 