YAML integer keys are not preserved Hub server-side
MEMBER
@huggingface/datasets if you agree, I can make the bulk edit on the Hub to fix integer keys into strings.
2023-01-04T08:00:22Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1370589408
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5RsYjg
https://api.github.com/repos/huggingface/datasets/issues/comments/1370589408/reactions
2023-01-04T08:00:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1370589408
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Ok for me, and we can merge (internal) https://github.com/huggingface/moon-landing/pull/4609
2023-01-04T09:15:16Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1370658857
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5Rspgp
https://api.github.com/repos/huggingface/datasets/issues/comments/1370658857/reactions
2023-01-04T09:15:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1370658857
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
MEMBER
FYI there are still 2k+ weekly users on `datasets` 2.6.1 which doesn't support the string label format for class labels. And among those, some are using datasets with class labels like imdb (60 users), conllpp (40), msra_ner (40), peoples_daily_enr (40), weibo_ner (30), conll2003 (20), etc. And renaming to string would break these users code.
2023-01-04T12:41:22Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1370881578
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5Rtf4q
https://api.github.com/repos/huggingface/datasets/issues/comments/1370881578/reactions
2023-01-04T12:42:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1370881578
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
but isn't `datasets 2.6.1` downloading files from the Hub with the corresponding tag? I thought we had something like this before
2023-01-04T12:57:23Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1370898845
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5RtkGd
https://api.github.com/repos/huggingface/datasets/issues/comments/1370898845/reactions
2023-01-04T12:57:23Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1370898845
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
MEMBER
We're using `main` as models do. Some datasets need to be updated from time to time, e.g. when a link to download the data is dead.

But yea a year ago we had those tags, we just ended up not using them
2023-01-04T13:56:54Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1370960191
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5RtzE_
https://api.github.com/repos/huggingface/datasets/issues/comments/1370960191/reactions
2023-01-04T13:56:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1370960191
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
I opened https://github.com/huggingface/datasets/issues/5406 to communicate on this. Let me know what you think, and if it sounds good to you I can pin this issue
2023-01-04T15:10:34Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1371050891
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5RuJOL
https://api.github.com/repos/huggingface/datasets/issues/comments/1371050891/reactions
2023-01-04T15:10:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1371050891
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
So, is it OK to make the bulk edit on the Hub now or should we wait longer? If the latter, how long?
2023-01-05T09:59:37Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1372007275
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5Rxytr
https://api.github.com/repos/huggingface/datasets/issues/comments/1372007275/reactions
2023-01-05T09:59:37Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1372007275
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
I think we can do it. If you want to be extra cautious you can do it for all datasets except imdb and conllpp for now which are actively used by 2.6.1 users. For those two we can keep the YAML like this for some more time, or alternatively use the old dataset_infos.json file
2023-01-05T14:02:32Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1372252420
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5RyukE
https://api.github.com/repos/huggingface/datasets/issues/comments/1372252420/reactions
2023-01-05T14:02:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1372252420
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
The bulk edit of canonical datasets (except imdb and conllpp) is running. 

See e.g.: https://huggingface.co/datasets/acronym_identification/discussions/3

EDITED: 
Done, except for "universal_morphologies", where I get
```
HTTPError: 413 Client Error: Payload Too Large for url: https://huggingface.co/api/validate-yaml
```

Also not done for the datasets missing matadata "dataset_info":
- mc4: https://huggingface.co/datasets/mc4/discussions/3
- the_pile: https://huggingface.co/datasets/the_pile/discussions/6
- timit_asr: https://huggingface.co/datasets/timit_asr/discussions/1
2023-01-25T14:30:49Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1403711916
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5TqvGs
https://api.github.com/repos/huggingface/datasets/issues/comments/1403711916/reactions
2023-01-25T15:07:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1403711916
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Thank you !
2023-01-25T15:27:51Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1403797424
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5TrD-w
https://api.github.com/repos/huggingface/datasets/issues/comments/1403797424/reactions
2023-01-25T15:27:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1403797424
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
@lhoestq, there are 6 community datasets with YAML integer keys in their `dataset_info` `class_label`:
- indonlp/indonlu
- rcds/swiss_judgment_prediction
- Jean-Baptiste/wikiner_fr
- Bingsu/Cat_and_Dog
- taskydata/tasky_or_not
- RCC-MSU/collection3

Maybe we could open a PR on them as well?
2023-01-26T07:26:11Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1404638800
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5TuRZQ
https://api.github.com/repos/huggingface/datasets/issues/comments/1404638800/reactions
2023-01-26T07:26:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1404638800
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Let's do this then:

- [x] [indonlp/indonlu](https://huggingface.co/datasets/indonlp/indonlu/discussions/3)
- [x] rcds/swiss_judgment_prediction
- [x] Jean-Baptiste/wikiner_fr
- [x] Bingsu/Cat_and_Dog -> merged
- [x] taskydata/tasky_or_not (was already using quotes)
- [x] RCC-MSU/collection3

EDIT: all done :)
2023-01-26T10:13:12Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1404792829
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5Tu2_9
https://api.github.com/repos/huggingface/datasets/issues/comments/1404792829/reactions
2023-01-26T10:52:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1404792829
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
@lhoestq I was not asking you to do it, but asking if you agree me to do it... :man_facepalming: 
As I self-assigned this issue... :sweat_smile: 
2023-01-26T10:33:26Z
https://github.com/huggingface/datasets/issues/5275#issuecomment-1404816113
https://api.github.com/repos/huggingface/datasets/issues/5275
IC_kwDODunzps5Tu8rx
https://api.github.com/repos/huggingface/datasets/issues/comments/1404816113/reactions
2023-01-26T10:37:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1404816113
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
After an internal discussion (https://github.com/huggingface/moon-landing/issues/4563):
- YAML integer keys are not preserved server-side: they are transformed to strings
  - See for example this Hub PR: https://huggingface.co/datasets/acronym_identification/discussions/1/files
    - Original:
      ```yaml
      class_label:
        names:
          0: B-long
          1: B-short
      ```
    - Returned by the server:
      ```yaml
      class_label:
        names:
          '0': B-long
          '1': B-short
      ```
- They are planning to enforce only string keys
- Other projects already use interger-transformed-to string keys: e.g. `transformers` models `id2label`: https://huggingface.co/roberta-large-mnli/blob/main/config.json
  ```yaml
  "id2label": {
    "0": "CONTRADICTION",
    "1": "NEUTRAL",
    "2": "ENTAILMENT"
  }
  ``` 

On the other hand, at `datasets` we are currently using YAML integer keys for `dataset_info` `class_label`.

Please note (thanks @lhoestq for pointing out) that previous versions (2.6 and 2.7) of `datasets` need being patched:
```python
In [18]: Features._from_yaml_list([{'dtype': {'class_label': {'names': {'0': 'neg', '1': 'pos'}}}, 'name': 'label'}])                                                                                      
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-18-974f07eea526> in <module>
----> 1 Features._from_yaml_list(ry)

~/Desktop/hf/nlp/src/datasets/features/features.py in _from_yaml_list(cls, yaml_data)
   1743                 raise TypeError(f"Expected a dict or a list but got {type(obj)}: {obj}")
   1744 
-> 1745         return cls.from_dict(from_yaml_inner(yaml_data))
   1746 
   1747     def encode_example(self, example):

~/Desktop/hf/nlp/src/datasets/features/features.py in from_yaml_inner(obj)
   1739             elif isinstance(obj, list):
   1740                 names = [_feature.pop("name") for _feature in obj]
-> 1741                 return {name: from_yaml_inner(_feature) for name, _feature in zip(names, obj)}
   1742             else:
   1743                 raise TypeError(f"Expected a dict or a list but got {type(obj)}: {obj}")

~/Desktop/hf/nlp/src/datasets/features/features.py in <dictcomp>(.0)
   1739             elif isinstance(obj, list):
   1740                 names = [_feature.pop("name") for _feature in obj]
-> 1741                 return {name: from_yaml_inner(_feature) for name, _feature in zip(names, obj)}
   1742             else:
   1743                 raise TypeError(f"Expected a dict or a list but got {type(obj)}: {obj}")

~/Desktop/hf/nlp/src/datasets/features/features.py in from_yaml_inner(obj)
   1734                             return {"_type": snakecase_to_camelcase(obj["dtype"])}
   1735                     else:
-> 1736                         return from_yaml_inner(obj["dtype"])
   1737                 else:
   1738                     return {"_type": snakecase_to_camelcase(_type), **unsimplify(obj)[_type]}

~/Desktop/hf/nlp/src/datasets/features/features.py in from_yaml_inner(obj)
   1736                         return from_yaml_inner(obj["dtype"])
   1737                 else:
-> 1738                     return {"_type": snakecase_to_camelcase(_type), **unsimplify(obj)[_type]}
   1739             elif isinstance(obj, list):
   1740                 names = [_feature.pop("name") for _feature in obj]

~/Desktop/hf/nlp/src/datasets/features/features.py in unsimplify(feature)
   1704             if isinstance(feature.get("class_label"), dict) and isinstance(feature["class_label"].get("names"), dict):
   1705                 label_ids = sorted(feature["class_label"]["names"])
-> 1706                 if label_ids and label_ids != list(range(label_ids[-1] + 1)):
   1707                     raise ValueError(
   1708                         f"ClassLabel expected a value for all label ids [0:{label_ids[-1] + 1}] but some ids are missing."

TypeError: can only concatenate str (not "int") to str
```

TODO:
- [x] Remove YAML integer keys from `dataset_info` metadata
- [x] Make a patch release for affected `datasets` versions: 2.6 and 2.7
- [x] Communicate on the fix
- [x] Wait for adoption
- [x] Bulk edit the Hub to fix this in all canonical datasets