Better DuplicateKeysError error to help the user debug the issue
NONE
excuse me, my `datasets` version is `2.2.2`, but I also just see the error info like 
```
DuplicatedKeysError: FAILURE TO GENERATE DATASET !
Found duplicate Key: 0
Keys should be unique and deterministic in nature
```
2022-05-21T11:55:42Z
https://github.com/huggingface/datasets/issues/2556#issuecomment-1133607102
https://api.github.com/repos/huggingface/datasets/issues/2556
IC_kwDODunzps5DkXi-
https://api.github.com/repos/huggingface/datasets/issues/comments/1133607102/reactions
2022-05-21T11:55:42Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1133607102
https://avatars.githubusercontent.com/u/30597946?v=4
https://api.github.com/users/SupritYoung/events{/privacy}
https://api.github.com/users/SupritYoung/followers
https://api.github.com/users/SupritYoung/following{/other_user}
https://api.github.com/users/SupritYoung/gists{/gist_id}
https://github.com/SupritYoung
SupritYoung
MDQ6VXNlcjMwNTk3OTQ2
https://api.github.com/users/SupritYoung/orgs
https://api.github.com/users/SupritYoung/received_events
https://api.github.com/users/SupritYoung/repos
https://api.github.com/users/SupritYoung/starred{/owner}{/repo}
https://api.github.com/users/SupritYoung/subscriptions
User
https://api.github.com/users/SupritYoung
public
MEMBER
Hi ! for which dataset do you have this error ?

Also note that this issue is just about improving the error message, which is not very friendly x)
2022-05-25T16:54:24Z
https://github.com/huggingface/datasets/issues/2556#issuecomment-1137550843
https://api.github.com/repos/huggingface/datasets/issues/2556
IC_kwDODunzps5DzaX7
https://api.github.com/repos/huggingface/datasets/issues/comments/1137550843/reactions
2022-05-25T16:54:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1137550843
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
@lhoestq  I would like to take a hit at improving the error message. Will open a draft PR and will reach out to you for review

2022-06-17T19:15:35Z
https://github.com/huggingface/datasets/issues/2556#issuecomment-1159162847
https://api.github.com/repos/huggingface/datasets/issues/2556
IC_kwDODunzps5FF2vf
https://api.github.com/repos/huggingface/datasets/issues/comments/1159162847/reactions
2022-06-17T19:15:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1159162847
https://avatars.githubusercontent.com/u/20517962?v=4
https://api.github.com/users/VijayKalmath/events{/privacy}
https://api.github.com/users/VijayKalmath/followers
https://api.github.com/users/VijayKalmath/following{/other_user}
https://api.github.com/users/VijayKalmath/gists{/gist_id}
https://github.com/VijayKalmath
VijayKalmath
MDQ6VXNlcjIwNTE3OTYy
https://api.github.com/users/VijayKalmath/orgs
https://api.github.com/users/VijayKalmath/received_events
https://api.github.com/users/VijayKalmath/repos
https://api.github.com/users/VijayKalmath/starred{/owner}{/repo}
https://api.github.com/users/VijayKalmath/subscriptions
User
https://api.github.com/users/VijayKalmath
public
CONTRIBUTOR
> DuplicateKeysError: both 42th and 1337th examples have the same keys `48`.

@lhoestq  when you mention 42th and 1337th in the above case , are these values the examples' "id" or are they the examples' index ? 
2022-06-17T20:12:51Z
https://github.com/huggingface/datasets/issues/2556#issuecomment-1159197746
https://api.github.com/repos/huggingface/datasets/issues/2556
IC_kwDODunzps5FF_Qy
https://api.github.com/repos/huggingface/datasets/issues/comments/1159197746/reactions
2022-06-17T20:12:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1159197746
https://avatars.githubusercontent.com/u/20517962?v=4
https://api.github.com/users/VijayKalmath/events{/privacy}
https://api.github.com/users/VijayKalmath/followers
https://api.github.com/users/VijayKalmath/following{/other_user}
https://api.github.com/users/VijayKalmath/gists{/gist_id}
https://github.com/VijayKalmath
VijayKalmath
MDQ6VXNlcjIwNTE3OTYy
https://api.github.com/users/VijayKalmath/orgs
https://api.github.com/users/VijayKalmath/received_events
https://api.github.com/users/VijayKalmath/repos
https://api.github.com/users/VijayKalmath/starred{/owner}{/repo}
https://api.github.com/users/VijayKalmath/subscriptions
User
https://api.github.com/users/VijayKalmath
public
MEMBER
Hi ! Thanks @VijayKalmath :)

In the general case, examples don't have an "id" field, so I think it should correspond to the index
2022-06-20T09:39:59Z
https://github.com/huggingface/datasets/issues/2556#issuecomment-1160215672
https://api.github.com/repos/huggingface/datasets/issues/2556
IC_kwDODunzps5FJ3x4
https://api.github.com/repos/huggingface/datasets/issues/comments/1160215672/reactions
2022-06-20T09:39:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1160215672
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
@lhoestq  , I have opened a draft PR for this Issue. 

I wanted to check with you if there is a way to get `<path/to/the/dataset/script>` currently or do I need to add extra code to find that. 

If I need to find the script , I can assume that the generator function will always be in `datasets/{dataset_name}/{dataset_name}.py`. 
2022-06-22T21:06:06Z
https://github.com/huggingface/datasets/issues/2556#issuecomment-1163600256
https://api.github.com/repos/huggingface/datasets/issues/2556
IC_kwDODunzps5FWyGA
https://api.github.com/repos/huggingface/datasets/issues/comments/1163600256/reactions
2022-06-22T21:06:06Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1163600256
https://avatars.githubusercontent.com/u/20517962?v=4
https://api.github.com/users/VijayKalmath/events{/privacy}
https://api.github.com/users/VijayKalmath/followers
https://api.github.com/users/VijayKalmath/following{/other_user}
https://api.github.com/users/VijayKalmath/gists{/gist_id}
https://github.com/VijayKalmath
VijayKalmath
MDQ6VXNlcjIwNTE3OTYy
https://api.github.com/users/VijayKalmath/orgs
https://api.github.com/users/VijayKalmath/received_events
https://api.github.com/users/VijayKalmath/repos
https://api.github.com/users/VijayKalmath/starred{/owner}{/repo}
https://api.github.com/users/VijayKalmath/subscriptions
User
https://api.github.com/users/VijayKalmath
public
MEMBER
Thanks !

> I wanted to check with you if there is a way to get <path/to/the/dataset/script> currently or do I need to add extra code to find that.

You don't have access to this info inside the ArrowWriter unfortunately. This info is available in builder.py in the DatasetBuilder code that uses the ArrowWriter though, maybe a try-catch there can do the job
2022-06-23T13:29:31Z
https://github.com/huggingface/datasets/issues/2556#issuecomment-1164410794
https://api.github.com/repos/huggingface/datasets/issues/2556
IC_kwDODunzps5FZ3-q
https://api.github.com/repos/huggingface/datasets/issues/comments/1164410794/reactions
2022-06-23T13:29:31Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1164410794
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
As mentioned in https://github.com/huggingface/datasets/issues/2552 it would be nice to improve the error message when a dataset fails to build because there are duplicate example keys.

The current one is
```python
datasets.keyhash.DuplicatedKeysError: FAILURE TO GENERATE DATASET !
Found duplicate Key: 48
Keys should be unique and deterministic in nature
```

and we could have something that guides the user to debugging the issue:
```python
DuplicateKeysError: both 42th and 1337th examples have the same keys `48`.
Please fix the dataset script at <path/to/the/dataset/script>
```