Refactor column mappings for question answering datasets
NONE
_The documentation is not available anymore as the PR was closed or merged._
2022-05-23T09:27:36Z
https://github.com/huggingface/datasets/pull/4391#issuecomment-1134420177
https://api.github.com/repos/huggingface/datasets/issues/4391
IC_kwDODunzps5DneDR
https://api.github.com/repos/huggingface/datasets/issues/comments/1134420177/reactions
2022-05-24T12:57:00Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1134420177
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
MEMBER
> Thanks.
> 
> I have no visibility about this, but if you say it is more useful for AutoTrain this way...

Thanks for the review @albertvillanova ! Yes, I need some way to reconstruct the original column names with a period because that's how they appear after we flatten the nested columns. In any case, we can adjust this later if needed :)
2022-05-23T20:17:12Z
https://github.com/huggingface/datasets/pull/4391#issuecomment-1135098677
https://api.github.com/repos/huggingface/datasets/issues/4391
IC_kwDODunzps5DqDs1
https://api.github.com/repos/huggingface/datasets/issues/comments/1135098677/reactions
2022-05-23T20:17:12Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1135098677
https://avatars.githubusercontent.com/u/26859204?v=4
https://api.github.com/users/lewtun/events{/privacy}
https://api.github.com/users/lewtun/followers
https://api.github.com/users/lewtun/following{/other_user}
https://api.github.com/users/lewtun/gists{/gist_id}
https://github.com/lewtun
lewtun
MDQ6VXNlcjI2ODU5MjA0
https://api.github.com/users/lewtun/orgs
https://api.github.com/users/lewtun/received_events
https://api.github.com/users/lewtun/repos
https://api.github.com/users/lewtun/starred{/owner}{/repo}
https://api.github.com/users/lewtun/subscriptions
User
https://api.github.com/users/lewtun
public
CONTRIBUTOR
Does that mean that we need to change the metadata?
2022-05-24T12:19:07Z
https://github.com/huggingface/datasets/pull/4391#issuecomment-1135848363
https://api.github.com/repos/huggingface/datasets/issues/4391
IC_kwDODunzps5Ds6ur
https://api.github.com/repos/huggingface/datasets/issues/comments/1135848363/reactions
2022-05-24T12:19:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1135848363
https://avatars.githubusercontent.com/u/14205986?v=4
https://api.github.com/users/sashavor/events{/privacy}
https://api.github.com/users/sashavor/followers
https://api.github.com/users/sashavor/following{/other_user}
https://api.github.com/users/sashavor/gists{/gist_id}
https://github.com/sashavor
sashavor
MDQ6VXNlcjE0MjA1OTg2
https://api.github.com/users/sashavor/orgs
https://api.github.com/users/sashavor/received_events
https://api.github.com/users/sashavor/repos
https://api.github.com/users/sashavor/starred{/owner}{/repo}
https://api.github.com/users/sashavor/subscriptions
User
https://api.github.com/users/sashavor
public
MEMBER
> Does that mean that we need to change the metadata?

Yes, but this PR takes care of it :)
2022-05-24T12:24:40Z
https://github.com/huggingface/datasets/pull/4391#issuecomment-1135853608
https://api.github.com/repos/huggingface/datasets/issues/4391
IC_kwDODunzps5Ds8Ao
https://api.github.com/repos/huggingface/datasets/issues/comments/1135853608/reactions
2022-05-24T12:24:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1135853608
https://avatars.githubusercontent.com/u/26859204?v=4
https://api.github.com/users/lewtun/events{/privacy}
https://api.github.com/users/lewtun/followers
https://api.github.com/users/lewtun/following{/other_user}
https://api.github.com/users/lewtun/gists{/gist_id}
https://github.com/lewtun
lewtun
MDQ6VXNlcjI2ODU5MjA0
https://api.github.com/users/lewtun/orgs
https://api.github.com/users/lewtun/received_events
https://api.github.com/users/lewtun/repos
https://api.github.com/users/lewtun/starred{/owner}{/repo}
https://api.github.com/users/lewtun/subscriptions
User
https://api.github.com/users/lewtun
public
CONTRIBUTOR
Oh good! thanks for the heads up!
2022-05-24T12:28:09Z
https://github.com/huggingface/datasets/pull/4391#issuecomment-1135856938
https://api.github.com/repos/huggingface/datasets/issues/4391
IC_kwDODunzps5Ds80q
https://api.github.com/repos/huggingface/datasets/issues/comments/1135856938/reactions
2022-05-24T12:28:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1135856938
https://avatars.githubusercontent.com/u/14205986?v=4
https://api.github.com/users/sashavor/events{/privacy}
https://api.github.com/users/sashavor/followers
https://api.github.com/users/sashavor/following{/other_user}
https://api.github.com/users/sashavor/gists{/gist_id}
https://github.com/sashavor
sashavor
MDQ6VXNlcjE0MjA1OTg2
https://api.github.com/users/sashavor/orgs
https://api.github.com/users/sashavor/received_events
https://api.github.com/users/sashavor/repos
https://api.github.com/users/sashavor/starred{/owner}{/repo}
https://api.github.com/users/sashavor/subscriptions
User
https://api.github.com/users/sashavor
public
This PR tweaks the keys in the metadata that are used to define the column mapping for question answering datasets. This is needed in order to faithfully reconstruct column names like `answers.text` and `answers.answer_start` from the keys in AutoTrain.

As observed in https://github.com/huggingface/datasets/pull/4367 we cannot use periods `.` in the keys of the YAML tags, so a decision was made to use a flat mapping with underscores. For QA datasets, however, it's handy to be able to reconstruct the nesting -- hence this PR.

cc @sashavor 