OpenBookQA has missing and inconsistent field names
MEMBER
Thanks for reporting, @vblagoje.

Indeed, I noticed some of these issues while reviewing this PR:
- #4259 

This is in my TODO list. 
2022-05-04T06:53:23Z
https://github.com/huggingface/datasets/issues/4276#issuecomment-1116983615
https://api.github.com/repos/huggingface/datasets/issues/4276
IC_kwDODunzps5Ck9E_
https://api.github.com/repos/huggingface/datasets/issues/comments/1116983615/reactions
2022-05-04T06:53:23Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1116983615
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
CONTRIBUTOR
Ok, awesome @albertvillanova How about #4275 ?
2022-05-04T08:37:45Z
https://github.com/huggingface/datasets/issues/4276#issuecomment-1117059833
https://api.github.com/repos/huggingface/datasets/issues/4276
IC_kwDODunzps5ClPr5
https://api.github.com/repos/huggingface/datasets/issues/comments/1117059833/reactions
2022-05-04T08:37:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1117059833
https://avatars.githubusercontent.com/u/458335?v=4
https://api.github.com/users/vblagoje/events{/privacy}
https://api.github.com/users/vblagoje/followers
https://api.github.com/users/vblagoje/following{/other_user}
https://api.github.com/users/vblagoje/gists{/gist_id}
https://github.com/vblagoje
vblagoje
MDQ6VXNlcjQ1ODMzNQ==
https://api.github.com/users/vblagoje/orgs
https://api.github.com/users/vblagoje/received_events
https://api.github.com/users/vblagoje/repos
https://api.github.com/users/vblagoje/starred{/owner}{/repo}
https://api.github.com/users/vblagoje/subscriptions
User
https://api.github.com/users/vblagoje
public
MEMBER
On the other hand, I am not sure if we should always preserve the original nested structure. I think we should also consider other factors as convenience or consistency.

For example, other datasets also flatten "question.stem" into "question":
- ai2_arc:
  ```python
  question = data["question"]["stem"]
  choices = data["question"]["choices"]
  text_choices = [choice["text"] for choice in choices]
  label_choices = [choice["label"] for choice in choices]
  yield id_, {
      "id": id_,
      "answerKey": answerkey,
      "question": question,
      "choices": {"text": text_choices, "label": label_choices},
  }
  ```
- commonsense_qa:
  ```python
  question = data["question"]
  stem = question["stem"]
  yield id_, {
      "answerKey": answerkey,
      "question": stem,
      "choices": {"label": labels, "text": texts},
  }
  ```
- cos_e:
  ```python
  "question": cqa["question"]["stem"],
  ```
- qasc
- quartz
- wiqa

Exceptions:
- exams

I think we should agree on a CONVENIENT format for QA and use always CONSISTENTLY the same.
2022-05-04T08:38:54Z
https://github.com/huggingface/datasets/issues/4276#issuecomment-1117060831
https://api.github.com/repos/huggingface/datasets/issues/4276
IC_kwDODunzps5ClP7f
https://api.github.com/repos/huggingface/datasets/issues/comments/1117060831/reactions
2022-05-04T08:38:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1117060831
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
CONTRIBUTOR
@albertvillanova I agree that we should be consistent. In the last month, I have come across tons of code that deals with OpenBookQA and CommonSenseQA and all of that code relies on the original data format structure. We can't expect users to adopt HF Datasets if we arbitrarily change the structure of the format just because we think something makes more sense. I am in that position now (downloading original data rather than using HF Datasets) and undoubtedly it hinders HF Datasets' widespread use and adoption. Missing fields like in the case of #4275 is definitely bad and not even up for a discussion IMHO! cc @lhoestq 
2022-05-04T08:42:59Z
https://github.com/huggingface/datasets/issues/4276#issuecomment-1117064097
https://api.github.com/repos/huggingface/datasets/issues/4276
IC_kwDODunzps5ClQuh
https://api.github.com/repos/huggingface/datasets/issues/comments/1117064097/reactions
2022-05-04T08:45:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1117064097
https://avatars.githubusercontent.com/u/458335?v=4
https://api.github.com/users/vblagoje/events{/privacy}
https://api.github.com/users/vblagoje/followers
https://api.github.com/users/vblagoje/following{/other_user}
https://api.github.com/users/vblagoje/gists{/gist_id}
https://github.com/vblagoje
vblagoje
MDQ6VXNlcjQ1ODMzNQ==
https://api.github.com/users/vblagoje/orgs
https://api.github.com/users/vblagoje/received_events
https://api.github.com/users/vblagoje/repos
https://api.github.com/users/vblagoje/starred{/owner}{/repo}
https://api.github.com/users/vblagoje/subscriptions
User
https://api.github.com/users/vblagoje
public
MEMBER
I'm opening a PR that adds the missing fields.

Let's agree on the feature structure: @lhoestq @mariosasko @polinaeterna 
2022-05-04T09:00:28Z
https://github.com/huggingface/datasets/issues/4276#issuecomment-1117079177
https://api.github.com/repos/huggingface/datasets/issues/4276
IC_kwDODunzps5ClUaJ
https://api.github.com/repos/huggingface/datasets/issues/comments/1117079177/reactions
2022-05-04T09:12:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1117079177
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
COLLABORATOR
IMO we should always try to preserve the original structure unless there is a good reason not to (and I don't see one in this case).
2022-05-04T11:51:28Z
https://github.com/huggingface/datasets/issues/4276#issuecomment-1117220326
https://api.github.com/repos/huggingface/datasets/issues/4276
IC_kwDODunzps5Cl23m
https://api.github.com/repos/huggingface/datasets/issues/comments/1117220326/reactions
2022-05-04T11:51:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1117220326
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
I agree with @mariosasko . The transition to the original format could be done in one PR for the next minor release, clearly documenting all dataset changes just as @albertvillanova outlined them above and perhaps even providing a per dataset util method to convert the new valid format to the old for backward compatibility. Users who relied on the old format will update their code with either the util method for a quick fix or slightly more elaborate for the new. 
2022-05-05T07:45:45Z
https://github.com/huggingface/datasets/issues/4276#issuecomment-1118263065
https://api.github.com/repos/huggingface/datasets/issues/4276
IC_kwDODunzps5Cp1cZ
https://api.github.com/repos/huggingface/datasets/issues/comments/1118263065/reactions
2022-05-05T07:45:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1118263065
https://avatars.githubusercontent.com/u/458335?v=4
https://api.github.com/users/vblagoje/events{/privacy}
https://api.github.com/users/vblagoje/followers
https://api.github.com/users/vblagoje/following{/other_user}
https://api.github.com/users/vblagoje/gists{/gist_id}
https://github.com/vblagoje
vblagoje
MDQ6VXNlcjQ1ODMzNQ==
https://api.github.com/users/vblagoje/orgs
https://api.github.com/users/vblagoje/received_events
https://api.github.com/users/vblagoje/repos
https://api.github.com/users/vblagoje/starred{/owner}{/repo}
https://api.github.com/users/vblagoje/subscriptions
User
https://api.github.com/users/vblagoje
public
MEMBER
I don't have a strong opinion on this, besides the fact that whatever decision we agree on, should be applied to all datasets.

There is always the tension between:
- preserving each dataset original structure (which has the advantage of not forcing users to learn other structure for the same dataset),
-  and on the other hand performing some kind of standardization/harmonization depending on the task (this has the advantage that once learnt, the same structure applies to all datasets; this has been done for e.g.  POS tagging: all datasets have been adapted to a certain "standard" structure).
   - Another advantage: datasets can easily be interchanged (or joined) to be used by the same model

Recently, in the BigScience BioMedical hackathon, they adopted a different approach:
- they implement a "source" config, respecting the original structure as much as possible
- they implement additional config for each task, with a "standard" nested structure per task, which is most useful for users.
2022-05-06T12:53:15Z
https://github.com/huggingface/datasets/issues/4276#issuecomment-1119585304
https://api.github.com/repos/huggingface/datasets/issues/4276
IC_kwDODunzps5Cu4QY
https://api.github.com/repos/huggingface/datasets/issues/comments/1119585304/reactions
2022-10-11T17:11:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1119585304
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
CONTRIBUTOR
@albertvillanova, thanks for the detailed answer and the new perspectives. I understand the friction for the best design approach much better now.  Ultimately, it is essential to include all the missing fields and the correct data first. Whatever approach is determined to be optimal is important but not as crucial once all the data is there, and users can create lambda functions to create whatever structure serves them best. 
2022-05-08T12:33:43Z
https://github.com/huggingface/datasets/issues/4276#issuecomment-1120410825
https://api.github.com/repos/huggingface/datasets/issues/4276
IC_kwDODunzps5CyBzJ
https://api.github.com/repos/huggingface/datasets/issues/comments/1120410825/reactions
2022-05-08T12:33:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1120410825
https://avatars.githubusercontent.com/u/458335?v=4
https://api.github.com/users/vblagoje/events{/privacy}
https://api.github.com/users/vblagoje/followers
https://api.github.com/users/vblagoje/following{/other_user}
https://api.github.com/users/vblagoje/gists{/gist_id}
https://github.com/vblagoje
vblagoje
MDQ6VXNlcjQ1ODMzNQ==
https://api.github.com/users/vblagoje/orgs
https://api.github.com/users/vblagoje/received_events
https://api.github.com/users/vblagoje/repos
https://api.github.com/users/vblagoje/starred{/owner}{/repo}
https://api.github.com/users/vblagoje/subscriptions
User
https://api.github.com/users/vblagoje
public
CONTRIBUTOR
Datasets are not tracked in this repository anymore. I think we must move this thread to the [discussions tab of the dataset](https://huggingface.co/datasets/openbookqa/discussions)
2022-10-05T10:35:19Z
https://github.com/huggingface/datasets/issues/4276#issuecomment-1268254491
https://api.github.com/repos/huggingface/datasets/issues/4276
IC_kwDODunzps5LmAcb
https://api.github.com/repos/huggingface/datasets/issues/comments/1268254491/reactions
2022-10-05T10:35:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1268254491
https://avatars.githubusercontent.com/u/74963545?v=4
https://api.github.com/users/osbm/events{/privacy}
https://api.github.com/users/osbm/followers
https://api.github.com/users/osbm/following{/other_user}
https://api.github.com/users/osbm/gists{/gist_id}
https://github.com/osbm
osbm
MDQ6VXNlcjc0OTYzNTQ1
https://api.github.com/users/osbm/orgs
https://api.github.com/users/osbm/received_events
https://api.github.com/users/osbm/repos
https://api.github.com/users/osbm/starred{/owner}{/repo}
https://api.github.com/users/osbm/subscriptions
User
https://api.github.com/users/osbm
public
MEMBER
Indeed @osbm thanks. I'm closing this issue if it's fine for you all then
2022-10-05T13:50:03Z
https://github.com/huggingface/datasets/issues/4276#issuecomment-1268466925
https://api.github.com/repos/huggingface/datasets/issues/4276
IC_kwDODunzps5Lm0Tt
https://api.github.com/repos/huggingface/datasets/issues/comments/1268466925/reactions
2022-10-05T13:50:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1268466925
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
## Describe the bug
OpenBookQA implementation is inconsistent with the original dataset.

We need to:

1. The dataset field [question][stem] is flattened into question_stem. Unflatten it to match the original format.
2. Add missing additional fields:
    - 'fact1': row['fact1'],
    - 'humanScore': row['humanScore'],
    - 'clarity': row['clarity'],
    - 'turkIdAnonymized': row['turkIdAnonymized']
3. Ensure the structure and every data item in the original OpenBookQA matches our OpenBookQA version.

## Expected results
The structure and every data item in the original OpenBookQA matches our OpenBookQA version.

## Actual results
TBD

## Environment info
- `datasets` version: 2.1.0
- Platform: macOS-10.15.7-x86_64-i386-64bit
- Python version: 3.8.13
- PyArrow version: 7.0.0
- Pandas version: 1.4.2