SubjQA wrong boolean values in entries
MEMBER
Hi @arnaudstiegler, thanks for reporting. I'm investigating it.
2021-06-15T08:04:14Z
https://github.com/huggingface/datasets/issues/2503#issuecomment-861280565
https://api.github.com/repos/huggingface/datasets/issues/2503
MDEyOklzc3VlQ29tbWVudDg2MTI4MDU2NQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/861280565/reactions
2021-06-15T08:04:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/861280565
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
@arnaudstiegler I have just checked that these mismatches are already present in the original dataset: https://github.com/megagonlabs/SubjQA

We are going to contact the dataset owners to report this.
2021-06-15T08:21:38Z
https://github.com/huggingface/datasets/issues/2503#issuecomment-861293139
https://api.github.com/repos/huggingface/datasets/issues/2503
MDEyOklzc3VlQ29tbWVudDg2MTI5MzEzOQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/861293139/reactions
2021-06-15T08:21:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/861293139
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
I have:
- opened an issue in their repo: https://github.com/megagonlabs/SubjQA/issues/3
- written an email to all the paper authors
2021-06-15T08:40:59Z
https://github.com/huggingface/datasets/issues/2503#issuecomment-861306783
https://api.github.com/repos/huggingface/datasets/issues/2503
MDEyOklzc3VlQ29tbWVudDg2MTMwNjc4Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/861306783/reactions
2021-06-15T08:40:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/861306783
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
Please [see my response](https://github.com/megagonlabs/SubjQA/issues/3#issuecomment-905160010). There will be a fix in a couple of days.
2021-08-25T03:52:05Z
https://github.com/huggingface/datasets/issues/2503#issuecomment-905160450
https://api.github.com/repos/huggingface/datasets/issues/2503
IC_kwDODunzps4186cC
https://api.github.com/repos/huggingface/datasets/issues/comments/905160450/reactions
2021-08-25T03:52:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/905160450
https://avatars.githubusercontent.com/u/22460080?v=4
https://api.github.com/users/behzadg/events{/privacy}
https://api.github.com/users/behzadg/followers
https://api.github.com/users/behzadg/following{/other_user}
https://api.github.com/users/behzadg/gists{/gist_id}
https://github.com/behzadg
behzadg
MDQ6VXNlcjIyNDYwMDgw
https://api.github.com/users/behzadg/orgs
https://api.github.com/users/behzadg/received_events
https://api.github.com/users/behzadg/repos
https://api.github.com/users/behzadg/starred{/owner}{/repo}
https://api.github.com/users/behzadg/subscriptions
User
https://api.github.com/users/behzadg
public
## Describe the bug
SubjQA seems to have a boolean that's consistently wrong.

It defines:
- question_subj_level: The subjectiviy level of the question (on a 1 to 5 scale with 1 being the most subjective).
- is_ques_subjective: A boolean subjectivity label derived from question_subj_level (i.e., scores below 4 are considered as subjective)

However, `is_ques_subjective` seems to have wrong values in the entire dataset.

For instance, in the example in the dataset card, we have:
- "question_subj_level": 2
- "is_ques_subjective": false

However, according to the description, the question should be subjective since the `question_subj_level` is below 4
