`None` converted to `"None"` when loading a dataset
MEMBER
Hi @eladsegal, thanks for reporting.

@mariosasko I saw you are already working on this, but maybe my comment will be useful to you.

All values are casted to their corresponding feature type (including `None` values). For example if the feature type is `Value("bool")`, `None` is casted to `False`.

It is true that strings were an exception, but this was recently fixed by @lhoestq (see #3158).
2021-10-29T16:19:08Z
https://github.com/huggingface/datasets/issues/3181#issuecomment-954875325
https://api.github.com/repos/huggingface/datasets/issues/3181
IC_kwDODunzps446j29
https://api.github.com/repos/huggingface/datasets/issues/comments/954875325/reactions
2021-10-29T16:19:31Z
https://api.github.com/repos/huggingface/datasets/issues/comments/954875325
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
MEMBER
Thanks for reporting.

This is actually a breaking change that I think can cause issues when users preprocess their data. String columns used to be nullable. Maybe we can correct https://github.com/huggingface/datasets/pull/3158 to keep the None values and avoid this breaking change ?

EDIT: the other types (bool, int, etc) can also become nullable IMO
2021-10-29T16:36:58Z
https://github.com/huggingface/datasets/issues/3181#issuecomment-954886997
https://api.github.com/repos/huggingface/datasets/issues/3181
IC_kwDODunzps446mtV
https://api.github.com/repos/huggingface/datasets/issues/comments/954886997/reactions
2021-10-29T16:37:42Z
https://api.github.com/repos/huggingface/datasets/issues/comments/954886997
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
So what would be the best way to handle a feature that can have a null value in some of the instances? So far I used `None`.
Using the empty string won't be a good option, as it can be an actual value in the data and is not the same as not having a value at all.
2021-10-29T16:58:53Z
https://github.com/huggingface/datasets/issues/3181#issuecomment-954900002
https://api.github.com/repos/huggingface/datasets/issues/3181
IC_kwDODunzps446p4i
https://api.github.com/repos/huggingface/datasets/issues/comments/954900002/reactions
2021-10-29T16:58:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/954900002
https://avatars.githubusercontent.com/u/13485709?v=4
https://api.github.com/users/eladsegal/events{/privacy}
https://api.github.com/users/eladsegal/followers
https://api.github.com/users/eladsegal/following{/other_user}
https://api.github.com/users/eladsegal/gists{/gist_id}
https://github.com/eladsegal
eladsegal
MDQ6VXNlcjEzNDg1NzA5
https://api.github.com/users/eladsegal/orgs
https://api.github.com/users/eladsegal/received_events
https://api.github.com/users/eladsegal/repos
https://api.github.com/users/eladsegal/starred{/owner}{/repo}
https://api.github.com/users/eladsegal/subscriptions
User
https://api.github.com/users/eladsegal
public
COLLABORATOR
Hi @eladsegal,

Use `None`. As @albertvillanova correctly pointed out, this change in conversion was introduced (by mistake) in #3158. To avoid it, install the earlier revision with:
```
pip install git+https://github.com/huggingface/datasets.git@8107844ec0e7add005db0585c772ee20adc01a5e
```

I'm making all the feature types nullable as we speak, and the fix will be merged probably early next week.
2021-10-29T17:14:16Z
https://github.com/huggingface/datasets/issues/3181#issuecomment-954909211
https://api.github.com/repos/huggingface/datasets/issues/3181
IC_kwDODunzps446sIb
https://api.github.com/repos/huggingface/datasets/issues/comments/954909211/reactions
2021-10-29T17:15:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/954909211
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
Hi @mariosasko, is there an estimation as to when this issue will be fixed?
2021-12-07T23:48:22Z
https://github.com/huggingface/datasets/issues/3181#issuecomment-988349332
https://api.github.com/repos/huggingface/datasets/issues/3181
IC_kwDODunzps466QOU
https://api.github.com/repos/huggingface/datasets/issues/comments/988349332/reactions
2021-12-07T23:48:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/988349332
https://avatars.githubusercontent.com/u/13485709?v=4
https://api.github.com/users/eladsegal/events{/privacy}
https://api.github.com/users/eladsegal/followers
https://api.github.com/users/eladsegal/following{/other_user}
https://api.github.com/users/eladsegal/gists{/gist_id}
https://github.com/eladsegal
eladsegal
MDQ6VXNlcjEzNDg1NzA5
https://api.github.com/users/eladsegal/orgs
https://api.github.com/users/eladsegal/received_events
https://api.github.com/users/eladsegal/repos
https://api.github.com/users/eladsegal/starred{/owner}{/repo}
https://api.github.com/users/eladsegal/subscriptions
User
https://api.github.com/users/eladsegal
public
MEMBER
https://github.com/huggingface/datasets/pull/3195 fixed it, we'll do a new release soon :)

For now feel free to install `datasets` from the master branch
2021-12-09T17:46:10Z
https://github.com/huggingface/datasets/issues/3181#issuecomment-990078261
https://api.github.com/repos/huggingface/datasets/issues/3181
IC_kwDODunzps47A2U1
https://api.github.com/repos/huggingface/datasets/issues/comments/990078261/reactions
2021-12-09T17:46:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/990078261
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Thanks, but unfortunately looks like it isn't fixed yet 😢 
[notebook for 1.14.0](https://colab.research.google.com/drive/1SV3sFXPJMWSQgbm4pr9Y1Q8OJ4JYKcDo?usp=sharing)
[notebook for master](https://colab.research.google.com/drive/145wDpuO74MmsuI0SVLcI1IswG6aHpyhi?usp=sharing)
2021-12-09T18:17:24Z
https://github.com/huggingface/datasets/issues/3181#issuecomment-990103711
https://api.github.com/repos/huggingface/datasets/issues/3181
IC_kwDODunzps47A8if
https://api.github.com/repos/huggingface/datasets/issues/comments/990103711/reactions
2021-12-09T18:17:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/990103711
https://avatars.githubusercontent.com/u/13485709?v=4
https://api.github.com/users/eladsegal/events{/privacy}
https://api.github.com/users/eladsegal/followers
https://api.github.com/users/eladsegal/following{/other_user}
https://api.github.com/users/eladsegal/gists{/gist_id}
https://github.com/eladsegal
eladsegal
MDQ6VXNlcjEzNDg1NzA5
https://api.github.com/users/eladsegal/orgs
https://api.github.com/users/eladsegal/received_events
https://api.github.com/users/eladsegal/repos
https://api.github.com/users/eladsegal/starred{/owner}{/repo}
https://api.github.com/users/eladsegal/subscriptions
User
https://api.github.com/users/eladsegal
public
COLLABORATOR
Oh, sorry. I deleted the fix by accident when I was resolving a merge conflict. Let me fix this real quick.
2021-12-09T21:07:31Z
https://github.com/huggingface/datasets/issues/3181#issuecomment-990254346
https://api.github.com/repos/huggingface/datasets/issues/3181
IC_kwDODunzps47BhUK
https://api.github.com/repos/huggingface/datasets/issues/comments/990254346/reactions
2021-12-09T21:07:31Z
https://api.github.com/repos/huggingface/datasets/issues/comments/990254346
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
Thank you, it works! 🎊 
2021-12-11T01:16:40Z
https://github.com/huggingface/datasets/issues/3181#issuecomment-991401194
https://api.github.com/repos/huggingface/datasets/issues/3181
IC_kwDODunzps47F5Tq
https://api.github.com/repos/huggingface/datasets/issues/comments/991401194/reactions
2021-12-11T01:16:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/991401194
https://avatars.githubusercontent.com/u/13485709?v=4
https://api.github.com/users/eladsegal/events{/privacy}
https://api.github.com/users/eladsegal/followers
https://api.github.com/users/eladsegal/following{/other_user}
https://api.github.com/users/eladsegal/gists{/gist_id}
https://github.com/eladsegal
eladsegal
MDQ6VXNlcjEzNDg1NzA5
https://api.github.com/users/eladsegal/orgs
https://api.github.com/users/eladsegal/received_events
https://api.github.com/users/eladsegal/repos
https://api.github.com/users/eladsegal/starred{/owner}{/repo}
https://api.github.com/users/eladsegal/subscriptions
User
https://api.github.com/users/eladsegal
public
## Describe the bug
When loading a dataset `None` values of the type `NoneType` are converted to `'None'` of the type `str`.

## Steps to reproduce the bug
```python
from datasets import load_dataset
qasper = load_dataset("qasper", split="train", download_mode="reuse_cache_if_exists")
print(qasper[60]["full_text"]["section_name"])
```

When installing version 1.1.40, the output is
`[None, 'Introduction', 'Benchmark Datasets', ...]`

When installing from the master branch, the output is
`['None', 'Introduction', 'Benchmark Datasets', ...]`

Notice how the first element was changed from `NoneType` to `str`.

## Expected results
`None` should stay as is.

## Actual results
`None` is converted to a string.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: master
- Platform: Linux-4.4.0-19041-Microsoft-x86_64-with-glibc2.17
- Python version: 3.8.10
- PyArrow version: 4.0.1
