Batched mapping of existing string column casts boolean to string
CONTRIBUTOR
This seems to be hardcoded behavior in table.py `array_cast`.
```python
if (
            not allow_number_to_str
            and pa.types.is_string(pa_type)
            and (pa.types.is_floating(array.type) or pa.types.is_integer(array.type))
        ):
            raise TypeError(
                f"Couldn't cast array of type {array.type} to {pa_type} since allow_number_to_str is set to {allow_number_to_str}"
            )
        if pa.types.is_null(pa_type) and not pa.types.is_null(array.type):
            raise TypeError(f"Couldn't cast array of type {array.type} to {pa_type}")
        return array.cast(pa_type)
```
where floats and integers are not cast to string but booleans are.
Maybe this should be extended to booleans?
2024-04-12T09:13:41Z
https://github.com/huggingface/datasets/issues/6805#issuecomment-2051364899
https://api.github.com/repos/huggingface/datasets/issues/6805
IC_kwDODunzps56RVgj
https://api.github.com/repos/huggingface/datasets/issues/comments/2051364899/reactions
2024-04-12T09:13:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2051364899
https://avatars.githubusercontent.com/u/37351874?v=4
https://api.github.com/users/Modexus/events{/privacy}
https://api.github.com/users/Modexus/followers
https://api.github.com/users/Modexus/following{/other_user}
https://api.github.com/users/Modexus/gists{/gist_id}
https://github.com/Modexus
Modexus
MDQ6VXNlcjM3MzUxODc0
https://api.github.com/users/Modexus/orgs
https://api.github.com/users/Modexus/received_events
https://api.github.com/users/Modexus/repos
https://api.github.com/users/Modexus/starred{/owner}{/repo}
https://api.github.com/users/Modexus/subscriptions
User
https://api.github.com/users/Modexus
public
COLLABORATOR
Thanks for reporting! @Modexus Do you want to open a PR with the suggested fix?
2024-04-12T16:33:14Z
https://github.com/huggingface/datasets/issues/6805#issuecomment-2052092372
https://api.github.com/repos/huggingface/datasets/issues/6805
IC_kwDODunzps56UHHU
https://api.github.com/repos/huggingface/datasets/issues/comments/2052092372/reactions
2024-04-12T16:33:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2052092372
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
I'll gladly create a PR but not sure what the behavior should be.

Should a value returned from map be cast to the current feature?
At the moment this seems very inconsistent since `datetime `is also cast (this would only fix `boolean`) but nested structures are not.

```python
dset = Dataset.from_dict({"a": ["Hello world!"]})
dset = dset.map(lambda x: {"a": date(2021, 1, 1)})
# dset[0]["a"] == '2021-01-01'
```
```python
dset = Dataset.from_dict({"a": ["Hello world!"]})
dset = dset.map(lambda x: {"a": [True]})
# dset[0]["a"] == [True]
```

Is there are reason to cast the value if the user doesn't specify it explicitly?
Seems tricky that some things are cast and some are not.
2024-04-12T19:18:56Z
https://github.com/huggingface/datasets/issues/6805#issuecomment-2052394320
https://api.github.com/repos/huggingface/datasets/issues/6805
IC_kwDODunzps56VQ1Q
https://api.github.com/repos/huggingface/datasets/issues/comments/2052394320/reactions
2024-04-12T19:18:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2052394320
https://avatars.githubusercontent.com/u/37351874?v=4
https://api.github.com/users/Modexus/events{/privacy}
https://api.github.com/users/Modexus/followers
https://api.github.com/users/Modexus/following{/other_user}
https://api.github.com/users/Modexus/gists{/gist_id}
https://github.com/Modexus
Modexus
MDQ6VXNlcjM3MzUxODc0
https://api.github.com/users/Modexus/orgs
https://api.github.com/users/Modexus/received_events
https://api.github.com/users/Modexus/repos
https://api.github.com/users/Modexus/starred{/owner}{/repo}
https://api.github.com/users/Modexus/subscriptions
User
https://api.github.com/users/Modexus
public
COLLABORATOR
Indeed, it also makes sense to raise a `TypeError` for temporal and decimal types.

> Is there are reason to cast the value if the user doesn't specify it explicitly?

This is how PyArrow's built-in `cast` behaves - it allows casting from primitive types to strings. Hence, we need `allow_number_to_str` to disallow such casts (e.g., in the [scenario](https://github.com/huggingface/datasets/blob/a3bc89d8bfd47c2a175c3ce16d92b7307cdeafd6/src/datasets/arrow_writer.py#L208) when we are "trying a type" to preserve the original type if there is a column in the output dataset with the same name as in the input one).

PS: In the PR, we can introduce `allow_numeric_to_str` (for floats, integers, decimals, booleans) and `allow_temporal_to_str` (for dates, timestamps, ...) and deprecate `allow_number_to_str` to make it clear what each parameter does.
2024-04-13T14:06:58Z
https://github.com/huggingface/datasets/issues/6805#issuecomment-2053656477
https://api.github.com/repos/huggingface/datasets/issues/6805
IC_kwDODunzps56aE-d
https://api.github.com/repos/huggingface/datasets/issues/comments/2053656477/reactions
2024-04-13T14:06:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2053656477
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
Would just `allow_primitive_to_str` work?
This should include all `numeric`, `boolean `and `temporal`formats.

Note that at least in the [ C++ implementation](https://arrow.apache.org/docs/cpp/api/utilities.html#_CPPv410is_numericRK8DataType) `numeric `seems to exclude `boolean`.
[](https://arrow.apache.org/docs/cpp/api/utilities.html#_CPPv410is_numericRK8DataType)
2024-04-15T12:18:09Z
https://github.com/huggingface/datasets/issues/6805#issuecomment-2056715299
https://api.github.com/repos/huggingface/datasets/issues/6805
IC_kwDODunzps56lvwj
https://api.github.com/repos/huggingface/datasets/issues/comments/2056715299/reactions
2024-04-15T12:18:09Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2056715299
https://avatars.githubusercontent.com/u/37351874?v=4
https://api.github.com/users/Modexus/events{/privacy}
https://api.github.com/users/Modexus/followers
https://api.github.com/users/Modexus/following{/other_user}
https://api.github.com/users/Modexus/gists{/gist_id}
https://github.com/Modexus
Modexus
MDQ6VXNlcjM3MzUxODc0
https://api.github.com/users/Modexus/orgs
https://api.github.com/users/Modexus/received_events
https://api.github.com/users/Modexus/repos
https://api.github.com/users/Modexus/starred{/owner}{/repo}
https://api.github.com/users/Modexus/subscriptions
User
https://api.github.com/users/Modexus
public
COLLABORATOR
Indeed, `allow_primitive_to_str` sounds better.

PS: PyArrow's `pa.types.is_primitive` returns `False` for decimal types, but I think is okay for us to treat decimals as primitive types (or we can have `allow_decimal_to_str` to be fully consistent with PyArrow)
2024-04-15T12:55:18Z
https://github.com/huggingface/datasets/issues/6805#issuecomment-2056788510
https://api.github.com/repos/huggingface/datasets/issues/6805
IC_kwDODunzps56mBoe
https://api.github.com/repos/huggingface/datasets/issues/comments/2056788510/reactions
2024-04-15T12:55:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2056788510
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
Fixed by:
- #6811
2024-07-03T15:00:07Z
https://github.com/huggingface/datasets/issues/6805#issuecomment-2206435341
https://api.github.com/repos/huggingface/datasets/issues/6805
IC_kwDODunzps6Dg4gN
https://api.github.com/repos/huggingface/datasets/issues/comments/2206435341/reactions
2024-07-03T15:00:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2206435341
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
### Describe the bug

Let the dataset contain a column named 'a', which is of the string type.
If 'a' is converted to a boolean using batched mapping, the mapper automatically casts the boolean to a string (e.g., True -> 'true').
It only happens when the original column and the mapped column name are identical.

Thank you!

### Steps to reproduce the bug

```python
from datasets import Dataset

dset = Dataset.from_dict({'a': ['11', '22']})
dset = dset.map(lambda x: {'a': [True for _ in x['a']]}, batched=True)
print(dset['a'])
```
```
> ['true', 'true']
```

### Expected behavior

[True, True]

### Environment info

- `datasets` version: 2.18.0
- Platform: Linux-5.4.0-148-generic-x86_64-with-glibc2.31
- Python version: 3.10.13
- `huggingface_hub` version: 0.21.4
- PyArrow version: 15.0.2
- Pandas version: 2.2.1
- `fsspec` version: 2023.12.2