Image Feature in Datasets Library Fails to Handle bytearray Objects from Spark DataFrames
MEMBER
Hi ! The `Image()` type accepts either
- a `bytes` object containing the image bytes
- a `str` object containing the image path
- a `PIL.Image` object

but it doesn't support `bytearray`, maybe you can convert to `bytes` beforehand ?
2025-04-15T12:49:47Z
https://github.com/huggingface/datasets/issues/7517#issuecomment-2804945078
https://api.github.com/repos/huggingface/datasets/issues/7517
IC_kwDODunzps6nMBC2
https://api.github.com/repos/huggingface/datasets/issues/comments/2804945078/reactions
2025-04-15T12:49:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2804945078
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq, 
converting to bytes is certainly possible and would work around the error. However, the core issue is that `Dataset` and `IterableDataset` behave differently with the features.

I’d be happy to work on a fix for this issue.
2025-04-15T14:10:51Z
https://github.com/huggingface/datasets/issues/7517#issuecomment-2805280000
https://api.github.com/repos/huggingface/datasets/issues/7517
IC_kwDODunzps6nNS0A
https://api.github.com/repos/huggingface/datasets/issues/comments/2805280000/reactions
2025-04-15T14:10:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2805280000
https://avatars.githubusercontent.com/u/73196164?v=4
https://api.github.com/users/giraffacarp/events{/privacy}
https://api.github.com/users/giraffacarp/followers
https://api.github.com/users/giraffacarp/following{/other_user}
https://api.github.com/users/giraffacarp/gists{/gist_id}
https://github.com/giraffacarp
giraffacarp
MDQ6VXNlcjczMTk2MTY0
https://api.github.com/users/giraffacarp/orgs
https://api.github.com/users/giraffacarp/received_events
https://api.github.com/users/giraffacarp/repos
https://api.github.com/users/giraffacarp/starred{/owner}{/repo}
https://api.github.com/users/giraffacarp/subscriptions
User
https://api.github.com/users/giraffacarp
public
MEMBER
I see, that's an issue indeed. Feel free to ping me if I can help with reviews or any guidance

If it can help, the code that takes a Spark DataFrame and iterates on the rows for `IterableDataset` is here: 

https://github.com/huggingface/datasets/blob/6a96bf313085d7538a999b929a550e14e1d406c9/src/datasets/packaged_modules/spark/spark.py#L49-L53
2025-04-15T15:46:44Z
https://github.com/huggingface/datasets/issues/7517#issuecomment-2806455626
https://api.github.com/repos/huggingface/datasets/issues/7517
IC_kwDODunzps6nRx1K
https://api.github.com/repos/huggingface/datasets/issues/comments/2806455626/reactions
2025-04-15T15:46:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2806455626
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
#self-assign
2025-04-15T15:50:54Z
https://github.com/huggingface/datasets/issues/7517#issuecomment-2806504239
https://api.github.com/repos/huggingface/datasets/issues/7517
IC_kwDODunzps6nR9sv
https://api.github.com/repos/huggingface/datasets/issues/comments/2806504239/reactions
2025-04-15T15:50:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2806504239
https://avatars.githubusercontent.com/u/73196164?v=4
https://api.github.com/users/giraffacarp/events{/privacy}
https://api.github.com/users/giraffacarp/followers
https://api.github.com/users/giraffacarp/following{/other_user}
https://api.github.com/users/giraffacarp/gists{/gist_id}
https://github.com/giraffacarp
giraffacarp
MDQ6VXNlcjczMTk2MTY0
https://api.github.com/users/giraffacarp/orgs
https://api.github.com/users/giraffacarp/received_events
https://api.github.com/users/giraffacarp/repos
https://api.github.com/users/giraffacarp/starred{/owner}{/repo}
https://api.github.com/users/giraffacarp/subscriptions
User
https://api.github.com/users/giraffacarp
public
### Describe the bug

When using `IterableDataset.from_spark()` with a Spark DataFrame containing image data, the `Image` feature class fails to properly process this data type, causing an `AttributeError: 'bytearray' object has no attribute 'get'`

### Steps to reproduce the bug

1. Create a Spark DataFrame with a column containing image data as bytearray objects
2. Define a Feature schema with an Image feature
3. Create an IterableDataset using `IterableDataset.from_spark()`
4. Attempt to iterate through the dataset

```
from pyspark.sql import SparkSession
from datasets import Dataset, IterableDataset, Features, Image, Value

# initialize spark
spark = SparkSession.builder.appName("MinimalRepro").getOrCreate()

# create spark dataframe
data = [(0, open("image.png", "rb").read())]
df = spark.createDataFrame(data, "idx: int, image: binary")

# convert to dataset
features = Features({"idx": Value("int64"), "image": Image()})
ds = Dataset.from_spark(df, features=features)
ds_iter = IterableDataset.from_spark(df, features=features)

# iterate
print(next(iter(ds)))
print(next(iter(ds_iter)))
```

### Expected behavior

The features should work on `IterableDataset` the same way they work on `Dataset`

### Environment info

- `datasets` version: 3.5.0
- Platform: macOS-15.3.2-arm64-arm-64bit
- Python version: 3.12.7
- `huggingface_hub` version: 0.30.2
- PyArrow version: 18.1.0
- Pandas version: 2.2.3
- `fsspec` version: 2024.12.0
