TorchFormatter images (H, W, C) instead of (C, H, W) format
CONTRIBUTOR
Here's a PR for that. https://github.com/huggingface/datasets/pull/6402

It's not backward compatible, unfortunately. 
2023-11-11T19:41:02Z
https://github.com/huggingface/datasets/issues/6394#issuecomment-1806900947
https://api.github.com/repos/huggingface/datasets/issues/6394
IC_kwDODunzps5rsx7T
https://api.github.com/repos/huggingface/datasets/issues/comments/1806900947/reactions
2023-11-11T19:41:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1806900947
https://avatars.githubusercontent.com/u/32204417?v=4
https://api.github.com/users/varunneal/events{/privacy}
https://api.github.com/users/varunneal/followers
https://api.github.com/users/varunneal/following{/other_user}
https://api.github.com/users/varunneal/gists{/gist_id}
https://github.com/varunneal
varunneal
MDQ6VXNlcjMyMjA0NDE3
https://api.github.com/users/varunneal/orgs
https://api.github.com/users/varunneal/received_events
https://api.github.com/users/varunneal/repos
https://api.github.com/users/varunneal/starred{/owner}{/repo}
https://api.github.com/users/varunneal/subscriptions
User
https://api.github.com/users/varunneal
public
NONE
Just ran into this working on data lib that's attempting to achieve common interfaces across hf datasets, webdataset, native torch style datasets. The defacto standards for image tensors are numpy == HWC, torch.Tensor == CHW. 

I had to drop use of 'torch' formatting because as is (H, W, C) makes it incompatible with pretty much all standard torch vision processing (torchvision, etc) including model inputs themselves... not sure what the breakage scope would be, but might be worth considering a breaking change since I'm not aware of many use cases where a torch.Tensor image is expected to be in HWC form. And if I set the format to 'torch', I'd expect to be able to apply torchvision transforms, etc directly to the output...

EDIT: For 'torch' output to be compatible with torch conventions (namely torchvision for images), should follow this https://pytorch.org/vision/0.17/transforms.html#supported-input-types-and-conventions

attn @lhoestq 


2024-03-09T00:36:56Z
https://github.com/huggingface/datasets/issues/6394#issuecomment-1986603309
https://api.github.com/repos/huggingface/datasets/issues/6394
IC_kwDODunzps52aSkt
https://api.github.com/repos/huggingface/datasets/issues/comments/1986603309/reactions
2024-03-09T03:47:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1986603309
https://avatars.githubusercontent.com/u/5702664?v=4
https://api.github.com/users/rwightman/events{/privacy}
https://api.github.com/users/rwightman/followers
https://api.github.com/users/rwightman/following{/other_user}
https://api.github.com/users/rwightman/gists{/gist_id}
https://github.com/rwightman
rwightman
MDQ6VXNlcjU3MDI2NjQ=
https://api.github.com/users/rwightman/orgs
https://api.github.com/users/rwightman/received_events
https://api.github.com/users/rwightman/repos
https://api.github.com/users/rwightman/starred{/owner}{/repo}
https://api.github.com/users/rwightman/subscriptions
User
https://api.github.com/users/rwightman
public
MEMBER
We can define something like `.with_format("torch", image_data_format="channels_first")` and recommend using this in the docs maybe ? also cc @NielsRogge 
2024-03-10T13:54:40Z
https://github.com/huggingface/datasets/issues/6394#issuecomment-1987241109
https://api.github.com/repos/huggingface/datasets/issues/6394
IC_kwDODunzps52cuSV
https://api.github.com/repos/huggingface/datasets/issues/comments/1987241109/reactions
2024-03-10T13:54:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1987241109
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Sounds good to me. I guess it's not allowed to use the channels first format by default for backwards compatibility purposes?
2024-03-10T18:13:28Z
https://github.com/huggingface/datasets/issues/6394#issuecomment-1987314858
https://api.github.com/repos/huggingface/datasets/issues/6394
IC_kwDODunzps52dASq
https://api.github.com/repos/huggingface/datasets/issues/comments/1987314858/reactions
2024-03-10T18:13:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1987314858
https://avatars.githubusercontent.com/u/48327001?v=4
https://api.github.com/users/NielsRogge/events{/privacy}
https://api.github.com/users/NielsRogge/followers
https://api.github.com/users/NielsRogge/following{/other_user}
https://api.github.com/users/NielsRogge/gists{/gist_id}
https://github.com/NielsRogge
NielsRogge
MDQ6VXNlcjQ4MzI3MDAx
https://api.github.com/users/NielsRogge/orgs
https://api.github.com/users/NielsRogge/received_events
https://api.github.com/users/NielsRogge/repos
https://api.github.com/users/NielsRogge/starred{/owner}{/repo}
https://api.github.com/users/NielsRogge/subscriptions
User
https://api.github.com/users/NielsRogge
public
NONE
This works, but am wondering how widespread the use of the function is for image datasets? My hunch would be that it's not used widely enough with image datasets to favour backwards compat (keeping default channels_last) over clumsiness of needing this to be 'correct' for typical use.. but don't have the data to back that up.
2024-03-10T18:40:40Z
https://github.com/huggingface/datasets/issues/6394#issuecomment-1987321376
https://api.github.com/repos/huggingface/datasets/issues/6394
IC_kwDODunzps52dB4g
https://api.github.com/repos/huggingface/datasets/issues/comments/1987321376/reactions
2024-03-10T18:41:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1987321376
https://avatars.githubusercontent.com/u/5702664?v=4
https://api.github.com/users/rwightman/events{/privacy}
https://api.github.com/users/rwightman/followers
https://api.github.com/users/rwightman/following{/other_user}
https://api.github.com/users/rwightman/gists{/gist_id}
https://github.com/rwightman
rwightman
MDQ6VXNlcjU3MDI2NjQ=
https://api.github.com/users/rwightman/orgs
https://api.github.com/users/rwightman/received_events
https://api.github.com/users/rwightman/repos
https://api.github.com/users/rwightman/starred{/owner}{/repo}
https://api.github.com/users/rwightman/subscriptions
User
https://api.github.com/users/rwightman
public
MEMBER
I see. I just checked in the HF libraries and it shouldn't break anything. And to be consistent with them we should actually use C H W. For example `transformers` image processors use C H W by default too.

So I'm ok with doing a breaking change to make it consistent with `transformers`, `torchvision`, etc.
2024-03-11T17:50:13Z
https://github.com/huggingface/datasets/issues/6394#issuecomment-1989071767
https://api.github.com/repos/huggingface/datasets/issues/6394
IC_kwDODunzps52jtOX
https://api.github.com/repos/huggingface/datasets/issues/comments/1989071767/reactions
2024-03-11T17:50:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1989071767
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Since it is quite connected, the proposed PR #6402 will not work for monochrome `PIL` images since they only have 2 dimensions as `numpy `arrays. [Torchvision ](https://pytorch.org/vision/stable/_modules/torchvision/transforms/functional.html#pil_to_tensor) adds a channel before permuting. Would that make sense here as well?
2024-03-11T20:18:21Z
https://github.com/huggingface/datasets/issues/6394#issuecomment-1989360585
https://api.github.com/repos/huggingface/datasets/issues/6394
IC_kwDODunzps52kzvJ
https://api.github.com/repos/huggingface/datasets/issues/comments/1989360585/reactions
2024-03-11T20:18:21Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1989360585
https://avatars.githubusercontent.com/u/37351874?v=4
https://api.github.com/users/Modexus/events{/privacy}
https://api.github.com/users/Modexus/followers
https://api.github.com/users/Modexus/following{/other_user}
https://api.github.com/users/Modexus/gists{/gist_id}
https://github.com/Modexus
Modexus
MDQ6VXNlcjM3MzUxODc0
https://api.github.com/users/Modexus/orgs
https://api.github.com/users/Modexus/received_events
https://api.github.com/users/Modexus/repos
https://api.github.com/users/Modexus/starred{/owner}{/repo}
https://api.github.com/users/Modexus/subscriptions
User
https://api.github.com/users/Modexus
public
NONE
@Modexus yes, indeed that would make sense as torch expects 1, H, W for monochrome, not H,W as you'd often see in numpy (via PIL), OpenCV, etc.

The reference should be the torchvision fn https://pytorch.org/vision/main/_modules/torchvision/transforms/functional.html#pil_to_tensor
2024-03-11T20:24:54Z
https://github.com/huggingface/datasets/issues/6394#issuecomment-1989376688
https://api.github.com/repos/huggingface/datasets/issues/6394
IC_kwDODunzps52k3qw
https://api.github.com/repos/huggingface/datasets/issues/comments/1989376688/reactions
2024-03-11T20:24:54Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1989376688
https://avatars.githubusercontent.com/u/5702664?v=4
https://api.github.com/users/rwightman/events{/privacy}
https://api.github.com/users/rwightman/followers
https://api.github.com/users/rwightman/following{/other_user}
https://api.github.com/users/rwightman/gists{/gist_id}
https://github.com/rwightman
rwightman
MDQ6VXNlcjU3MDI2NjQ=
https://api.github.com/users/rwightman/orgs
https://api.github.com/users/rwightman/received_events
https://api.github.com/users/rwightman/repos
https://api.github.com/users/rwightman/starred{/owner}{/repo}
https://api.github.com/users/rwightman/subscriptions
User
https://api.github.com/users/rwightman
public
CONTRIBUTOR
My PR now should handle monochrome PIL image. Thanks for the heads up :)
2024-03-11T22:29:48Z
https://github.com/huggingface/datasets/issues/6394#issuecomment-1989556834
https://api.github.com/repos/huggingface/datasets/issues/6394
IC_kwDODunzps52ljpi
https://api.github.com/repos/huggingface/datasets/issues/comments/1989556834/reactions
2024-03-11T22:29:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1989556834
https://avatars.githubusercontent.com/u/32204417?v=4
https://api.github.com/users/varunneal/events{/privacy}
https://api.github.com/users/varunneal/followers
https://api.github.com/users/varunneal/following{/other_user}
https://api.github.com/users/varunneal/gists{/gist_id}
https://github.com/varunneal
varunneal
MDQ6VXNlcjMyMjA0NDE3
https://api.github.com/users/varunneal/orgs
https://api.github.com/users/varunneal/received_events
https://api.github.com/users/varunneal/repos
https://api.github.com/users/varunneal/starred{/owner}{/repo}
https://api.github.com/users/varunneal/subscriptions
User
https://api.github.com/users/varunneal
public
### Describe the bug

Using .set_format("torch") leads to images having shape (H, W, C), the same as in numpy. 
However, pytorch normally uses (C, H, W) format.

Maybe I'm missing something but this makes the format a lot less useful as I then have to permute it anyways.
If not using the format it is possible to directly use torchvision transforms but any non-transformed value will not be a tensor.

Is there a reason for this choice?

### Steps to reproduce the bug

```python
from datasets import Dataset, Features, Audio, Image
images = ["path/to/image.png"] * 10
features = Features({"image": Image()})
ds = Dataset.from_dict({"image": images}, features=features) 
ds = ds.with_format("torch")
ds[0]["image"].shape
```
```python
torch.Size([512, 512, 4])
```

### Expected behavior

```python
from datasets import Dataset, Features, Audio, Image
images = ["path/to/image.png"] * 10
features = Features({"image": Image()})
ds = Dataset.from_dict({"image": images}, features=features) 
ds = ds.with_format("torch")
ds[0]["image"].shape
```
```python
torch.Size([4, 512, 512])
```

### Environment info

- `datasets` version: 2.14.6
- Platform: Linux-6.5.9-100.fc37.x86_64-x86_64-with-glibc2.31
- Python version: 3.11.6
- Huggingface_hub version: 0.18.0
- PyArrow version: 14.0.1
- Pandas version: 2.1.2