IndexError: invalid index of a 0-dim tensor. Use `tensor.item()` in Python or `tensor.item<T>()` in C++ to convert a 0-dim tensor to a number
COLLABORATOR
Hi! The `set_transform` does not apply a custom formatting transform on a single example but the entire batch, so the fixed version of your transform would look as follows:
```python
from datasets import load_dataset
import torch

dataset = load_dataset("lambdalabs/pokemon-blip-captions", split='train')
def t(batch):
    return {"test": torch.tensor([1] * len(batch[next(iter(batch))]))}
    
dataset.set_transform(t)
d_0 = dataset[0]
```

Still, the formatter's error message should mention that a dict of **sequences** is expected as the returned value (not just a dict) to make debugging easier.
2023-02-20T15:47:15Z
https://github.com/huggingface/datasets/issues/5539#issuecomment-1437223277
https://api.github.com/repos/huggingface/datasets/issues/5539
IC_kwDODunzps5Vqklt
https://api.github.com/repos/huggingface/datasets/issues/comments/1437223277/reactions
2023-02-20T15:47:15Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1437223277
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
I can take this
2023-02-20T17:14:04Z
https://github.com/huggingface/datasets/issues/5539#issuecomment-1437335883
https://api.github.com/repos/huggingface/datasets/issues/5539
IC_kwDODunzps5VrAFL
https://api.github.com/repos/huggingface/datasets/issues/comments/1437335883/reactions
2023-02-20T17:14:04Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1437335883
https://avatars.githubusercontent.com/u/26489385?v=4
https://api.github.com/users/Plutone11011/events{/privacy}
https://api.github.com/users/Plutone11011/followers
https://api.github.com/users/Plutone11011/following{/other_user}
https://api.github.com/users/Plutone11011/gists{/gist_id}
https://github.com/Plutone11011
Plutone11011
MDQ6VXNlcjI2NDg5Mzg1
https://api.github.com/users/Plutone11011/orgs
https://api.github.com/users/Plutone11011/received_events
https://api.github.com/users/Plutone11011/repos
https://api.github.com/users/Plutone11011/starred{/owner}{/repo}
https://api.github.com/users/Plutone11011/subscriptions
User
https://api.github.com/users/Plutone11011
public
COLLABORATOR
Fixed in #5553 
2023-02-21T13:03:57Z
https://github.com/huggingface/datasets/issues/5539#issuecomment-1438451721
https://api.github.com/repos/huggingface/datasets/issues/5539
IC_kwDODunzps5VvQgJ
https://api.github.com/repos/huggingface/datasets/issues/comments/1438451721/reactions
2023-02-21T13:03:57Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1438451721
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
> Hi! The `set_transform` does not apply a custom formatting transform on a single example but the entire batch, so the fixed version of your transform would look as follows:
> 
> ```python
> from datasets import load_dataset
> import torch
> 
> dataset = load_dataset("lambdalabs/pokemon-blip-captions", split='train')
> def t(batch):
>     return {"test": torch.tensor([1] * len(batch[next(iter(batch))]))}
>     
> dataset.set_transform(t)
> d_0 = dataset[0]
> ```
> 
> Still, the formatter's error message should mention that a dict of **sequences** is expected as the returned value (not just a dict) to make debugging easier.

ok, will change it according to suggestion. Thanks for the reply!
2023-02-22T10:30:29Z
https://github.com/huggingface/datasets/issues/5539#issuecomment-1439781727
https://api.github.com/repos/huggingface/datasets/issues/5539
IC_kwDODunzps5V0VNf
https://api.github.com/repos/huggingface/datasets/issues/comments/1439781727/reactions
2023-02-22T10:30:29Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1439781727
https://avatars.githubusercontent.com/u/41912135?v=4
https://api.github.com/users/aalbersk/events{/privacy}
https://api.github.com/users/aalbersk/followers
https://api.github.com/users/aalbersk/following{/other_user}
https://api.github.com/users/aalbersk/gists{/gist_id}
https://github.com/aalbersk
aalbersk
MDQ6VXNlcjQxOTEyMTM1
https://api.github.com/users/aalbersk/orgs
https://api.github.com/users/aalbersk/received_events
https://api.github.com/users/aalbersk/repos
https://api.github.com/users/aalbersk/starred{/owner}{/repo}
https://api.github.com/users/aalbersk/subscriptions
User
https://api.github.com/users/aalbersk
public
### Describe the bug

When dataset contains a 0-dim tensor, formatting.py raises a following error and fails.
```bash
Traceback (most recent call last):
  File "<path>/lib/python3.8/site-packages/datasets/formatting/formatting.py", line 501, in format_row
    return _unnest(formatted_batch)
  File "<path>/lib/python3.8/site-packages/datasets/formatting/formatting.py", line 137, in _unnest
    return {key: array[0] for key, array in py_dict.items()}
  File "<path>/lib/python3.8/site-packages/datasets/formatting/formatting.py", line 137, in <dictcomp>
    return {key: array[0] for key, array in py_dict.items()}
IndexError: invalid index of a 0-dim tensor. Use `tensor.item()` in Python or `tensor.item<T>()` in C++ to convert a 0-dim tensor to a number
```

### Steps to reproduce the bug

Load whichever dataset and add transform method to add 0-dim tensor. Or create/find a dataset containing 0-dim tensor. E.g.

```python
from datasets import load_dataset
import torch

dataset = load_dataset("lambdalabs/pokemon-blip-captions", split='train')
def t(batch):
    return {"test": torch.tensor(1)}
    
dataset.set_transform(t)
d_0 = dataset[0]
```

### Expected behavior

Extractor will correctly get a row from the dataset, even if it contains 0-dim tensor.

### Environment info

`datasets==2.8.0`, but it looks like it is also applicable to main branch version (as of 16th February)