🐛 Trying to use ROUGE metric : pyarrow.lib.ArrowInvalid: Column 1 named references expected length 534 but got length 323
MEMBER
Can you share your data files or a minimally reproducible example?
2020-05-15T07:42:01Z
https://github.com/huggingface/datasets/issues/116#issuecomment-629084165
https://api.github.com/repos/huggingface/datasets/issues/116
MDEyOklzc3VlQ29tbWVudDYyOTA4NDE2NQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/629084165/reactions
2020-05-15T07:42:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/629084165
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
NONE
Sure, [here is a Colab notebook](https://colab.research.google.com/drive/1uiS89fnHMG7HV_cYxp3r-_LqJQvNNKs9?usp=sharing) reproducing the error.

> ArrowInvalid: Column 1 named references expected length 36 but got length 56
2020-05-17T22:21:47Z
https://github.com/huggingface/datasets/issues/116#issuecomment-629869693
https://api.github.com/repos/huggingface/datasets/issues/116
MDEyOklzc3VlQ29tbWVudDYyOTg2OTY5Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/629869693/reactions
2020-05-17T22:21:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/629869693
https://avatars.githubusercontent.com/u/43774355?v=4
https://api.github.com/users/astariul/events{/privacy}
https://api.github.com/users/astariul/followers
https://api.github.com/users/astariul/following{/other_user}
https://api.github.com/users/astariul/gists{/gist_id}
https://github.com/astariul
astariul
MDQ6VXNlcjQzNzc0MzU1
https://api.github.com/users/astariul/orgs
https://api.github.com/users/astariul/received_events
https://api.github.com/users/astariul/repos
https://api.github.com/users/astariul/starred{/owner}{/repo}
https://api.github.com/users/astariul/subscriptions
User
https://api.github.com/users/astariul
public
MEMBER
This is because `add` takes as input a batch of elements and you provided only one. I think we should have `add` for one prediction/reference and `add_batch` for a batch of predictions/references. This would make it more coherent with the way we use Arrow.

Let me do this change
2020-05-28T13:37:03Z
https://github.com/huggingface/datasets/issues/116#issuecomment-635355007
https://api.github.com/repos/huggingface/datasets/issues/116
MDEyOklzc3VlQ29tbWVudDYzNTM1NTAwNw==
https://api.github.com/repos/huggingface/datasets/issues/comments/635355007/reactions
2020-05-28T13:37:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/635355007
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Thanks for noticing though. I was mainly used to do `.compute` directly ^^
2020-05-28T13:37:39Z
https://github.com/huggingface/datasets/issues/116#issuecomment-635355357
https://api.github.com/repos/huggingface/datasets/issues/116
MDEyOklzc3VlQ29tbWVudDYzNTM1NTM1Nw==
https://api.github.com/repos/huggingface/datasets/issues/comments/635355357/reactions
2020-05-28T13:37:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/635355357
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Thanks @lhoestq it works :)
2020-05-28T23:43:07Z
https://github.com/huggingface/datasets/issues/116#issuecomment-635674202
https://api.github.com/repos/huggingface/datasets/issues/116
MDEyOklzc3VlQ29tbWVudDYzNTY3NDIwMg==
https://api.github.com/repos/huggingface/datasets/issues/comments/635674202/reactions
2020-05-28T23:43:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/635674202
https://avatars.githubusercontent.com/u/43774355?v=4
https://api.github.com/users/astariul/events{/privacy}
https://api.github.com/users/astariul/followers
https://api.github.com/users/astariul/following{/other_user}
https://api.github.com/users/astariul/gists{/gist_id}
https://github.com/astariul
astariul
MDQ6VXNlcjQzNzc0MzU1
https://api.github.com/users/astariul/orgs
https://api.github.com/users/astariul/received_events
https://api.github.com/users/astariul/repos
https://api.github.com/users/astariul/starred{/owner}{/repo}
https://api.github.com/users/astariul/subscriptions
User
https://api.github.com/users/astariul
public
I'm trying to use rouge metric.

I have to files : `test.pred.tokenized` and `test.gold.tokenized` with each line containing a sentence.  
I tried :

```python
import nlp

rouge = nlp.load_metric('rouge')
with open("test.pred.tokenized") as p, open("test.gold.tokenized") as g:
    for lp, lg in zip(p, g):
            rouge.add(lp, lg)
```

But I meet following error :

> pyarrow.lib.ArrowInvalid: Column 1 named references expected length 534 but got length 323

---

Full stack-trace :

```
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "/home/me/.venv/transformers/lib/python3.6/site-packages/nlp/metric.py", line 224, in add
    self.writer.write_batch(batch)
  File "/home/me/.venv/transformers/lib/python3.6/site-packages/nlp/arrow_writer.py", line 148, in write_batch
    pa_table: pa.Table = pa.Table.from_pydict(batch_examples, schema=self._schema)
  File "pyarrow/table.pxi", line 1550, in pyarrow.lib.Table.from_pydict
  File "pyarrow/table.pxi", line 1503, in pyarrow.lib.Table.from_arrays
  File "pyarrow/public-api.pxi", line 390, in pyarrow.lib.pyarrow_wrap_table
  File "pyarrow/error.pxi", line 85, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Column 1 named references expected length 534 but got length 323
```

(`nlp` installed from source)