Issues when run two programs compute the same metrics
MEMBER
Hi ! To avoid collisions you can specify a `experiment_id` when instantiating your metric using `load_metric`. It will replace "default_experiment" with the experiment id that you provide in the arrow filename. 

Also when two `experiment_id` collide we're supposed to detect it using our locking mechanism. Not sure why it didn't work in your case. Could you share some code that reproduces the issue ? This would help us investigate.
2021-01-25T10:39:17Z
https://github.com/huggingface/datasets/issues/1766#issuecomment-766722836
https://api.github.com/repos/huggingface/datasets/issues/1766
MDEyOklzc3VlQ29tbWVudDc2NjcyMjgzNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/766722836/reactions
2021-01-25T10:39:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/766722836
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Thank you for your response. I fixed the issue by set "keep_in_memory=True" when load_metric. 
I cannot share the entire source code but below is the wrapper I wrote:

```python
class Evaluation:
    def __init__(self, metric='sacrebleu'):
        # self.metric = load_metric(metric, keep_in_memory=True)
        self.metric = load_metric(metric)

    def add(self, predictions, references):
        self.metric.add_batch(predictions=predictions, references=references)

    def compute(self):
        return self.metric.compute()['score']
```

Then call the given wrapper as follows:

```python
eval = Evaluation(metric='sacrebleu')
for query, candidates,  labels in tqdm(dataset):
    predictions = net.generate(query)
    references = [[s] for s in labels]
    eval.add(predictions, references)
    if n % 100 == 0:
        bleu += eval.compute()
        eval = Evaluation(metric='sacrebleu')
2021-01-25T11:25:08Z
https://github.com/huggingface/datasets/issues/1766#issuecomment-766748875
https://api.github.com/repos/huggingface/datasets/issues/1766
MDEyOklzc3VlQ29tbWVudDc2Njc0ODg3NQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/766748875/reactions
2021-01-25T11:25:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/766748875
https://avatars.githubusercontent.com/u/8089862?v=4
https://api.github.com/users/lamthuy/events{/privacy}
https://api.github.com/users/lamthuy/followers
https://api.github.com/users/lamthuy/following{/other_user}
https://api.github.com/users/lamthuy/gists{/gist_id}
https://github.com/lamthuy
lamthuy
MDQ6VXNlcjgwODk4NjI=
https://api.github.com/users/lamthuy/orgs
https://api.github.com/users/lamthuy/received_events
https://api.github.com/users/lamthuy/repos
https://api.github.com/users/lamthuy/starred{/owner}{/repo}
https://api.github.com/users/lamthuy/subscriptions
User
https://api.github.com/users/lamthuy
public
I got the following error when running two different programs that both compute sacreblue metrics. It seems that both read/and/write to the same location (.cache/huggingface/metrics/sacrebleu/default/default_experiment-1-0.arrow) where it caches the batches:

```
File "train_matching_min.py", line 160, in <module>ch_9_label
    avg_loss = valid(epoch, args.batch, args.validation, args.with_label)
  File "train_matching_min.py", line 93, in valid
    bleu += eval.compute()
  File "/u/tlhoang/projects/seal/match/models/eval.py", line 23, in compute
    return self.metric.compute()['score']
  File "/dccstor/know/anaconda3/lib/python3.7/site-packages/datasets/metric.py", line 387, in compute
    self._finalize()
  File "/dccstor/know/anaconda3/lib/python3.7/site-packages/datasets/metric.py", line 355, in _finalize
    self.data = Dataset(**reader.read_files([{"filename": f} for f in file_paths]))
  File "/dccstor/know/anaconda3/lib/python3.7/site-packages/datasets/arrow_reader.py", line 231, in read_files
    pa_table = self._read_files(files)
  File "/dccstor/know/anaconda3/lib/python3.7/site-packages/datasets/arrow_reader.py", line 170, in _read_files
    pa_table: pa.Table = self._get_dataset_from_filename(f_dict)
  File "/dccstor/know/anaconda3/lib/python3.7/site-packages/datasets/arrow_reader.py", line 299, in _get_dataset_from_filename
    pa_table = f.read_all()
  File "pyarrow/ipc.pxi", line 481, in pyarrow.lib.RecordBatchReader.read_all
  File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Expected to read 1819307375 metadata bytes, but only read 454396
``` 