Empty output/answer in TriviaQA test set (both in 'kilt_tasks' and 'trivia_qa')
MEMBER
cc @yjernite is this expected ?
2020-11-09T09:09:56Z
https://github.com/huggingface/datasets/issues/804#issuecomment-723876580
https://api.github.com/repos/huggingface/datasets/issues/804
MDEyOklzc3VlQ29tbWVudDcyMzg3NjU4MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/723876580/reactions
2020-11-09T09:09:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/723876580
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
Yes: TriviaQA has a private test set for the leaderboard [here](https://competitions.codalab.org/competitions/17208)

For the KILT training and validation portions, you need to link the examples from the TriviaQA dataset as detailed here:
https://github.com/huggingface/datasets/blob/master/datasets/kilt_tasks/README.md
2020-11-09T13:36:18Z
https://github.com/huggingface/datasets/issues/804#issuecomment-724017517
https://api.github.com/repos/huggingface/datasets/issues/804
MDEyOklzc3VlQ29tbWVudDcyNDAxNzUxNw==
https://api.github.com/repos/huggingface/datasets/issues/comments/724017517/reactions
2020-11-09T13:36:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/724017517
https://avatars.githubusercontent.com/u/10469459?v=4
https://api.github.com/users/yjernite/events{/privacy}
https://api.github.com/users/yjernite/followers
https://api.github.com/users/yjernite/following{/other_user}
https://api.github.com/users/yjernite/gists{/gist_id}
https://github.com/yjernite
yjernite
MDQ6VXNlcjEwNDY5NDU5
https://api.github.com/users/yjernite/orgs
https://api.github.com/users/yjernite/received_events
https://api.github.com/users/yjernite/repos
https://api.github.com/users/yjernite/starred{/owner}{/repo}
https://api.github.com/users/yjernite/subscriptions
User
https://api.github.com/users/yjernite
public
CONTRIBUTOR
Oh ok, I guess I read the paper too fast 😅, thank you for your answer!
2020-11-09T14:14:58Z
https://github.com/huggingface/datasets/issues/804#issuecomment-724039334
https://api.github.com/repos/huggingface/datasets/issues/804
MDEyOklzc3VlQ29tbWVudDcyNDAzOTMzNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/724039334/reactions
2020-11-09T14:14:58Z
https://api.github.com/repos/huggingface/datasets/issues/comments/724039334
https://avatars.githubusercontent.com/u/25532159?v=4
https://api.github.com/users/PaulLerner/events{/privacy}
https://api.github.com/users/PaulLerner/followers
https://api.github.com/users/PaulLerner/following{/other_user}
https://api.github.com/users/PaulLerner/gists{/gist_id}
https://github.com/PaulLerner
PaulLerner
MDQ6VXNlcjI1NTMyMTU5
https://api.github.com/users/PaulLerner/orgs
https://api.github.com/users/PaulLerner/received_events
https://api.github.com/users/PaulLerner/repos
https://api.github.com/users/PaulLerner/starred{/owner}{/repo}
https://api.github.com/users/PaulLerner/subscriptions
User
https://api.github.com/users/PaulLerner
public
# The issue

It's all in the title, it appears to be fine on the train and validation sets.

Is there some kind of mapping to do like for the questions (see https://github.com/huggingface/datasets/blob/master/datasets/kilt_tasks/README.md) ? 

# How to reproduce
```py
from datasets import load_dataset
kilt_tasks = load_dataset("kilt_tasks")
trivia_qa = load_dataset('trivia_qa', 'unfiltered.nocontext')
# both in "kilt_tasks"
In [18]: any([output['answer'] for output in kilt_tasks['test_triviaqa']['output']])                                                                                                                        
Out[18]: False
# and "trivia_qa"
In [13]: all([answer['value'] == '<unk>' for answer in trivia_qa['test']['answer']])                                                                                                                        
Out[13]: True
# appears to be fine on the train and validation sets.
In [14]: all([answer['value'] == '<unk>' for answer in trivia_qa['train']['answer']])                                                                                                                       
Out[14]: False

In [15]: all([answer['value'] == '<unk>' for answer in trivia_qa['validation']['answer']])                                                                                                                  
Out[15]: False

In [16]: any([output['answer'] for output in kilt_tasks['train_triviaqa']['output']])                                                                                                                       
Out[16]: True

In [17]: any([output['answer'] for output in kilt_tasks['validation_triviaqa']['output']])                                                                                                                  
Out[17]: True

```