Empty evidence document ("provenance") in KILT ELI5 dataset
NONE
Why did you close this issue? How did you end up finding the evidence documents? I'm running into a similar issue with other KILT tasks.
2022-12-19T19:25:14Z
https://github.com/huggingface/datasets/issues/2001#issuecomment-1358148165
https://api.github.com/repos/huggingface/datasets/issues/2001
IC_kwDODunzps5Q87JF
https://api.github.com/repos/huggingface/datasets/issues/comments/1358148165/reactions
2022-12-19T19:25:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1358148165
https://avatars.githubusercontent.com/u/821442?v=4
https://api.github.com/users/rhofour/events{/privacy}
https://api.github.com/users/rhofour/followers
https://api.github.com/users/rhofour/following{/other_user}
https://api.github.com/users/rhofour/gists{/gist_id}
https://github.com/rhofour
rhofour
MDQ6VXNlcjgyMTQ0Mg==
https://api.github.com/users/rhofour/orgs
https://api.github.com/users/rhofour/received_events
https://api.github.com/users/rhofour/repos
https://api.github.com/users/rhofour/starred{/owner}{/repo}
https://api.github.com/users/rhofour/subscriptions
User
https://api.github.com/users/rhofour
public
In the  original KILT benchmark(https://github.com/facebookresearch/KILT), 

all samples has its evidence document (i.e. wikipedia page id) for prediction.

For example, a sample in ELI5 dataset has the format including provenance (=evidence document) like this

`{"id": "1kiwfx", "input": "In Trading Places (1983, Akroyd/Murphy) how does the scheme at the end of the movie work? Why would buying a lot of OJ at a high price ruin the Duke Brothers?", "output": [{"answer": "I feel so old. People have been askinbg what happened at the end of this movie for what must be the last 15 years of my life. It never stops. Every year/month/fortnight, I see someone asking what happened, and someone explaining. Andf it will keep on happening, until I am 90yrs old, in a home, with nothing but the Internet and my bladder to keep me going. And there it will be: \"what happens at the end of Trading Places?\""}, {"provenance": [{"wikipedia_id": "242855", "title": "Futures contract", "section": "Section::::Abstract.", "start_paragraph_id": 1, "start_character": 14, "end_paragraph_id": 1, "end_character": 612, "bleu_score": 0.9232808519770748}]}], "meta": {"partial_evidence": [{"wikipedia_id": "520990", "title": "Trading Places", "section": "Section::::Plot.\n", "start_paragraph_id": 7, "end_paragraph_id": 7, "meta": {"evidence_span": ["On television, they learn that Clarence Beeks is transporting a secret USDA report on orange crop forecasts.", "On television, they learn that Clarence Beeks is transporting a secret USDA report on orange crop forecasts. Winthorpe and Valentine recall large payments made to Beeks by the Dukes and realize that the Dukes plan to obtain the report to corner the market on frozen orange juice.", "Winthorpe and Valentine recall large payments made to Beeks by the Dukes and realize that the Dukes plan to obtain the report to corner the market on frozen orange juice."]}}]}}`

However, KILT ELI5 dataset from huggingface datasets library only contain empty list of provenance.

`{'id': '1oy5tc', 'input': 'in football whats the point of wasting the first two plays with a rush - up the middle - not regular rush plays i get those', 'meta': {'left_context': '', 'mention': '', 'obj_surface': [], 'partial_evidence': [], 'right_context': '', 'sub_surface': [], 'subj_aliases': [], 'template_questions': []}, 'output': [{'answer': 'In most cases the O-Line is supposed to make a hole for the running back to go through. If you run too many plays to the outside/throws the defense will catch on.\n\nAlso, 2 5 yard plays gets you a new set of downs.', 'meta': {'score': 2}, 'provenance': []}, {'answer': "I you don't like those type of plays, watch CFL.  We only get 3 downs so you can't afford to waste one.  Lots more passing.", 'meta': {'score': 2}, 'provenance': []}]}
`

should i perform other procedure to obtain evidence documents?