Add JFLEG
MEMBER
Hi @j-chim ! You're right it does feel redundant: your option works better, but I'd even suggest having the references in a Sequence feature, which you can declare as:
```
	    features=datasets.Features(
                {
                    "sentence": datasets.Value("string"),
                    "corrections": datasets.Sequence(datasets.Value("string")),
                }
            ),
```

To create the dummy data, you just need to tell the generator which files it should use, which you can do with:
`python datasets-cli dummy_data datasets/<your-dataset-folder> --auto_generate --match_text_files "train*,dev*,test*"`

2020-12-05T20:43:06Z
https://github.com/huggingface/datasets/pull/1144#issuecomment-739413466
https://api.github.com/repos/huggingface/datasets/issues/1144
MDEyOklzc3VlQ29tbWVudDczOTQxMzQ2Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/739413466/reactions
2020-12-05T20:43:06Z
https://api.github.com/repos/huggingface/datasets/issues/comments/739413466
https://avatars.githubusercontent.com/u/10469459?v=4
https://api.github.com/users/yjernite/events{/privacy}
https://api.github.com/users/yjernite/followers
https://api.github.com/users/yjernite/following{/other_user}
https://api.github.com/users/yjernite/gists{/gist_id}
https://github.com/yjernite
yjernite
MDQ6VXNlcjEwNDY5NDU5
https://api.github.com/users/yjernite/orgs
https://api.github.com/users/yjernite/received_events
https://api.github.com/users/yjernite/repos
https://api.github.com/users/yjernite/starred{/owner}{/repo}
https://api.github.com/users/yjernite/subscriptions
User
https://api.github.com/users/yjernite
public
CONTRIBUTOR
Many thanks for this @yjernite! I've incorporated your feedback and sorted out the dummy data.
2020-12-05T22:44:27Z
https://github.com/huggingface/datasets/pull/1144#issuecomment-739426236
https://api.github.com/repos/huggingface/datasets/issues/1144
MDEyOklzc3VlQ29tbWVudDczOTQyNjIzNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/739426236/reactions
2020-12-05T22:44:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/739426236
https://avatars.githubusercontent.com/u/22435209?v=4
https://api.github.com/users/j-chim/events{/privacy}
https://api.github.com/users/j-chim/followers
https://api.github.com/users/j-chim/following{/other_user}
https://api.github.com/users/j-chim/gists{/gist_id}
https://github.com/j-chim
j-chim
MDQ6VXNlcjIyNDM1MjA5
https://api.github.com/users/j-chim/orgs
https://api.github.com/users/j-chim/received_events
https://api.github.com/users/j-chim/repos
https://api.github.com/users/j-chim/starred{/owner}{/repo}
https://api.github.com/users/j-chim/subscriptions
User
https://api.github.com/users/j-chim
public
This PR adds [JFLEG ](https://www.aclweb.org/anthology/E17-2037/), an English grammatical error correction benchmark. 

The tests were successful on real data, although it would be great if I can get some guidance on the **dummy data**. Basically, **for each source sentence there are 4 possible gold standard target sentences**. The original dataset comprise files in a flat structure, labelled by split then by source/target (e.g., dev.src, dev.ref0, ..., dev.ref3). Not sure what is the best way of adding this.

I imagine I can treat each distinct source-target pair as its own split? But having so many copies of the source sentence feels redundant, and it would make it less convenient to end-users who might want to access multiple gold standard targets simultaneously. 