add wit dataset
CONTRIBUTOR
Opening this up - Here's whats left to make it really shine:
- [ ] Update dataset card (this is being worked on by the author)
- [ ] Add `dataset_info.json`. Requires downloading the entire dataset. I believe @lhoestq mentioned he may have a machine he's using for this sort of thing.

I think both of these _could_ be done in separate PRs, but I'm not picky either way.

Also, we may find that users (and/or the authors) would like additional configurations for languages, image only, embeddings only, etc. I think we can wait and see what the community says on this and address in different PR.
2021-09-29T04:28:17Z
https://github.com/huggingface/datasets/pull/2981#issuecomment-929825844
https://api.github.com/repos/huggingface/datasets/issues/2981
IC_kwDODunzps43bAQ0
https://api.github.com/repos/huggingface/datasets/issues/comments/929825844/reactions
2021-09-29T04:28:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/929825844
https://avatars.githubusercontent.com/u/32437151?v=4
https://api.github.com/users/nateraw/events{/privacy}
https://api.github.com/users/nateraw/followers
https://api.github.com/users/nateraw/following{/other_user}
https://api.github.com/users/nateraw/gists{/gist_id}
https://github.com/nateraw
nateraw
MDQ6VXNlcjMyNDM3MTUx
https://api.github.com/users/nateraw/orgs
https://api.github.com/users/nateraw/received_events
https://api.github.com/users/nateraw/repos
https://api.github.com/users/nateraw/starred{/owner}{/repo}
https://api.github.com/users/nateraw/subscriptions
User
https://api.github.com/users/nateraw
public
MEMBER
Finally got around to this PR, sorry for the delay!

I added some minimal info to the dataset card so we can at least have it on the hub and work more on it later.

Unfortunately, this re-triggered the tests and it looks like we need a rebase? cc @nateraw 
2021-12-14T16:31:32Z
https://github.com/huggingface/datasets/pull/2981#issuecomment-993719520
https://api.github.com/repos/huggingface/datasets/issues/2981
IC_kwDODunzps47OvTg
https://api.github.com/repos/huggingface/datasets/issues/comments/993719520/reactions
2021-12-14T16:31:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/993719520
https://avatars.githubusercontent.com/u/10469459?v=4
https://api.github.com/users/yjernite/events{/privacy}
https://api.github.com/users/yjernite/followers
https://api.github.com/users/yjernite/following{/other_user}
https://api.github.com/users/yjernite/gists{/gist_id}
https://github.com/yjernite
yjernite
MDQ6VXNlcjEwNDY5NDU5
https://api.github.com/users/yjernite/orgs
https://api.github.com/users/yjernite/received_events
https://api.github.com/users/yjernite/repos
https://api.github.com/users/yjernite/starred{/owner}{/repo}
https://api.github.com/users/yjernite/subscriptions
User
https://api.github.com/users/yjernite
public
CONTRIBUTOR
Yea we need to rebase. Also I think it would be best to update the features to all be strings to match the original datasets format. I ran into a lot of issues with incorrect data types deep into the Dataset info Json process because it couldn't cast correctly. 

I mentioned this to @mariosasko - he might pick up this PR if I don't have bandwidth to finish soon. 
2021-12-14T16:36:53Z
https://github.com/huggingface/datasets/pull/2981#issuecomment-993725938
https://api.github.com/repos/huggingface/datasets/issues/2981
IC_kwDODunzps47Ow3y
https://api.github.com/repos/huggingface/datasets/issues/comments/993725938/reactions
2021-12-14T16:36:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/993725938
https://avatars.githubusercontent.com/u/32437151?v=4
https://api.github.com/users/nateraw/events{/privacy}
https://api.github.com/users/nateraw/followers
https://api.github.com/users/nateraw/following{/other_user}
https://api.github.com/users/nateraw/gists{/gist_id}
https://github.com/nateraw
nateraw
MDQ6VXNlcjMyNDM3MTUx
https://api.github.com/users/nateraw/orgs
https://api.github.com/users/nateraw/received_events
https://api.github.com/users/nateraw/repos
https://api.github.com/users/nateraw/starred{/owner}{/repo}
https://api.github.com/users/nateraw/subscriptions
User
https://api.github.com/users/nateraw
public
MEMBER
> I think this is a clear occasion to start supporting "canonical" datasets:
> - without dataset_infos.json file
> - without dummy data

I agree we can support datasets without them. However the dataset_infos.json file can at least be **highly encouraged** since it has very nice info about the size of the dataset in MB and in number of examples, and it also helps guaranteeing the integrity of the dataset when it's downloaded (verification of the file checksums and the split sizes).
2022-01-05T13:08:52Z
https://github.com/huggingface/datasets/pull/2981#issuecomment-1005671349
https://api.github.com/repos/huggingface/datasets/issues/2981
IC_kwDODunzps478VO1
https://api.github.com/repos/huggingface/datasets/issues/comments/1005671349/reactions
2022-01-05T13:08:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1005671349
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
COLLABORATOR
This dataset now lives here: https://huggingface.co/datasets/wikimedia/wit_base. @nateraw @yjernite Thanks for the initial work.
2022-05-05T14:26:21Z
https://github.com/huggingface/datasets/pull/2981#issuecomment-1118620705
https://api.github.com/repos/huggingface/datasets/issues/2981
IC_kwDODunzps5CrMwh
https://api.github.com/repos/huggingface/datasets/issues/comments/1118620705/reactions
2022-05-05T14:26:21Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1118620705
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
Resolves #2902 based on conversation there - would also close #2810. Open to suggestions/help 😀

CC @hassiahk @lhoestq @yjernite 