Docs for creating a loading script for image datasets
NONE
_The documentation is not available anymore as the PR was closed or merged._
2022-08-02T20:44:02Z
https://github.com/huggingface/datasets/pull/4783#issuecomment-1203196260
https://api.github.com/repos/huggingface/datasets/issues/4783
IC_kwDODunzps5Ht1Fk
https://api.github.com/repos/huggingface/datasets/issues/comments/1203196260/reactions
2022-09-07T19:10:16Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1203196260
https://avatars.githubusercontent.com/u/99929124?v=4
https://api.github.com/users/HuggingFaceDocBuilderDev/events{/privacy}
https://api.github.com/users/HuggingFaceDocBuilderDev/followers
https://api.github.com/users/HuggingFaceDocBuilderDev/following{/other_user}
https://api.github.com/users/HuggingFaceDocBuilderDev/gists{/gist_id}
https://github.com/HuggingFaceDocBuilderDev
HuggingFaceDocBuilderDev
U_kgDOBfTMJA
https://api.github.com/users/HuggingFaceDocBuilderDev/orgs
https://api.github.com/users/HuggingFaceDocBuilderDev/received_events
https://api.github.com/users/HuggingFaceDocBuilderDev/repos
https://api.github.com/users/HuggingFaceDocBuilderDev/starred{/owner}{/repo}
https://api.github.com/users/HuggingFaceDocBuilderDev/subscriptions
User
https://api.github.com/users/HuggingFaceDocBuilderDev
public
COLLABORATOR
IMO it would make more sense to add a "Create image dataset" page with two main sections - a no-code approach with `imagefolder` + metadata (preferred way), and with a loading script (advanced). It should be clear when to choose which. If we leave this as-is, the user who jumps straight to the Vision section could be under the impression that writing a loading script is the preferred way to share a vision dataset due to how this subsection starts:
```
Write a dataset loading script to share a dataset.
```
  
Also, I think a note explaining how to make a dataset gated/disable the viewer to hide the data would be beneficial (it's pretty common to require submitting a form to access a CV dataset).
2022-08-19T17:21:22Z
https://github.com/huggingface/datasets/pull/4783#issuecomment-1220912639
https://api.github.com/repos/huggingface/datasets/issues/4783
IC_kwDODunzps5IxaX_
https://api.github.com/repos/huggingface/datasets/issues/comments/1220912639/reactions
2022-08-19T17:21:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1220912639
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
Great suggestion @mariosasko! I added your suggestions, let me know what you think. For gated dataset access, I just added a tip referring users to the relevant docs since it's more of a Hub feature than `datasets` feature.
2022-08-20T00:54:50Z
https://github.com/huggingface/datasets/pull/4783#issuecomment-1221193342
https://api.github.com/repos/huggingface/datasets/issues/4783
IC_kwDODunzps5Iye5-
https://api.github.com/repos/huggingface/datasets/issues/comments/1221193342/reactions
2022-08-20T00:54:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1221193342
https://avatars.githubusercontent.com/u/59462357?v=4
https://api.github.com/users/stevhliu/events{/privacy}
https://api.github.com/users/stevhliu/followers
https://api.github.com/users/stevhliu/following{/other_user}
https://api.github.com/users/stevhliu/gists{/gist_id}
https://github.com/stevhliu
stevhliu
MDQ6VXNlcjU5NDYyMzU3
https://api.github.com/users/stevhliu/orgs
https://api.github.com/users/stevhliu/received_events
https://api.github.com/users/stevhliu/repos
https://api.github.com/users/stevhliu/starred{/owner}{/repo}
https://api.github.com/users/stevhliu/subscriptions
User
https://api.github.com/users/stevhliu
public
COLLABORATOR
Thanks, looks much better now :). I would also move the sections explaining how to create an `imagefolder` for the specific task from the [loading page](https://raw.githubusercontent.com/huggingface/datasets/main/docs/source/image_load.mdx) to this one. IMO it makes more sense to have the basic info (imagefolder structure + `load_dataset` call) there + a link to this page for info on how to create an image folder dataset.
2022-08-29T11:29:48Z
https://github.com/huggingface/datasets/pull/4783#issuecomment-1230159561
https://api.github.com/repos/huggingface/datasets/issues/4783
IC_kwDODunzps5JUr7J
https://api.github.com/repos/huggingface/datasets/issues/comments/1230159561/reactions
2022-08-29T11:29:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1230159561
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
MEMBER
Good idea! Moved everything about `imagefolder` + metadata to the create an image dataset section since the `load_dataset` call is the same for different computer vision tasks. 
2022-08-29T17:03:28Z
https://github.com/huggingface/datasets/pull/4783#issuecomment-1230584228
https://api.github.com/repos/huggingface/datasets/issues/4783
IC_kwDODunzps5JWTmk
https://api.github.com/repos/huggingface/datasets/issues/comments/1230584228/reactions
2022-08-29T17:03:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1230584228
https://avatars.githubusercontent.com/u/59462357?v=4
https://api.github.com/users/stevhliu/events{/privacy}
https://api.github.com/users/stevhliu/followers
https://api.github.com/users/stevhliu/following{/other_user}
https://api.github.com/users/stevhliu/gists{/gist_id}
https://github.com/stevhliu
stevhliu
MDQ6VXNlcjU5NDYyMzU3
https://api.github.com/users/stevhliu/orgs
https://api.github.com/users/stevhliu/received_events
https://api.github.com/users/stevhliu/repos
https://api.github.com/users/stevhliu/starred{/owner}{/repo}
https://api.github.com/users/stevhliu/subscriptions
User
https://api.github.com/users/stevhliu
public
MEMBER
Thanks for all the feedbacks! 🥰

What do you think about creating how to share an `ImageFolder` dataset in a separate PR? I think we should create a new section under `Vision` for how to share an image dataset.
2022-09-06T17:02:51Z
https://github.com/huggingface/datasets/pull/4783#issuecomment-1238429218
https://api.github.com/repos/huggingface/datasets/issues/4783
IC_kwDODunzps5J0O4i
https://api.github.com/repos/huggingface/datasets/issues/comments/1238429218/reactions
2022-09-06T17:02:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1238429218
https://avatars.githubusercontent.com/u/59462357?v=4
https://api.github.com/users/stevhliu/events{/privacy}
https://api.github.com/users/stevhliu/followers
https://api.github.com/users/stevhliu/following{/other_user}
https://api.github.com/users/stevhliu/gists{/gist_id}
https://github.com/stevhliu
stevhliu
MDQ6VXNlcjU5NDYyMzU3
https://api.github.com/users/stevhliu/orgs
https://api.github.com/users/stevhliu/received_events
https://api.github.com/users/stevhliu/repos
https://api.github.com/users/stevhliu/starred{/owner}{/repo}
https://api.github.com/users/stevhliu/subscriptions
User
https://api.github.com/users/stevhliu
public
MEMBER
I love it thanks ! I think moving forward we can use CSV instead of JSON Lines in the docs ;)
2022-09-09T17:07:11Z
https://github.com/huggingface/datasets/pull/4783#issuecomment-1242234565
https://api.github.com/repos/huggingface/datasets/issues/4783
IC_kwDODunzps5KCv7F
https://api.github.com/repos/huggingface/datasets/issues/comments/1242234565/reactions
2022-09-09T17:07:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1242234565
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
This PR is a first draft of creating a loading script for image datasets. Feel free to let me know if there are any specificities I'm missing for this. 🙂 

To do:
- [x] Document how to create different configurations.