Can you please add the Stanford dog dataset?
MEMBER
would you like to give it a try, @dgrnd4? (maybe with the help of the dataset author?)
2022-06-15T15:46:07Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1156638252
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5E8OYs
https://api.github.com/repos/huggingface/datasets/issues/comments/1156638252/reactions
2022-06-15T15:46:07Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1156638252
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
NONE
@julien-c i am sorry but I have no idea about how it works: can I add the dataset by myself, following "instructions to add a new dataset"?
Can I add a dataset even if it's not mine? (it's public in the link that I wrote on the post)

2022-06-15T15:49:27Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1156642375
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5E8PZH
https://api.github.com/repos/huggingface/datasets/issues/comments/1156642375/reactions
2022-06-15T15:49:27Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1156642375
https://avatars.githubusercontent.com/u/69434832?v=4
https://api.github.com/users/dgrnd4/events{/privacy}
https://api.github.com/users/dgrnd4/followers
https://api.github.com/users/dgrnd4/following{/other_user}
https://api.github.com/users/dgrnd4/gists{/gist_id}
https://github.com/dgrnd4
dgrnd4
MDQ6VXNlcjY5NDM0ODMy
https://api.github.com/users/dgrnd4/orgs
https://api.github.com/users/dgrnd4/received_events
https://api.github.com/users/dgrnd4/repos
https://api.github.com/users/dgrnd4/starred{/owner}{/repo}
https://api.github.com/users/dgrnd4/subscriptions
User
https://api.github.com/users/dgrnd4
public
COLLABORATOR
Hi! The [ADD NEW DATASET](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md) instructions are indeed the best place to start. It's also perfectly fine to add a dataset if it's public, even if it's not yours. Let me know if you need some additional pointers.
2022-06-16T10:47:26Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1157515993
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5E_krZ
https://api.github.com/repos/huggingface/datasets/issues/comments/1157515993/reactions
2022-06-16T10:47:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1157515993
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
If no one is working on this, I could take this up!
2022-07-03T07:33:45Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1173029669
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5F6wMl
https://api.github.com/repos/huggingface/datasets/issues/comments/1173029669/reactions
2022-07-03T07:33:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1173029669
https://avatars.githubusercontent.com/u/8711912?v=4
https://api.github.com/users/khushmeeet/events{/privacy}
https://api.github.com/users/khushmeeet/followers
https://api.github.com/users/khushmeeet/following{/other_user}
https://api.github.com/users/khushmeeet/gists{/gist_id}
https://github.com/khushmeeet
khushmeeet
MDQ6VXNlcjg3MTE5MTI=
https://api.github.com/users/khushmeeet/orgs
https://api.github.com/users/khushmeeet/received_events
https://api.github.com/users/khushmeeet/repos
https://api.github.com/users/khushmeeet/starred{/owner}{/repo}
https://api.github.com/users/khushmeeet/subscriptions
User
https://api.github.com/users/khushmeeet
public
NONE
@khushmeeet this is the [link](https://huggingface.co/datasets/dgrnd4/stanford_dog_dataset) where I added the dataset already. If you can I would ask you to do this:
1) The dataset it's all in TRAINING SET: can you please divide it in Training,Test and Validation Set? If you can for each class, take the 80% for the Training set and the 10% for Test and 10% Validation
2) The images has different size, can you please resize all the images in 224,224,3? Look even at the last dimension "3" because some images has dimension 4!

Thank you!!
2022-07-03T08:32:26Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1173037834
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5F6yMK
https://api.github.com/repos/huggingface/datasets/issues/comments/1173037834/reactions
2022-07-04T12:31:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1173037834
https://avatars.githubusercontent.com/u/69434832?v=4
https://api.github.com/users/dgrnd4/events{/privacy}
https://api.github.com/users/dgrnd4/followers
https://api.github.com/users/dgrnd4/following{/other_user}
https://api.github.com/users/dgrnd4/gists{/gist_id}
https://github.com/dgrnd4
dgrnd4
MDQ6VXNlcjY5NDM0ODMy
https://api.github.com/users/dgrnd4/orgs
https://api.github.com/users/dgrnd4/received_events
https://api.github.com/users/dgrnd4/repos
https://api.github.com/users/dgrnd4/starred{/owner}{/repo}
https://api.github.com/users/dgrnd4/subscriptions
User
https://api.github.com/users/dgrnd4
public
COLLABORATOR
Hi @khushmeeet! Thanks for the interest. You can self-assign the issue by commenting `#self-assign` on it. 

Also, I think we can skip @dgrnd4's steps as we try to avoid any custom processing on top of raw data. One can later copy the script and override `_post_process` in it to perform such processing on the generated dataset.
2022-07-04T11:45:38Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1173722026
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5F9ZOq
https://api.github.com/repos/huggingface/datasets/issues/comments/1173722026/reactions
2022-07-04T11:45:38Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1173722026
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
CONTRIBUTOR
Thanks @mariosasko 

@dgrnd4 As dataset is there on Hub, and preprocessing is not recommended. I am not sure if there is any other task to do. However, I can't seem to find relevant `.py` files for this dataset in GitHub repo.
2022-07-04T20:21:01Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1174269184
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5F_e0A
https://api.github.com/repos/huggingface/datasets/issues/comments/1174269184/reactions
2022-07-04T20:21:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1174269184
https://avatars.githubusercontent.com/u/8711912?v=4
https://api.github.com/users/khushmeeet/events{/privacy}
https://api.github.com/users/khushmeeet/followers
https://api.github.com/users/khushmeeet/following{/other_user}
https://api.github.com/users/khushmeeet/gists{/gist_id}
https://github.com/khushmeeet
khushmeeet
MDQ6VXNlcjg3MTE5MTI=
https://api.github.com/users/khushmeeet/orgs
https://api.github.com/users/khushmeeet/received_events
https://api.github.com/users/khushmeeet/repos
https://api.github.com/users/khushmeeet/starred{/owner}{/repo}
https://api.github.com/users/khushmeeet/subscriptions
User
https://api.github.com/users/khushmeeet
public
NONE
@khushmeeet @mariosasko The point is that the images must be processed and must have the same size in order to can be used for things for example "Training". 
2022-07-06T09:39:01Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1176008801
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5GGHhh
https://api.github.com/repos/huggingface/datasets/issues/comments/1176008801/reactions
2022-07-06T09:39:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1176008801
https://avatars.githubusercontent.com/u/69434832?v=4
https://api.github.com/users/dgrnd4/events{/privacy}
https://api.github.com/users/dgrnd4/followers
https://api.github.com/users/dgrnd4/following{/other_user}
https://api.github.com/users/dgrnd4/gists{/gist_id}
https://github.com/dgrnd4
dgrnd4
MDQ6VXNlcjY5NDM0ODMy
https://api.github.com/users/dgrnd4/orgs
https://api.github.com/users/dgrnd4/received_events
https://api.github.com/users/dgrnd4/repos
https://api.github.com/users/dgrnd4/starred{/owner}{/repo}
https://api.github.com/users/dgrnd4/subscriptions
User
https://api.github.com/users/dgrnd4
public
COLLABORATOR
@dgrnd4 Yes, but this can be done after loading (`map` to resize images and `train_test_split` to create extra splits)

@khushmeeet The linked version is implemented as a no-code dataset and is generated directly from the ZIP archive, but our "GitHub" datasets (these are datasets without a user/org namespace on the Hub) need a generation script, and you can find one [here](https://github.com/tensorflow/datasets/blob/master/tensorflow_datasets/image_classification/stanford_dogs.py). `datasets` started as a fork of TFDS, so we share similar script structure, which makes it trivial to adapt it.
2022-07-06T11:41:46Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1176118835
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5GGiYz
https://api.github.com/repos/huggingface/datasets/issues/comments/1176118835/reactions
2022-07-06T11:41:46Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1176118835
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
@mariosasko The point is that if I use something like this:
x_train, x_test = train_test_split(dataset, test_size=0.1) 

to get Train 90% and Test 10%, and then to get the Validation Set (10% of the whole 100%):

```
train_ratio = 0.80
validation_ratio = 0.10
test_ratio = 0.10

x_train, x_test, y_train, y_test = train_test_split(dataX, dataY, test_size=1 - train_ratio)
x_val, x_test, y_val, y_test = train_test_split(x_test, y_test, test_size=test_ratio/(test_ratio + validation_ratio)) 

```

The point is that the structure of the data is:
```
DatasetDict({
    train: Dataset({
        features: ['image', 'label'],
        num_rows: 20580
    })
})

```

So how to extract images and labels?

EDIT --> Split of the dataset in Train-Test-Validation:
```
import datasets
from datasets.dataset_dict import DatasetDict
from datasets import Dataset

percentage_divison_test = int(len(dataset['train'])/100 *10)       # 10%  --> 2058 
percentage_divison_validation = int(len(dataset['train'])/100 *20) # 20%  --> 4116

dataset_ = datasets.DatasetDict({"train": Dataset.from_dict({

                                  'image':  dataset['train'][0 : len(dataset['train']) ]['image'],    
                                  'labels': dataset['train'][0 : len(dataset['train']) ]['label'] }), 
                                 
                                 "test": Dataset.from_dict({  #20580-4116 (validation) ,20580-2058 (test)
                                  'image':  dataset['train'][len(dataset['train']) - percentage_divison_validation : len(dataset['train']) - percentage_divison_test]['image'], 
                                  'labels': dataset['train'][len(dataset['train']) - percentage_divison_validation : len(dataset['train']) - percentage_divison_test]['label'] }), 
                                 
                                  "validation": Dataset.from_dict({ # 20580-2058 (test)
                                  'image':  dataset['train'][len(dataset['train']) - percentage_divison_test : len(dataset['train'])]['image'], 
                                  'labels': dataset['train'][len(dataset['train']) - percentage_divison_test : len(dataset['train'])]['label'] }), 
                                })
```
2022-07-06T18:55:36Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1176565081
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5GIPVZ
https://api.github.com/repos/huggingface/datasets/issues/comments/1176565081/reactions
2022-07-07T08:55:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1176565081
https://avatars.githubusercontent.com/u/69434832?v=4
https://api.github.com/users/dgrnd4/events{/privacy}
https://api.github.com/users/dgrnd4/followers
https://api.github.com/users/dgrnd4/following{/other_user}
https://api.github.com/users/dgrnd4/gists{/gist_id}
https://github.com/dgrnd4
dgrnd4
MDQ6VXNlcjY5NDM0ODMy
https://api.github.com/users/dgrnd4/orgs
https://api.github.com/users/dgrnd4/received_events
https://api.github.com/users/dgrnd4/repos
https://api.github.com/users/dgrnd4/starred{/owner}{/repo}
https://api.github.com/users/dgrnd4/subscriptions
User
https://api.github.com/users/dgrnd4
public
NONE
@mariosasko in order to resize images I'm trying this method: 
```
for i in range(0,len(dataset['train'])): #len(dataset['train'])

  ex = dataset['train'][i] #i
  image = ex['image']
  image = image.convert("RGB") # <class 'PIL.Image.Image'> <PIL.Image.Image image mode=RGB size=500x333 at 0x7F84F1948150>
  image_resized = image.resize(size_to_resize) # <PIL.Image.Image image mode=RGB size=224x224 at 0x7F84F17885D0>

  dataset['train'][i]['image'] = image_resized 
```

Because the DatasetDict is formed by arrows that are immutable, the changing assignment in the last line of code, doesn't work!
Do you have any idea in order to get a valid result?
2022-07-07T13:32:41Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1177621626
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5GMRR6
https://api.github.com/repos/huggingface/datasets/issues/comments/1177621626/reactions
2022-07-07T14:21:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1177621626
https://avatars.githubusercontent.com/u/69434832?v=4
https://api.github.com/users/dgrnd4/events{/privacy}
https://api.github.com/users/dgrnd4/followers
https://api.github.com/users/dgrnd4/following{/other_user}
https://api.github.com/users/dgrnd4/gists{/gist_id}
https://github.com/dgrnd4
dgrnd4
MDQ6VXNlcjY5NDM0ODMy
https://api.github.com/users/dgrnd4/orgs
https://api.github.com/users/dgrnd4/received_events
https://api.github.com/users/dgrnd4/repos
https://api.github.com/users/dgrnd4/starred{/owner}{/repo}
https://api.github.com/users/dgrnd4/subscriptions
User
https://api.github.com/users/dgrnd4
public
CONTRIBUTOR
#self-assign
2022-07-09T00:08:52Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1179437981
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5GTMud
https://api.github.com/repos/huggingface/datasets/issues/comments/1179437981/reactions
2022-07-09T00:08:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1179437981
https://avatars.githubusercontent.com/u/8711912?v=4
https://api.github.com/users/khushmeeet/events{/privacy}
https://api.github.com/users/khushmeeet/followers
https://api.github.com/users/khushmeeet/following{/other_user}
https://api.github.com/users/khushmeeet/gists{/gist_id}
https://github.com/khushmeeet
khushmeeet
MDQ6VXNlcjg3MTE5MTI=
https://api.github.com/users/khushmeeet/orgs
https://api.github.com/users/khushmeeet/received_events
https://api.github.com/users/khushmeeet/repos
https://api.github.com/users/khushmeeet/starred{/owner}{/repo}
https://api.github.com/users/khushmeeet/subscriptions
User
https://api.github.com/users/khushmeeet
public
CONTRIBUTOR
I have raised PR for adding stanford-dog dataset. I have not added any data preprocessing code. Only dataset generation script is there. Let me know any changes required, or anything to add to README.
2022-07-09T04:47:47Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1179478950
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5GTWum
https://api.github.com/repos/huggingface/datasets/issues/comments/1179478950/reactions
2022-07-09T04:47:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1179478950
https://avatars.githubusercontent.com/u/8711912?v=4
https://api.github.com/users/khushmeeet/events{/privacy}
https://api.github.com/users/khushmeeet/followers
https://api.github.com/users/khushmeeet/following{/other_user}
https://api.github.com/users/khushmeeet/gists{/gist_id}
https://github.com/khushmeeet
khushmeeet
MDQ6VXNlcjg3MTE5MTI=
https://api.github.com/users/khushmeeet/orgs
https://api.github.com/users/khushmeeet/received_events
https://api.github.com/users/khushmeeet/repos
https://api.github.com/users/khushmeeet/starred{/owner}{/repo}
https://api.github.com/users/khushmeeet/subscriptions
User
https://api.github.com/users/khushmeeet
public
NONE
Is this issue still open, i am new to open source thus want to take this one as my start.
2023-10-09T06:53:08Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1752434575
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5odAeP
https://api.github.com/repos/huggingface/datasets/issues/comments/1752434575/reactions
2023-10-09T06:53:08Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1752434575
https://avatars.githubusercontent.com/u/95092167?v=4
https://api.github.com/users/zutarich/events{/privacy}
https://api.github.com/users/zutarich/followers
https://api.github.com/users/zutarich/following{/other_user}
https://api.github.com/users/zutarich/gists{/gist_id}
https://github.com/zutarich
zutarich
U_kgDOBar9xw
https://api.github.com/users/zutarich/orgs
https://api.github.com/users/zutarich/received_events
https://api.github.com/users/zutarich/repos
https://api.github.com/users/zutarich/starred{/owner}{/repo}
https://api.github.com/users/zutarich/subscriptions
User
https://api.github.com/users/zutarich
public
COLLABORATOR
@zutarich This issue should have been closed since the dataset in question is available on the Hub [here](https://huggingface.co/datasets/dgrnd4/stanford_dog_dataset).
2023-10-18T18:55:22Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-1769137956
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps5pcuck
https://api.github.com/repos/huggingface/datasets/issues/comments/1769137956/reactions
2023-10-18T18:55:22Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1769137956
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
I didn't know about this issue until now but i added my version of the dataset on the hub **with the bboxes** :
https://huggingface.co/datasets/Alanox/stanford-dogs

Although I could have made it cleaner and built the splits from the .txt files + put into the coco format.
There is a [stanford-dogs.py](https://huggingface.co/datasets/Alanox/stanford-dogs/blob/main/stanford-dogs.py) file if you want to help adding these missing metadatas.
Hope this helps
2024-12-09T15:37:28Z
https://github.com/huggingface/datasets/issues/4504#issuecomment-2528411522
https://api.github.com/repos/huggingface/datasets/issues/4504
IC_kwDODunzps6WtH-C
https://api.github.com/repos/huggingface/datasets/issues/comments/2528411522/reactions
2024-12-09T15:44:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2528411522
https://avatars.githubusercontent.com/u/22567306?v=4
https://api.github.com/users/AlanBlanchet/events{/privacy}
https://api.github.com/users/AlanBlanchet/followers
https://api.github.com/users/AlanBlanchet/following{/other_user}
https://api.github.com/users/AlanBlanchet/gists{/gist_id}
https://github.com/AlanBlanchet
AlanBlanchet
MDQ6VXNlcjIyNTY3MzA2
https://api.github.com/users/AlanBlanchet/orgs
https://api.github.com/users/AlanBlanchet/received_events
https://api.github.com/users/AlanBlanchet/repos
https://api.github.com/users/AlanBlanchet/starred{/owner}{/repo}
https://api.github.com/users/AlanBlanchet/subscriptions
User
https://api.github.com/users/AlanBlanchet
public
## Adding a Dataset
- **Name:** *Stanford dog dataset*
- **Description:** *The dataset is about 120 classes for a total of 20.580 images. You can find the dataset here http://vision.stanford.edu/aditya86/ImageNetDogs/*
- **Paper:** *http://vision.stanford.edu/aditya86/ImageNetDogs/*
- **Data:** *[link to the Github repository or current dataset location](http://vision.stanford.edu/aditya86/ImageNetDogs/)*
- **Motivation:** *The dataset has been built using images and annotation from ImageNet for the task of fine-grained image categorization. It is useful for fine-grain purpose *


Instructions to add a new dataset can be found [here](https://github.com/huggingface/datasets/blob/master/ADD_NEW_DATASET.md).
