How to use Image folder
NONE
Even this from docs throw same error
```
dataset = load_dataset("imagefolder", data_files="https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_3367a.zip", split="train")

```
2022-03-09T21:20:01Z
https://github.com/huggingface/datasets/issues/3881#issuecomment-1063381707
https://api.github.com/repos/huggingface/datasets/issues/3881
IC_kwDODunzps4_YerL
https://api.github.com/repos/huggingface/datasets/issues/comments/1063381707/reactions
2022-03-09T21:20:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1063381707
https://avatars.githubusercontent.com/u/45640029?v=4
https://api.github.com/users/rozeappletree/events{/privacy}
https://api.github.com/users/rozeappletree/followers
https://api.github.com/users/rozeappletree/following{/other_user}
https://api.github.com/users/rozeappletree/gists{/gist_id}
https://github.com/rozeappletree
rozeappletree
MDQ6VXNlcjQ1NjQwMDI5
https://api.github.com/users/rozeappletree/orgs
https://api.github.com/users/rozeappletree/received_events
https://api.github.com/users/rozeappletree/repos
https://api.github.com/users/rozeappletree/starred{/owner}{/repo}
https://api.github.com/users/rozeappletree/subscriptions
User
https://api.github.com/users/rozeappletree
public
MEMBER
Hi @INF800,

Please note that the `imagefolder` feature enhancement was just recently merged to our master branch (https://github.com/huggingface/datasets/commit/207be676bffe9d164740a41a883af6125edef135), but has not yet been released.

We are planning to make the 2.0 release of our library in the coming days and then that feature will be available by updating your `datasets` library from PyPI.

In the meantime, you can incorporate that feature if you install our library from our GitHub master branch:
```shell
pip install git+https://github.com/huggingface/datasets#egg=datasets
```

Then:
```python
In [1]: from datasets import load_dataset
        ds = load_dataset("imagefolder", data_files="https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_3367a.zip", split="train")
Using custom data configuration default-7eb4e80d960deb18
Downloading and preparing dataset image_folder/default to .../.cache/huggingface/datasets/image_folder/default-7eb4e80d960deb18/0.0.0/8de8dc6d68ce3c81cc102b93cc82ede27162b5d30cd003094f935942c8294f60...
Downloading data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 690.19it/s]
Extracting data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 852.85it/s]
Dataset image_folder downloaded and prepared to .../.cache/huggingface/datasets/image_folder/default-7eb4e80d960deb18/0.0.0/8de8dc6d68ce3c81cc102b93cc82ede27162b5d30cd003094f935942c8294f60. Subsequent calls will reuse this data.

In [2]: ds
Out[2]: 
Dataset({
    features: ['image', 'label'],
    num_rows: 25000
})
```
2022-03-10T06:29:51Z
https://github.com/huggingface/datasets/issues/3881#issuecomment-1063710342
https://api.github.com/repos/huggingface/datasets/issues/3881
IC_kwDODunzps4_Zu6G
https://api.github.com/repos/huggingface/datasets/issues/comments/1063710342/reactions
2022-03-10T06:29:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1063710342
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
NONE
Hey @albertvillanova. Does this load entire dataset in memory? Because I am facing huge trouble with loading very big datasets (OOM errors)
2022-03-10T06:43:28Z
https://github.com/huggingface/datasets/issues/3881#issuecomment-1063716789
https://api.github.com/repos/huggingface/datasets/issues/3881
IC_kwDODunzps4_Zwe1
https://api.github.com/repos/huggingface/datasets/issues/comments/1063716789/reactions
2022-03-10T06:43:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1063716789
https://avatars.githubusercontent.com/u/45640029?v=4
https://api.github.com/users/rozeappletree/events{/privacy}
https://api.github.com/users/rozeappletree/followers
https://api.github.com/users/rozeappletree/following{/other_user}
https://api.github.com/users/rozeappletree/gists{/gist_id}
https://github.com/rozeappletree
rozeappletree
MDQ6VXNlcjQ1NjQwMDI5
https://api.github.com/users/rozeappletree/orgs
https://api.github.com/users/rozeappletree/received_events
https://api.github.com/users/rozeappletree/repos
https://api.github.com/users/rozeappletree/starred{/owner}{/repo}
https://api.github.com/users/rozeappletree/subscriptions
User
https://api.github.com/users/rozeappletree
public
COLLABORATOR
Can you provide the error stack trace? The loader only stores the `data_files` dict, which can get big after globbing. Then, the OOM error would mean you don't have enough memory to keep all the paths to the image files. You can circumvent this by generating an archive and loading the dataset from there. Maybe we can optimize the globbing part in our data files resolution at some point, cc @lhoestq for visibility.
2022-03-10T12:27:59Z
https://github.com/huggingface/datasets/issues/3881#issuecomment-1064002694
https://api.github.com/repos/huggingface/datasets/issues/3881
IC_kwDODunzps4_a2SG
https://api.github.com/repos/huggingface/datasets/issues/comments/1064002694/reactions
2022-03-10T12:27:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1064002694
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
Hey, memory error is resolved. It was fluke.

But there is another issue. Currently `load_dataset("imagefolder", data_dir="./path/to/train",)` takes only `train` as arg to `split` parameter.

I am creating vaildation dataset using

```
ds_valid = datasets.DatasetDict(valid=load_dataset("imagefolder", data_dir="./path/to/valid",)['train'])
```
2022-03-10T12:36:57Z
https://github.com/huggingface/datasets/issues/3881#issuecomment-1064010251
https://api.github.com/repos/huggingface/datasets/issues/3881
IC_kwDODunzps4_a4IL
https://api.github.com/repos/huggingface/datasets/issues/comments/1064010251/reactions
2022-03-10T12:37:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1064010251
https://avatars.githubusercontent.com/u/45640029?v=4
https://api.github.com/users/rozeappletree/events{/privacy}
https://api.github.com/users/rozeappletree/followers
https://api.github.com/users/rozeappletree/following{/other_user}
https://api.github.com/users/rozeappletree/gists{/gist_id}
https://github.com/rozeappletree
rozeappletree
MDQ6VXNlcjQ1NjQwMDI5
https://api.github.com/users/rozeappletree/orgs
https://api.github.com/users/rozeappletree/received_events
https://api.github.com/users/rozeappletree/repos
https://api.github.com/users/rozeappletree/starred{/owner}{/repo}
https://api.github.com/users/rozeappletree/subscriptions
User
https://api.github.com/users/rozeappletree
public
COLLABORATOR
`data_dir="path/to/folder"` is a shorthand syntax fox `data_files={"train": "path/to/folder/**"}`, so use `data_files` in that case instead:
```python
ds = load_dataset("imagefolder", data_files={"train": "path/to/train/**", "test": "path/to/test/**", "valid": "path/to/valid/**"})
```
2022-03-10T13:24:01Z
https://github.com/huggingface/datasets/issues/3881#issuecomment-1064058708
https://api.github.com/repos/huggingface/datasets/issues/3881
IC_kwDODunzps4_bD9U
https://api.github.com/repos/huggingface/datasets/issues/comments/1064058708/reactions
2022-03-10T13:24:01Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1064058708
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
And there was another issue. I loaded black and white images (jpeg file). Using load dataset. It reads it as PIL jpeg data format. But instead of converting it into 3 channel tensor, input to collator function is coming as a single channel tensor.
2022-03-10T13:43:17Z
https://github.com/huggingface/datasets/issues/3881#issuecomment-1064078328
https://api.github.com/repos/huggingface/datasets/issues/3881
IC_kwDODunzps4_bIv4
https://api.github.com/repos/huggingface/datasets/issues/comments/1064078328/reactions
2022-03-10T13:43:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1064078328
https://avatars.githubusercontent.com/u/45640029?v=4
https://api.github.com/users/rozeappletree/events{/privacy}
https://api.github.com/users/rozeappletree/followers
https://api.github.com/users/rozeappletree/following{/other_user}
https://api.github.com/users/rozeappletree/gists{/gist_id}
https://github.com/rozeappletree
rozeappletree
MDQ6VXNlcjQ1NjQwMDI5
https://api.github.com/users/rozeappletree/orgs
https://api.github.com/users/rozeappletree/received_events
https://api.github.com/users/rozeappletree/repos
https://api.github.com/users/rozeappletree/starred{/owner}{/repo}
https://api.github.com/users/rozeappletree/subscriptions
User
https://api.github.com/users/rozeappletree
public
COLLABORATOR
We don't apply any additional preprocessing on top of `PIL.Image.open(image_file)`, so you need to do the conversion yourself:

```python
def to_rgb(batch):
    batch["image"] = [img.convert("RGB") for img in batch["image"]]
    return batch

ds_rgb = ds.map(to_rgb, batched=True)
```

Please use our Forum for questions of this kind in the future.
2022-03-10T15:04:33Z
https://github.com/huggingface/datasets/issues/3881#issuecomment-1064159251
https://api.github.com/repos/huggingface/datasets/issues/3881
IC_kwDODunzps4_bcgT
https://api.github.com/repos/huggingface/datasets/issues/comments/1064159251/reactions
2022-03-10T15:04:33Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1064159251
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
Ran this code
```
 load_dataset("imagefolder", data_dir="./my-dataset")
```

`https://raw.githubusercontent.com/huggingface/datasets/master/datasets/imagefolder/imagefolder.py` missing
```
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
/tmp/ipykernel_33/1648737256.py in <module>
----> 1 load_dataset("imagefolder", data_dir="./my-dataset")

/opt/conda/lib/python3.7/site-packages/datasets/load.py in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, ignore_verifications, keep_in_memory, save_infos, revision, use_auth_token, task, streaming, script_version, **config_kwargs)
   1684         revision=revision,
   1685         use_auth_token=use_auth_token,
-> 1686         **config_kwargs,
   1687     )
   1688 

/opt/conda/lib/python3.7/site-packages/datasets/load.py in load_dataset_builder(path, name, data_dir, data_files, cache_dir, features, download_config, download_mode, revision, use_auth_token, script_version, **config_kwargs)
   1511         download_config.use_auth_token = use_auth_token
   1512     dataset_module = dataset_module_factory(
-> 1513         path, revision=revision, download_config=download_config, download_mode=download_mode, data_files=data_files
   1514     )
   1515 

/opt/conda/lib/python3.7/site-packages/datasets/load.py in dataset_module_factory(path, revision, download_config, download_mode, force_local_path, dynamic_modules_path, data_files, **download_kwargs)
   1200                         f"Couldn't find a dataset script at {relative_to_absolute_path(combined_path)} or any data file in the same directory. "
   1201                         f"Couldn't find '{path}' on the Hugging Face Hub either: {type(e1).__name__}: {e1}"
-> 1202                     ) from None
   1203                 raise e1 from None
   1204     else:

FileNotFoundError: Couldn't find a dataset script at /kaggle/working/imagefolder/imagefolder.py or any data file in the same directory. Couldn't find 'imagefolder' on the Hugging Face Hub either: FileNotFoundError: Couldn't find file at https://raw.githubusercontent.com/huggingface/datasets/master/datasets/imagefolder/imagefolder.py
```