DatasetInfo.__init__() got an unexpected keyword argument '_column_requires_decoding'
COLLABORATOR
Thanks for reporting, but we can only fix this issue if you can provide a reproducer that consistently reproduces it.
2023-08-17T17:39:00Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1682702201
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5kS_95
https://api.github.com/repos/huggingface/datasets/issues/comments/1682702201/reactions
2023-08-17T17:39:00Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1682702201
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
@mariosasko Ok. What exactly does it mean to provide a reproducer
2023-08-18T08:20:56Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1683546457
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5kWOFZ
https://api.github.com/repos/huggingface/datasets/issues/comments/1683546457/reactions
2023-08-18T08:20:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1683546457
https://avatars.githubusercontent.com/u/51043929?v=4
https://api.github.com/users/aihao2000/events{/privacy}
https://api.github.com/users/aihao2000/followers
https://api.github.com/users/aihao2000/following{/other_user}
https://api.github.com/users/aihao2000/gists{/gist_id}
https://github.com/aihao2000
aihao2000
MDQ6VXNlcjUxMDQzOTI5
https://api.github.com/users/aihao2000/orgs
https://api.github.com/users/aihao2000/received_events
https://api.github.com/users/aihao2000/repos
https://api.github.com/users/aihao2000/starred{/owner}{/repo}
https://api.github.com/users/aihao2000/subscriptions
User
https://api.github.com/users/aihao2000
public
COLLABORATOR
To provide a code that reproduces the issue :)
2023-08-18T12:26:28Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1683844747
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5kXW6L
https://api.github.com/repos/huggingface/datasets/issues/comments/1683844747/reactions
2023-08-18T12:26:28Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1683844747
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
@mariosasko I complete the above code, is it enough?
2023-08-18T14:21:46Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1683994371
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5kX7cD
https://api.github.com/repos/huggingface/datasets/issues/comments/1683994371/reactions
2023-08-18T14:21:46Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1683994371
https://avatars.githubusercontent.com/u/51043929?v=4
https://api.github.com/users/aihao2000/events{/privacy}
https://api.github.com/users/aihao2000/followers
https://api.github.com/users/aihao2000/following{/other_user}
https://api.github.com/users/aihao2000/gists{/gist_id}
https://github.com/aihao2000
aihao2000
MDQ6VXNlcjUxMDQzOTI5
https://api.github.com/users/aihao2000/orgs
https://api.github.com/users/aihao2000/received_events
https://api.github.com/users/aihao2000/repos
https://api.github.com/users/aihao2000/starred{/owner}{/repo}
https://api.github.com/users/aihao2000/subscriptions
User
https://api.github.com/users/aihao2000
public
NONE
@mariosasko That's all the code, I'm using locally stored data
2023-08-19T17:30:03Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1685058309
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5kb_MF
https://api.github.com/repos/huggingface/datasets/issues/comments/1685058309/reactions
2023-08-19T17:30:03Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1685058309
https://avatars.githubusercontent.com/u/51043929?v=4
https://api.github.com/users/aihao2000/events{/privacy}
https://api.github.com/users/aihao2000/followers
https://api.github.com/users/aihao2000/following{/other_user}
https://api.github.com/users/aihao2000/gists{/gist_id}
https://github.com/aihao2000
aihao2000
MDQ6VXNlcjUxMDQzOTI5
https://api.github.com/users/aihao2000/orgs
https://api.github.com/users/aihao2000/received_events
https://api.github.com/users/aihao2000/repos
https://api.github.com/users/aihao2000/starred{/owner}{/repo}
https://api.github.com/users/aihao2000/subscriptions
User
https://api.github.com/users/aihao2000
public
COLLABORATOR
Does this error occur even if you change the cache directory (the `cache_dir` parameter in `load_dataset`)?
2023-08-21T17:14:53Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1686725181
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5kiWI9
https://api.github.com/repos/huggingface/datasets/issues/comments/1686725181/reactions
2023-08-21T17:14:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1686725181
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
@mariosasko I didn't add any parameters for catch. Nor did any cache configuration change.
2023-08-22T08:09:56Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1687690842
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5kmB5a
https://api.github.com/repos/huggingface/datasets/issues/comments/1687690842/reactions
2023-08-22T08:09:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1687690842
https://avatars.githubusercontent.com/u/51043929?v=4
https://api.github.com/users/aihao2000/events{/privacy}
https://api.github.com/users/aihao2000/followers
https://api.github.com/users/aihao2000/following{/other_user}
https://api.github.com/users/aihao2000/gists{/gist_id}
https://github.com/aihao2000
aihao2000
MDQ6VXNlcjUxMDQzOTI5
https://api.github.com/users/aihao2000/orgs
https://api.github.com/users/aihao2000/received_events
https://api.github.com/users/aihao2000/repos
https://api.github.com/users/aihao2000/starred{/owner}{/repo}
https://api.github.com/users/aihao2000/subscriptions
User
https://api.github.com/users/aihao2000
public
NONE
@mariosasko And I changed the data file, but executing load_dataset is always the previous result. I had to change something in images.py to use the new results. Using 'cleanup_cache_files' is invalid! Help me.
2023-08-27T12:31:50Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1694656370
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5lAmdy
https://api.github.com/repos/huggingface/datasets/issues/comments/1694656370/reactions
2023-08-27T12:31:50Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1694656370
https://avatars.githubusercontent.com/u/51043929?v=4
https://api.github.com/users/aihao2000/events{/privacy}
https://api.github.com/users/aihao2000/followers
https://api.github.com/users/aihao2000/following{/other_user}
https://api.github.com/users/aihao2000/gists{/gist_id}
https://github.com/aihao2000
aihao2000
MDQ6VXNlcjUxMDQzOTI5
https://api.github.com/users/aihao2000/orgs
https://api.github.com/users/aihao2000/received_events
https://api.github.com/users/aihao2000/repos
https://api.github.com/users/aihao2000/starred{/owner}{/repo}
https://api.github.com/users/aihao2000/subscriptions
User
https://api.github.com/users/aihao2000
public
NONE
@mariosasko I added a comprehensive error message. Check that _column_requires_decoding is being passed where it shouldn't be. DatasetInfo.__init__() Whether this parameter is required
2023-08-27T16:34:40Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1694709517
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5lAzcN
https://api.github.com/repos/huggingface/datasets/issues/comments/1694709517/reactions
2023-08-27T16:34:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1694709517
https://avatars.githubusercontent.com/u/51043929?v=4
https://api.github.com/users/aihao2000/events{/privacy}
https://api.github.com/users/aihao2000/followers
https://api.github.com/users/aihao2000/following{/other_user}
https://api.github.com/users/aihao2000/gists{/gist_id}
https://github.com/aihao2000
aihao2000
MDQ6VXNlcjUxMDQzOTI5
https://api.github.com/users/aihao2000/orgs
https://api.github.com/users/aihao2000/received_events
https://api.github.com/users/aihao2000/repos
https://api.github.com/users/aihao2000/starred{/owner}{/repo}
https://api.github.com/users/aihao2000/subscriptions
User
https://api.github.com/users/aihao2000
public
COLLABORATOR
I can see the issue now... 

You can fix it by returning a `DatasetInfo` object in the `_info` method as follows:
```python
    def _info(self):
        if self.config.name == "similar_pairs":
            features = datasets.Features(
                {
                    "image1": datasets.features.Image(),
                    "prompt1": datasets.Value("string"),
                    "image2": datasets.features.Image(),
                    "prompt2": datasets.Value("string"),
                    "similarity": datasets.Value("float32"),
                }
            )
        elif self.config.name == "image_prompt_pairs":
            features = datasets.Features(
                {"image": datasets.features.Image(), "prompt": datasets.Value("string")}
            )
        return datasets.DatasetInfo(features=features)
```
2023-08-31T16:55:17Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1701416493
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5laY4t
https://api.github.com/repos/huggingface/datasets/issues/comments/1701416493/reactions
2023-08-31T16:55:17Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1701416493
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
NONE
@mariosasko Oh, that's the problem. Thank you very much. Returned the wrong object and it actually works? I've been training with it for a long time
2023-09-01T17:38:26Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1703105894
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5lg1Vm
https://api.github.com/repos/huggingface/datasets/issues/comments/1703105894/reactions
2023-09-01T17:38:26Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1703105894
https://avatars.githubusercontent.com/u/51043929?v=4
https://api.github.com/users/aihao2000/events{/privacy}
https://api.github.com/users/aihao2000/followers
https://api.github.com/users/aihao2000/following{/other_user}
https://api.github.com/users/aihao2000/gists{/gist_id}
https://github.com/aihao2000
aihao2000
MDQ6VXNlcjUxMDQzOTI5
https://api.github.com/users/aihao2000/orgs
https://api.github.com/users/aihao2000/received_events
https://api.github.com/users/aihao2000/repos
https://api.github.com/users/aihao2000/starred{/owner}{/repo}
https://api.github.com/users/aihao2000/subscriptions
User
https://api.github.com/users/aihao2000
public
NONE
@mariosasko The original code can still see progress. emmm, I can't see how many examples is generated so far, so I don't know if we should wait
2023-09-08T12:27:53Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1711592173
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5mBNLt
https://api.github.com/repos/huggingface/datasets/issues/comments/1711592173/reactions
2023-09-08T12:27:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1711592173
https://avatars.githubusercontent.com/u/51043929?v=4
https://api.github.com/users/aihao2000/events{/privacy}
https://api.github.com/users/aihao2000/followers
https://api.github.com/users/aihao2000/following{/other_user}
https://api.github.com/users/aihao2000/gists{/gist_id}
https://github.com/aihao2000
aihao2000
MDQ6VXNlcjUxMDQzOTI5
https://api.github.com/users/aihao2000/orgs
https://api.github.com/users/aihao2000/received_events
https://api.github.com/users/aihao2000/repos
https://api.github.com/users/aihao2000/starred{/owner}{/repo}
https://api.github.com/users/aihao2000/subscriptions
User
https://api.github.com/users/aihao2000
public
COLLABORATOR
The original issue has been addressed, so I'm closing it. 

Please open a new issue if you encounter more errors.
2023-09-27T17:36:14Z
https://github.com/huggingface/datasets/issues/6157#issuecomment-1737817627
https://api.github.com/repos/huggingface/datasets/issues/6157
IC_kwDODunzps5nlP4b
https://api.github.com/repos/huggingface/datasets/issues/comments/1737817627/reactions
2023-09-27T17:36:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1737817627
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
### Describe the bug

When I was in load_dataset, it said "DatasetInfo.__init__() got an unexpected keyword argument '_column_requires_decoding'". The second time I ran it, there was no error and the dataset object worked
```python
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[3], line 1
----> 1 dataset = load_dataset(
      2     "/home/aihao/workspace/DeepLearningContent/datasets/manga",
      3     data_dir="/home/aihao/workspace/DeepLearningContent/datasets/manga",
      4     split="train",
      5 )

File [~/miniconda3/envs/torch/lib/python3.11/site-packages/datasets/load.py:2146](https://vscode-remote+ssh-002dremote-002bhome.vscode-resource.vscode-cdn.net/home/aihao/workspace/DeepLearningContent/datasets/~/miniconda3/envs/torch/lib/python3.11/site-packages/datasets/load.py:2146), in load_dataset(path, name, data_dir, data_files, split, cache_dir, features, download_config, download_mode, verification_mode, ignore_verifications, keep_in_memory, save_infos, revision, token, use_auth_token, task, streaming, num_proc, storage_options, **config_kwargs)
   2142 # Build dataset for splits
   2143 keep_in_memory = (
   2144     keep_in_memory if keep_in_memory is not None else is_small_dataset(builder_instance.info.dataset_size)
   2145 )
-> 2146 ds = builder_instance.as_dataset(split=split, verification_mode=verification_mode, in_memory=keep_in_memory)
   2147 # Rename and cast features to match task schema
   2148 if task is not None:
   2149     # To avoid issuing the same warning twice

File [~/miniconda3/envs/torch/lib/python3.11/site-packages/datasets/builder.py:1190](https://vscode-remote+ssh-002dremote-002bhome.vscode-resource.vscode-cdn.net/home/aihao/workspace/DeepLearningContent/datasets/~/miniconda3/envs/torch/lib/python3.11/site-packages/datasets/builder.py:1190), in DatasetBuilder.as_dataset(self, split, run_post_process, verification_mode, ignore_verifications, in_memory)
   1187 verification_mode = VerificationMode(verification_mode or VerificationMode.BASIC_CHECKS)
   1189 # Create a dataset for each of the given splits
-> 1190 datasets = map_nested(
   1191     partial(
   1192         self._build_single_dataset,
...
File [~/miniconda3/envs/torch/lib/python3.11/site-packages/datasets/info.py:379](https://vscode-remote+ssh-002dremote-002bhome.vscode-resource.vscode-cdn.net/home/aihao/workspace/DeepLearningContent/datasets/~/miniconda3/envs/torch/lib/python3.11/site-packages/datasets/info.py:379), in DatasetInfo.copy(self)
    378 def copy(self) -> "DatasetInfo":
--> 379     return self.__class__(**{k: copy.deepcopy(v) for k, v in self.__dict__.items()})

TypeError: DatasetInfo.__init__() got an unexpected keyword argument '_column_requires_decoding'
```
### Steps to reproduce the bug
/home/aihao/workspace/DeepLearningContent/datasets/images/images.py
```python
from logging import config
import datasets
import os
from PIL import Image
import csv
import json


class ImagesConfig(datasets.BuilderConfig):
    def __init__(self, **kwargs):
        super(ImagesConfig, self).__init__(**kwargs)


class Images(datasets.GeneratorBasedBuilder):
    def _split_generators(self, dl_manager: datasets.DownloadManager):
        return [
            datasets.SplitGenerator(
                name=datasets.Split.TRAIN,
                gen_kwargs={"split": datasets.Split.TRAIN},
            )
        ]

    BUILDER_CONFIGS = [
        ImagesConfig(
            name="similar_pairs",
            description="simliar pair dataset,item is a pair of similar images",
        ),
        ImagesConfig(
            name="image_prompt_pairs",
            description="image prompt pairs",
        ),
    ]

    def _info(self):
        if self.config.name == "similar_pairs":
            return datasets.Features(
                {
                    "image1": datasets.features.Image(),
                    "image2": datasets.features.Image(),
                    "similarity": datasets.Value("float32"),
                }
            )
        elif self.config.name == "image_prompt_pairs":
            return datasets.Features(
                {"image": datasets.features.Image(), "prompt": datasets.Value("string")}
            )

    def _generate_examples(self, split):
        data_path = os.path.join(self.config.data_dir, "data")
        if self.config.name == "similar_pairs":
            prompts = {}
            with open(os.path.join(data_path ,"prompts.json"), "r") as f:
                prompts = json.load(f)
            with open(os.path.join(data_path, "similar_pairs.csv"), "r") as f:
                reader = csv.reader(f)
                for row in reader:
                    image1_path, image2_path, similarity = row
                    yield image1_path + ":" + image2_path + ":", {
                        "image1": Image.open(image1_path),
                        "prompt1": prompts[image1_path],
                        "image2": Image.open(image2_path),
                        "prompt2": prompts[image2_path],
                        "similarity": float(similarity),
                    }
```

Code that indicates an error:
```python
from datasets import load_dataset
import json
import csv
import ast
import torch
data_dir = "/home/aihao/workspace/DeepLearningContent/datasets/images"
dataset = load_dataset(data_dir, data_dir=data_dir, name="similar_pairs")
```

### Expected behavior

The first execution gives an error, but it works fine

### Environment info

- `datasets` version: 2.14.3
- Platform: Linux-6.2.0-26-generic-x86_64-with-glibc2.35
- Python version: 3.11.4
- Huggingface_hub version: 0.16.4
- PyArrow version: 12.0.1
- Pandas version: 2.0.3