DatasetDict save load Failing test in 1.6 not in 1.5
MEMBER
Thanks for reporting ! We're looking into it
2021-04-27T09:16:59Z
https://github.com/huggingface/datasets/issues/2267#issuecomment-827454318
https://api.github.com/repos/huggingface/datasets/issues/2267
MDEyOklzc3VlQ29tbWVudDgyNzQ1NDMxOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/827454318/reactions
2021-04-27T09:16:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/827454318
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
I'm not able to reproduce this, do you think you can provide a code that creates a DatasetDict that has this issue when saving and reloading ?
2021-04-30T12:58:49Z
https://github.com/huggingface/datasets/issues/2267#issuecomment-830077182
https://api.github.com/repos/huggingface/datasets/issues/2267
MDEyOklzc3VlQ29tbWVudDgzMDA3NzE4Mg==
https://api.github.com/repos/huggingface/datasets/issues/comments/830077182/reactions
2021-04-30T12:58:49Z
https://api.github.com/repos/huggingface/datasets/issues/comments/830077182
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Hi, I just ran into a similar error. Here is the minimal code to reproduce:
```python
from datasets import load_dataset, DatasetDict
ds = load_dataset('super_glue', 'multirc')

ds.save_to_disk('tempds')

ds = DatasetDict.load_from_disk('tempds')

```

```bash
Reusing dataset super_glue (/home/idahl/.cache/huggingface/datasets/super_glue/multirc/1.0.2/2fb163bca9085c1deb906aff20f00c242227ff704a4e8c9cfdfe820be3abfc83)
Traceback (most recent call last):
  File "/home/idahl/eval-util-expl/multirc/tmp.py", line 7, in <module>
    ds = DatasetDict.load_from_disk('tempds')
  File "/home/idahl/miniconda3/envs/eval-util-expl/lib/python3.9/site-packages/datasets/dataset_dict.py", line 710, in load_from_disk
    dataset_dict[k] = Dataset.load_from_disk(dataset_dict_split_path, fs, keep_in_memory=keep_in_memory)
  File "/home/idahl/miniconda3/envs/eval-util-expl/lib/python3.9/site-packages/datasets/arrow_dataset.py", line 687, in load_from_disk
    return Dataset(
  File "/home/idahl/miniconda3/envs/eval-util-expl/lib/python3.9/site-packages/datasets/arrow_dataset.py", line 274, in __init__
    raise ValueError(
ValueError: External features info don't match the dataset:
Got
{'answer': Value(dtype='string', id=None), 'idx': {'answer': Value(dtype='int32', id=None), 'paragraph': Value(dtype='int32', id=None), 'question': Value(dtype='int32', id=None)}, 'label': ClassLabel(num_classes=2, names=['False', 'True'], names_file=None, id=None), 'paragraph': Value(dtype='string', id=None), 'question': Value(dtype='string', id=None)}
with type
struct<answer: string, idx: struct<answer: int32, paragraph: int32, question: int32>, label: int64, paragraph: string, question: string>

but expected something like
{'answer': Value(dtype='string', id=None), 'idx': {'paragraph': Value(dtype='int32', id=None), 'question': Value(dtype='int32', id=None), 'answer': Value(dtype='int32', id=None)}, 'label': Value(dtype='int64', id=None), 'paragraph': Value(dtype='string', id=None), 'question': Value(dtype='string', id=None)}
with type
struct<answer: string, idx: struct<paragraph: int32, question: int32, answer: int32>, label: int64, paragraph: string, question: string>

```

The non-matching part seems to be
`'label': ClassLabel(num_classes=2, names=['False', 'True'], names_file=None, id=None),`
vs 
`'label': Value(dtype='int64', id=None),`

And the order in the `<struct...` being different, which might cause the [features.type != inferred_features.type](https://github.com/huggingface/datasets/blob/master/src/datasets/arrow_dataset.py#L274) condition to become true and raise this ValueError.


I am using datasets version 1.6.2.

Edit: can confirm, this works without error in version 1.5.0
2021-05-21T10:41:33Z
https://github.com/huggingface/datasets/issues/2267#issuecomment-845859429
https://api.github.com/repos/huggingface/datasets/issues/2267
MDEyOklzc3VlQ29tbWVudDg0NTg1OTQyOQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/845859429/reactions
2021-05-21T13:51:11Z
https://api.github.com/repos/huggingface/datasets/issues/comments/845859429
https://avatars.githubusercontent.com/u/22561809?v=4
https://api.github.com/users/maxidl/events{/privacy}
https://api.github.com/users/maxidl/followers
https://api.github.com/users/maxidl/following{/other_user}
https://api.github.com/users/maxidl/gists{/gist_id}
https://github.com/maxidl
maxidl
MDQ6VXNlcjIyNTYxODA5
https://api.github.com/users/maxidl/orgs
https://api.github.com/users/maxidl/received_events
https://api.github.com/users/maxidl/repos
https://api.github.com/users/maxidl/starred{/owner}{/repo}
https://api.github.com/users/maxidl/subscriptions
User
https://api.github.com/users/maxidl
public
NONE
My current workaround is to remove the idx feature:

```

from datasets import load_dataset, DatasetDict, Value
ds = load_dataset('super_glue', 'multirc')
ds = ds.remove_columns('idx')

ds.save_to_disk('tempds')

ds = DatasetDict.load_from_disk('tempds')

```

works.
2021-05-21T14:13:47Z
https://github.com/huggingface/datasets/issues/2267#issuecomment-845979805
https://api.github.com/repos/huggingface/datasets/issues/2267
MDEyOklzc3VlQ29tbWVudDg0NTk3OTgwNQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/845979805/reactions
2021-05-21T14:13:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/845979805
https://avatars.githubusercontent.com/u/22561809?v=4
https://api.github.com/users/maxidl/events{/privacy}
https://api.github.com/users/maxidl/followers
https://api.github.com/users/maxidl/following{/other_user}
https://api.github.com/users/maxidl/gists{/gist_id}
https://github.com/maxidl
maxidl
MDQ6VXNlcjIyNTYxODA5
https://api.github.com/users/maxidl/orgs
https://api.github.com/users/maxidl/received_events
https://api.github.com/users/maxidl/repos
https://api.github.com/users/maxidl/starred{/owner}{/repo}
https://api.github.com/users/maxidl/subscriptions
User
https://api.github.com/users/maxidl
public
MEMBER
It looks like this issue comes from the order of the fields in the 'idx' struct that is different for some reason.
I'm looking into it. Note that as a workaround you can also flatten the nested features with `ds = ds.flatten()`
2021-05-28T14:12:14Z
https://github.com/huggingface/datasets/issues/2267#issuecomment-850447775
https://api.github.com/repos/huggingface/datasets/issues/2267
MDEyOklzc3VlQ29tbWVudDg1MDQ0Nzc3NQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/850447775/reactions
2021-05-28T14:12:14Z
https://api.github.com/repos/huggingface/datasets/issues/comments/850447775
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
I just pushed a fix on `master`. We'll do a new release soon !

Thanks for reporting
2021-05-28T15:27:34Z
https://github.com/huggingface/datasets/issues/2267#issuecomment-850497783
https://api.github.com/repos/huggingface/datasets/issues/2267
MDEyOklzc3VlQ29tbWVudDg1MDQ5Nzc4Mw==
https://api.github.com/repos/huggingface/datasets/issues/comments/850497783/reactions
2021-05-28T15:27:34Z
https://api.github.com/repos/huggingface/datasets/issues/comments/850497783
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
## Describe the bug

We have a test that saves a DatasetDict to disk and then loads it from disk. In 1.6 there is an incompatibility in the schema.




Downgrading to `>1.6` -- fixes the problem.

## Steps to reproduce the bug
```python

### Load a dataset dict from jsonl 

path = '/test/foo'

ds_dict.save_to_disk(path)

ds_from_disk = DatasetDict.load_from_disk(path).  ## <-- this is where I see the error on 1.6
```

## Expected results

Upgrading to 1.6 shouldn't break that test. We should be able to serialize to and from disk.

## Actual results
```
        # Infer features if None
        inferred_features = Features.from_arrow_schema(arrow_table.schema)
        if self.info.features is None:
            self.info.features = inferred_features
    
        # Infer fingerprint if None
    
        if self._fingerprint is None:
            self._fingerprint = generate_fingerprint(self)
    
        # Sanity checks
    
        assert self.features is not None, "Features can't be None in a Dataset object"
        assert self._fingerprint is not None, "Fingerprint can't be None in a Dataset object"
        if self.info.features.type != inferred_features.type:
>           raise ValueError(
                "External features info don't match the dataset:\nGot\n{}\nwith type\n{}\n\nbut expected something like\n{}\nwith type\n{}".format(
                    self.info.features, self.info.features.type, inferred_features, inferred_features.type
                )
            )
E           ValueError: External features info don't match the dataset:
E           Got
E           {'_input_hash': Value(dtype='int64', id=None), '_task_hash': Value(dtype='int64', id=None), '_view_id': Value(dtype='string', id=None), 'answer': Value(dtype='string', id=None), 'encoding__ids': Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), 'encoding__offsets': Sequence(feature=Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), length=-1, id=None), 'encoding__overflowing': Sequence(feature=Value(dtype='null', id=None), length=-1, id=None), 'encoding__tokens': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None), 'encoding__words': Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), 'ner_ids': Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), 'ner_labels': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None), 'relations': [{'child': Value(dtype='int64', id=None), 'child_span': {'end': Value(dtype='int64', id=None), 'label': Value(dtype='string', id=None), 'start': Value(dtype='int64', id=None), 'token_end': Value(dtype='int64', id=None), 'token_start': Value(dtype='int64', id=None)}, 'color': Value(dtype='string', id=None), 'head': Value(dtype='int64', id=None), 'head_span': {'end': Value(dtype='int64', id=None), 'label': Value(dtype='string', id=None), 'start': Value(dtype='int64', id=None), 'token_end': Value(dtype='int64', id=None), 'token_start': Value(dtype='int64', id=None)}, 'label': Value(dtype='string', id=None)}], 'spans': [{'end': Value(dtype='int64', id=None), 'label': Value(dtype='string', id=None), 'start': Value(dtype='int64', id=None), 'text': Value(dtype='string', id=None), 'token_end': Value(dtype='int64', id=None), 'token_start': Value(dtype='int64', id=None), 'type': Value(dtype='string', id=None)}], 'text': Value(dtype='string', id=None), 'tokens': [{'disabled': Value(dtype='bool', id=None), 'end': Value(dtype='int64', id=None), 'id': Value(dtype='int64', id=None), 'start': Value(dtype='int64', id=None), 'text': Value(dtype='string', id=None), 'ws': Value(dtype='bool', id=None)}]}
E           with type
E           struct<_input_hash: int64, _task_hash: int64, _view_id: string, answer: string, encoding__ids: list<item: int64>, encoding__offsets: list<item: list<item: int64>>, encoding__overflowing: list<item: null>, encoding__tokens: list<item: string>, encoding__words: list<item: int64>, ner_ids: list<item: int64>, ner_labels: list<item: string>, relations: list<item: struct<child: int64, child_span: struct<end: int64, label: string, start: int64, token_end: int64, token_start: int64>, color: string, head: int64, head_span: struct<end: int64, label: string, start: int64, token_end: int64, token_start: int64>, label: string>>, spans: list<item: struct<end: int64, label: string, start: int64, text: string, token_end: int64, token_start: int64, type: string>>, text: string, tokens: list<item: struct<disabled: bool, end: int64, id: int64, start: int64, text: string, ws: bool>>>
E           
E           but expected something like
E           {'_input_hash': Value(dtype='int64', id=None), '_task_hash': Value(dtype='int64', id=None), '_view_id': Value(dtype='string', id=None), 'answer': Value(dtype='string', id=None), 'encoding__ids': Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), 'encoding__offsets': Sequence(feature=Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), length=-1, id=None), 'encoding__overflowing': Sequence(feature=Value(dtype='null', id=None), length=-1, id=None), 'encoding__tokens': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None), 'encoding__words': Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), 'ner_ids': Sequence(feature=Value(dtype='int64', id=None), length=-1, id=None), 'ner_labels': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None), 'relations': [{'head': Value(dtype='int64', id=None), 'child': Value(dtype='int64', id=None), 'head_span': {'start': Value(dtype='int64', id=None), 'end': Value(dtype='int64', id=None), 'token_start': Value(dtype='int64', id=None), 'token_end': Value(dtype='int64', id=None), 'label': Value(dtype='string', id=None)}, 'child_span': {'start': Value(dtype='int64', id=None), 'end': Value(dtype='int64', id=None), 'token_start': Value(dtype='int64', id=None), 'token_end': Value(dtype='int64', id=None), 'label': Value(dtype='string', id=None)}, 'color': Value(dtype='string', id=None), 'label': Value(dtype='string', id=None)}], 'spans': [{'text': Value(dtype='string', id=None), 'start': Value(dtype='int64', id=None), 'token_start': Value(dtype='int64', id=None), 'token_end': Value(dtype='int64', id=None), 'end': Value(dtype='int64', id=None), 'type': Value(dtype='string', id=None), 'label': Value(dtype='string', id=None)}], 'text': Value(dtype='string', id=None), 'tokens': [{'text': Value(dtype='string', id=None), 'start': Value(dtype='int64', id=None), 'end': Value(dtype='int64', id=None), 'id': Value(dtype='int64', id=None), 'ws': Value(dtype='bool', id=None), 'disabled': Value(dtype='bool', id=None)}]}
E           with type
E           struct<_input_hash: int64, _task_hash: int64, _view_id: string, answer: string, encoding__ids: list<item: int64>, encoding__offsets: list<item: list<item: int64>>, encoding__overflowing: list<item: null>, encoding__tokens: list<item: string>, encoding__words: list<item: int64>, ner_ids: list<item: int64>, ner_labels: list<item: string>, relations: list<item: struct<head: int64, child: int64, head_span: struct<start: int64, end: int64, token_start: int64, token_end: int64, label: string>, child_span: struct<start: int64, end: int64, token_start: int64, token_end: int64, label: string>, color: string, label: string>>, spans: list<item: struct<text: string, start: int64, token_start: int64, token_end: int64, end: int64, type: string, label: string>>, text: string, tokens: list<item: struct<text: string, start: int64, end: int64, id: int64, ws: bool, disabled: bool>>>

../../../../../.virtualenvs/tf_ner_rel_lib/lib/python3.8/site-packages/datasets/arrow_dataset.py:274: ValueError
```
## Versions
- Datasets: 1.6.1
- Python: 3.8.5 (default, Jan 26 2021, 10:01:04) 
[Clang 12.0.0 (clang-1200.0.32.2)]
- Platform: macOS-10.15.7-x86_64-i386-64bit

```
