The Pile's _FEATURE spec seems to be incorrect
MEMBER
Hi @dlwh, thanks for reporting.

Please note, that the source data files for "all" config are different from the other configurations.

The "all" config contains the official Pile data files, from https://mystic.the-eye.eu/public/AI/pile/
All data examples contain a "meta" dict with a single "pile_set_name" key:
```python
In [1]: from datasets import load_dataset
        ds = load_dataset("the_pile", "all", split="train", streaming=True)
        item = next(iter(ds))
Downloading builder script: 9.09kB [00:00, 4.42MB/s]

In [3]: item["meta"]
Out[3]: {'pile_set_name': 'Pile-CC'}
```

On the other hand, all the other subset configs data files come from the Pile preliminary components directory: https://mystic.the-eye.eu/public/AI/pile_preliminary_components/
For theses components, the "meta" field may have different keys depending on the subset: "id", "language", "pmid",... Because of that, if we had kept the `dict` data format for the "meta" field, we would have an error when trying to concatenate different subsets, whose "meta" keys are not identical. In order to avoid that, the "meta" field is cast to `str` in all these cases, so that there is no incompatibility in their "meta" data type when concatenating.

You can check, for example, that for "pubmed_central" the "meta" field is cast to `str`:
```python
In [4]: from datasets import load_dataset
        ds = load_dataset("the_pile", "pubmed_central", split="train", streaming=True)
        item = next(iter(ds))

In [5]: item["meta"]
Out[5]: "{'id': 'PMC6071596'}"
```

Feel free to reopen this issue if you have further questions. 
2022-03-08T09:30:48Z
https://github.com/huggingface/datasets/issues/3828#issuecomment-1061577710
https://api.github.com/repos/huggingface/datasets/issues/3828
IC_kwDODunzps4_RmPu
https://api.github.com/repos/huggingface/datasets/issues/comments/1061577710/reactions
2022-03-08T09:30:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1061577710
https://avatars.githubusercontent.com/u/8515462?v=4
https://api.github.com/users/albertvillanova/events{/privacy}
https://api.github.com/users/albertvillanova/followers
https://api.github.com/users/albertvillanova/following{/other_user}
https://api.github.com/users/albertvillanova/gists{/gist_id}
https://github.com/albertvillanova
albertvillanova
MDQ6VXNlcjg1MTU0NjI=
https://api.github.com/users/albertvillanova/orgs
https://api.github.com/users/albertvillanova/received_events
https://api.github.com/users/albertvillanova/repos
https://api.github.com/users/albertvillanova/starred{/owner}{/repo}
https://api.github.com/users/albertvillanova/subscriptions
User
https://api.github.com/users/albertvillanova
public
## Describe the bug
If you look at https://huggingface.co/datasets/the_pile/blob/main/the_pile.py:

For "all"
*  the pile_set_name is never set for data
* there's actually an id field inside of "meta" 

For subcorpora pubmed_central and hacker_news:
* the meta is specified to be a string, but it's actually a dict with an id field inside.

## Steps to reproduce the bug


## Expected results
Feature spec should match the data I'd think?

## Actual results
Specify the actual results or traceback.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform:
- Python version:
- PyArrow version:
