Add option to ignore keys/columns when loading a dataset from jsonl(or any other data format)
MEMBER
Good point, I'd be in favor of having the `columns` argument in `JsonConfig` (and the others) to align with `ParquetConfig` to let users choose which columns to load and ignore the rest
2025-06-05T12:29:48Z
https://github.com/huggingface/datasets/issues/7594#issuecomment-2944050758
https://api.github.com/repos/huggingface/datasets/issues/7594
IC_kwDODunzps6veqZG
https://api.github.com/repos/huggingface/datasets/issues/comments/2944050758/reactions
2025-06-05T12:29:48Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2944050758
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Is it possible to ignore columns when using parquet? 
2025-06-05T12:32:24Z
https://github.com/huggingface/datasets/issues/7594#issuecomment-2944062498
https://api.github.com/repos/huggingface/datasets/issues/7594
IC_kwDODunzps6vetQi
https://api.github.com/repos/huggingface/datasets/issues/comments/2944062498/reactions
2025-06-05T12:32:24Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2944062498
https://avatars.githubusercontent.com/u/36810152?v=4
https://api.github.com/users/avishaiElmakies/events{/privacy}
https://api.github.com/users/avishaiElmakies/followers
https://api.github.com/users/avishaiElmakies/following{/other_user}
https://api.github.com/users/avishaiElmakies/gists{/gist_id}
https://github.com/avishaiElmakies
avishaiElmakies
MDQ6VXNlcjM2ODEwMTUy
https://api.github.com/users/avishaiElmakies/orgs
https://api.github.com/users/avishaiElmakies/received_events
https://api.github.com/users/avishaiElmakies/repos
https://api.github.com/users/avishaiElmakies/starred{/owner}{/repo}
https://api.github.com/users/avishaiElmakies/subscriptions
User
https://api.github.com/users/avishaiElmakies
public
MEMBER
Yes, you can pass `columns=...` to load_dataset to select which columns to load, and it is passed to `ParquetConfig` :)
2025-06-05T12:56:31Z
https://github.com/huggingface/datasets/issues/7594#issuecomment-2944181857
https://api.github.com/repos/huggingface/datasets/issues/7594
IC_kwDODunzps6vfKZh
https://api.github.com/repos/huggingface/datasets/issues/comments/2944181857/reactions
2025-06-05T12:56:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2944181857
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
Ok, i didn't know that. 
Anyway, it would be good to add this to others
2025-06-05T12:58:12Z
https://github.com/huggingface/datasets/issues/7594#issuecomment-2944189379
https://api.github.com/repos/huggingface/datasets/issues/7594
IC_kwDODunzps6vfMPD
https://api.github.com/repos/huggingface/datasets/issues/comments/2944189379/reactions
2025-06-05T12:58:12Z
https://api.github.com/repos/huggingface/datasets/issues/comments/2944189379
https://avatars.githubusercontent.com/u/36810152?v=4
https://api.github.com/users/avishaiElmakies/events{/privacy}
https://api.github.com/users/avishaiElmakies/followers
https://api.github.com/users/avishaiElmakies/following{/other_user}
https://api.github.com/users/avishaiElmakies/gists{/gist_id}
https://github.com/avishaiElmakies
avishaiElmakies
MDQ6VXNlcjM2ODEwMTUy
https://api.github.com/users/avishaiElmakies/orgs
https://api.github.com/users/avishaiElmakies/received_events
https://api.github.com/users/avishaiElmakies/repos
https://api.github.com/users/avishaiElmakies/starred{/owner}{/repo}
https://api.github.com/users/avishaiElmakies/subscriptions
User
https://api.github.com/users/avishaiElmakies
public
CONTRIBUTOR
Hi @lhoestq 

I'd like to take this up!

As you suggested, I’ll extend the support for the columns parameter (currently used in ParquetConfig) to JsonConfig as well. This will allow users to selectively load specific keys/columns from .jsonl (or .json) files and ignore the rest — solving the type inconsistency issues in unclean datasets.
2025-06-27T06:15:10Z
https://github.com/huggingface/datasets/issues/7594#issuecomment-3011805626
https://api.github.com/repos/huggingface/datasets/issues/7594
IC_kwDODunzps6zhIG6
https://api.github.com/repos/huggingface/datasets/issues/comments/3011805626/reactions
2025-06-27T17:33:36Z
https://api.github.com/repos/huggingface/datasets/issues/comments/3011805626
https://avatars.githubusercontent.com/u/142811259?v=4
https://api.github.com/users/ArjunJagdale/events{/privacy}
https://api.github.com/users/ArjunJagdale/followers
https://api.github.com/users/ArjunJagdale/following{/other_user}
https://api.github.com/users/ArjunJagdale/gists{/gist_id}
https://github.com/ArjunJagdale
ArjunJagdale
U_kgDOCIMgew
https://api.github.com/users/ArjunJagdale/orgs
https://api.github.com/users/ArjunJagdale/received_events
https://api.github.com/users/ArjunJagdale/repos
https://api.github.com/users/ArjunJagdale/starred{/owner}{/repo}
https://api.github.com/users/ArjunJagdale/subscriptions
User
https://api.github.com/users/ArjunJagdale
public
CONTRIBUTOR
Hi @avishaiElmakies and @lhoestq 

Just wanted to let you know that this is now implemented in #7594
As suggested, support for the `columns=...` argument (previously available for Parquet) has now been extended to **JSON and JSONL** loading via `load_dataset(...)`. You can now load only specific keys/columns and skip the rest — which should help in cases where some fields are unclean, inconsistent, or just unnecessary.

### ✅ Example:

```python
from datasets import load_dataset

dataset = load_dataset("json", data_files="your_data.jsonl", columns=["id", "title"])
print(dataset["train"].column_names)
# Output: ['id', 'title']
```

### 🔧 Summary of changes:

* Added `columns: Optional[List[str]]` to `JsonConfig`
* Updated `_generate_tables()` to filter selected columns
* Forwarded `columns` argument from `load_dataset()` to the config
* Added test case to validate behavior

Let me know if you'd like the same to be added for CSV or others as a follow-up — happy to help.
2025-06-27T17:33:40Z
https://github.com/huggingface/datasets/issues/7594#issuecomment-3013885719
https://api.github.com/repos/huggingface/datasets/issues/7594
IC_kwDODunzps6zpD8X
https://api.github.com/repos/huggingface/datasets/issues/comments/3013885719/reactions
2025-06-27T17:33:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/3013885719
https://avatars.githubusercontent.com/u/142811259?v=4
https://api.github.com/users/ArjunJagdale/events{/privacy}
https://api.github.com/users/ArjunJagdale/followers
https://api.github.com/users/ArjunJagdale/following{/other_user}
https://api.github.com/users/ArjunJagdale/gists{/gist_id}
https://github.com/ArjunJagdale
ArjunJagdale
U_kgDOCIMgew
https://api.github.com/users/ArjunJagdale/orgs
https://api.github.com/users/ArjunJagdale/received_events
https://api.github.com/users/ArjunJagdale/repos
https://api.github.com/users/ArjunJagdale/starred{/owner}{/repo}
https://api.github.com/users/ArjunJagdale/subscriptions
User
https://api.github.com/users/ArjunJagdale
public
NONE
@ArjunJagdale this looks great! Thanks!
I believe that every format that is supported by `datasets` should probably have this feature since it is very useful and will streamline the api (people will know that they can just use `columns` to select the columns they want, and it will  not be dependent on the data format) 
2025-06-27T17:44:35Z
https://github.com/huggingface/datasets/issues/7594#issuecomment-3013907789
https://api.github.com/repos/huggingface/datasets/issues/7594
IC_kwDODunzps6zpJVN
https://api.github.com/repos/huggingface/datasets/issues/comments/3013907789/reactions
2025-06-27T17:45:12Z
https://api.github.com/repos/huggingface/datasets/issues/comments/3013907789
https://avatars.githubusercontent.com/u/36810152?v=4
https://api.github.com/users/avishaiElmakies/events{/privacy}
https://api.github.com/users/avishaiElmakies/followers
https://api.github.com/users/avishaiElmakies/following{/other_user}
https://api.github.com/users/avishaiElmakies/gists{/gist_id}
https://github.com/avishaiElmakies
avishaiElmakies
MDQ6VXNlcjM2ODEwMTUy
https://api.github.com/users/avishaiElmakies/orgs
https://api.github.com/users/avishaiElmakies/received_events
https://api.github.com/users/avishaiElmakies/repos
https://api.github.com/users/avishaiElmakies/starred{/owner}{/repo}
https://api.github.com/users/avishaiElmakies/subscriptions
User
https://api.github.com/users/avishaiElmakies
public
CONTRIBUTOR
Thanks @avishaiElmakies — totally agree, making `columns=...` support consistent across all formats would be really helpful for users.
2025-06-28T09:03:00Z
https://github.com/huggingface/datasets/issues/7594#issuecomment-3015129044
https://api.github.com/repos/huggingface/datasets/issues/7594
IC_kwDODunzps6ztzfU
https://api.github.com/repos/huggingface/datasets/issues/comments/3015129044/reactions
2025-06-28T09:03:00Z
https://api.github.com/repos/huggingface/datasets/issues/comments/3015129044
https://avatars.githubusercontent.com/u/142811259?v=4
https://api.github.com/users/ArjunJagdale/events{/privacy}
https://api.github.com/users/ArjunJagdale/followers
https://api.github.com/users/ArjunJagdale/following{/other_user}
https://api.github.com/users/ArjunJagdale/gists{/gist_id}
https://github.com/ArjunJagdale
ArjunJagdale
U_kgDOCIMgew
https://api.github.com/users/ArjunJagdale/orgs
https://api.github.com/users/ArjunJagdale/received_events
https://api.github.com/users/ArjunJagdale/repos
https://api.github.com/users/ArjunJagdale/starred{/owner}{/repo}
https://api.github.com/users/ArjunJagdale/subscriptions
User
https://api.github.com/users/ArjunJagdale
public
### Feature request

Hi, I would like the option to ignore keys/columns when loading a dataset from files (e.g. jsonl).

### Motivation

I am working on a dataset which is built on jsonl. It seems the dataset is unclean and a column has different types in each row. I can't clean this or remove the column (It is not my data and it is too big for me to clean and save on my own hardware). 
I would like the option to just ignore this column when using `load_dataset`, since i don't need it.
I tried to look if this is already possible but couldn't find a solution. if there is I would love some help. If it is not currently possible, I would love this feature

### Your contribution

I don't think I can help this time, unfortunately.