Private/sensitive data
MEMBER
Hi @MFreidank, you should already be able to load a dataset from local sources, indeed. (ping @lhoestq and @jplu)

We're also thinking about the ability to host private datasets on a hosted bucket with permission management, but that's further down the road.
2020-06-18T10:43:43Z
https://github.com/huggingface/datasets/issues/281#issuecomment-645937456
https://api.github.com/repos/huggingface/datasets/issues/281
MDEyOklzc3VlQ29tbWVudDY0NTkzNzQ1Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/645937456/reactions
2020-06-18T10:43:43Z
https://api.github.com/repos/huggingface/datasets/issues/comments/645937456
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
CONTRIBUTOR
Hi @MFreidank, it is possible to load a dataset from your local storage, but only CSV/TSV and JSON are supported. To load a dataset in JSON format:

```
nlp.load_dataset(path="json", data_files={nlp.Split.TRAIN: ["path/to/train.json"], nlp.Split.TEST: ["path/to/test.json"]})
```

For CSV/TSV datasets, you have to replace `json` by `csv`.
2020-06-18T11:17:40Z
https://github.com/huggingface/datasets/issues/281#issuecomment-645951971
https://api.github.com/repos/huggingface/datasets/issues/281
MDEyOklzc3VlQ29tbWVudDY0NTk1MTk3MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/645951971/reactions
2020-06-18T11:17:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/645951971
https://avatars.githubusercontent.com/u/959590?v=4
https://api.github.com/users/jplu/events{/privacy}
https://api.github.com/users/jplu/followers
https://api.github.com/users/jplu/following{/other_user}
https://api.github.com/users/jplu/gists{/gist_id}
https://github.com/jplu
jplu
MDQ6VXNlcjk1OTU5MA==
https://api.github.com/users/jplu/orgs
https://api.github.com/users/jplu/received_events
https://api.github.com/users/jplu/repos
https://api.github.com/users/jplu/starred{/owner}{/repo}
https://api.github.com/users/jplu/subscriptions
User
https://api.github.com/users/jplu
public
CONTRIBUTOR
Hi @julien-c @jplu,
Thanks for sharing this solution with me, it helps, this is what I was looking for. 
If not already there and only missed by me, this could be a great addition in the docs.

Closing my issue as resolved, thanks again.
2020-06-20T13:15:12Z
https://github.com/huggingface/datasets/issues/281#issuecomment-646993224
https://api.github.com/repos/huggingface/datasets/issues/281
MDEyOklzc3VlQ29tbWVudDY0Njk5MzIyNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/646993224/reactions
2020-06-20T13:15:12Z
https://api.github.com/repos/huggingface/datasets/issues/comments/646993224
https://avatars.githubusercontent.com/u/6368040?v=4
https://api.github.com/users/MFreidank/events{/privacy}
https://api.github.com/users/MFreidank/followers
https://api.github.com/users/MFreidank/following{/other_user}
https://api.github.com/users/MFreidank/gists{/gist_id}
https://github.com/MFreidank
MFreidank
MDQ6VXNlcjYzNjgwNDA=
https://api.github.com/users/MFreidank/orgs
https://api.github.com/users/MFreidank/received_events
https://api.github.com/users/MFreidank/repos
https://api.github.com/users/MFreidank/starred{/owner}{/repo}
https://api.github.com/users/MFreidank/subscriptions
User
https://api.github.com/users/MFreidank
public
Hi all,
Thanks for this fantastic library, it makes it very easy to do prototyping for NLP projects interchangeably between TF/Pytorch. 

Unfortunately, there is data that cannot easily be shared publicly as it may contain sensitive information. 
Is there support/a plan to support such data with NLP, e.g. by reading it from local sources?

Use case flow could look like this: use NLP to prototype an approach on similar, public data and apply the resulting prototype on sensitive/private data without the need to rethink data processing pipelines. 

Many thanks for your responses ahead of time and kind regards,
MFreidank