Cannot upload my own dataset
NONE
It's my misunderstanding. I cannot just upload a csv. I need to write a dataset loading script too.
2020-05-21T16:55:53Z
https://github.com/huggingface/datasets/issues/181#issuecomment-632222420
https://api.github.com/repos/huggingface/datasets/issues/181
MDEyOklzc3VlQ29tbWVudDYzMjIyMjQyMA==
https://api.github.com/repos/huggingface/datasets/issues/comments/632222420/reactions
2020-05-21T16:55:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/632222420
https://avatars.githubusercontent.com/u/3155646?v=4
https://api.github.com/users/korakot/events{/privacy}
https://api.github.com/users/korakot/followers
https://api.github.com/users/korakot/following{/other_user}
https://api.github.com/users/korakot/gists{/gist_id}
https://github.com/korakot
korakot
MDQ6VXNlcjMxNTU2NDY=
https://api.github.com/users/korakot/orgs
https://api.github.com/users/korakot/received_events
https://api.github.com/users/korakot/repos
https://api.github.com/users/korakot/starred{/owner}{/repo}
https://api.github.com/users/korakot/subscriptions
User
https://api.github.com/users/korakot
public
NONE
I now try with the sample `datasets/csv` folder. 

    nlp-cli upload csv

The error is still the same

```
2020-05-21 17:20:56.394659: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
About to upload file /content/csv/csv.py to S3 under filename csv/csv.py and namespace korakot
About to upload file /content/csv/dummy/0.0.0/dummy_data.zip to S3 under filename csv/dummy/0.0.0/dummy_data.zip and namespace korakot
Proceed? [Y/n] y
Uploading... This might take a while if files are large
Traceback (most recent call last):
  File "/usr/local/bin/nlp-cli", line 33, in <module>
    service.run()
  File "/usr/local/lib/python3.6/dist-packages/nlp/commands/user.py", line 234, in run
    token=token, filename=filename, filepath=filepath, organization=self.args.organization
  File "/usr/local/lib/python3.6/dist-packages/nlp/hf_api.py", line 141, in presign_and_upload
    urls = self.presign(token, filename=filename, organization=organization)
  File "/usr/local/lib/python3.6/dist-packages/nlp/hf_api.py", line 132, in presign
    return PresignedUrl(**d)
TypeError: __init__() got an unexpected keyword argument 'cdn'
```

2020-05-21T17:23:20Z
https://github.com/huggingface/datasets/issues/181#issuecomment-632237016
https://api.github.com/repos/huggingface/datasets/issues/181
MDEyOklzc3VlQ29tbWVudDYzMjIzNzAxNg==
https://api.github.com/repos/huggingface/datasets/issues/comments/632237016/reactions
2020-05-21T17:25:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/632237016
https://avatars.githubusercontent.com/u/3155646?v=4
https://api.github.com/users/korakot/events{/privacy}
https://api.github.com/users/korakot/followers
https://api.github.com/users/korakot/following{/other_user}
https://api.github.com/users/korakot/gists{/gist_id}
https://github.com/korakot
korakot
MDQ6VXNlcjMxNTU2NDY=
https://api.github.com/users/korakot/orgs
https://api.github.com/users/korakot/received_events
https://api.github.com/users/korakot/repos
https://api.github.com/users/korakot/starred{/owner}{/repo}
https://api.github.com/users/korakot/subscriptions
User
https://api.github.com/users/korakot
public
MEMBER
We haven't tested the dataset upload feature yet cc @julien-c 
This is on our short/mid-term roadmap though
2020-05-24T17:57:32Z
https://github.com/huggingface/datasets/issues/181#issuecomment-633267596
https://api.github.com/repos/huggingface/datasets/issues/181
MDEyOklzc3VlQ29tbWVudDYzMzI2NzU5Ng==
https://api.github.com/repos/huggingface/datasets/issues/comments/633267596/reactions
2020-05-24T17:57:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/633267596
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
MEMBER
Even if I fix the `TypeError: __init__() got an unexpected keyword argument 'cdn'` error, it looks like it still uploads to                                                                                                                               `https://s3.amazonaws.com/models.huggingface.co/bert/<namespace>/<dataset_name>` instead of `https://s3.amazonaws.com/datasets.huggingface.co/nlp/<namespace>/<dataset_name>`
2020-05-29T14:50:06Z
https://github.com/huggingface/datasets/issues/181#issuecomment-636015818
https://api.github.com/repos/huggingface/datasets/issues/181
MDEyOklzc3VlQ29tbWVudDYzNjAxNTgxOA==
https://api.github.com/repos/huggingface/datasets/issues/comments/636015818/reactions
2020-05-29T14:50:06Z
https://api.github.com/repos/huggingface/datasets/issues/comments/636015818
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
MEMBER
@lhoestq The endpoints in https://github.com/huggingface/nlp/blob/master/src/nlp/hf_api.py should be (depending on the type of file):
```
POST   /api/datasets/presign
GET    /api/datasets/listObjs
DELETE /api/datasets/deleteObj
POST   /api/metrics/presign  
GET    /api/metrics/listObjs
DELETE /api/metrics/deleteObj
```

In addition to this, @thomwolf cleaned up the objects with dataclasses but you should revert this and re-align to the hf_api that's in this branch of transformers: https://github.com/huggingface/transformers/pull/4632 (so that potential new JSON attributes in the API output don't break existing versions of any library)
2020-06-01T15:52:18Z
https://github.com/huggingface/datasets/issues/181#issuecomment-636935471
https://api.github.com/repos/huggingface/datasets/issues/181
MDEyOklzc3VlQ29tbWVudDYzNjkzNTQ3MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/636935471/reactions
2020-06-01T15:52:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/636935471
https://avatars.githubusercontent.com/u/326577?v=4
https://api.github.com/users/julien-c/events{/privacy}
https://api.github.com/users/julien-c/followers
https://api.github.com/users/julien-c/following{/other_user}
https://api.github.com/users/julien-c/gists{/gist_id}
https://github.com/julien-c
julien-c
MDQ6VXNlcjMyNjU3Nw==
https://api.github.com/users/julien-c/orgs
https://api.github.com/users/julien-c/received_events
https://api.github.com/users/julien-c/repos
https://api.github.com/users/julien-c/starred{/owner}{/repo}
https://api.github.com/users/julien-c/subscriptions
User
https://api.github.com/users/julien-c
public
MEMBER
New commands are
```
nlp-cli upload_dataset <path/to/dataset>
nlp-cli upload_metric <path/to/metric>
nlp-cli s3_datasets {rm, ls}
nlp-cli s3_metrics {rm, ls}
```
Closing this issue.
2020-06-18T22:14:42Z
https://github.com/huggingface/datasets/issues/181#issuecomment-646332168
https://api.github.com/repos/huggingface/datasets/issues/181
MDEyOklzc3VlQ29tbWVudDY0NjMzMjE2OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/646332168/reactions
2020-06-18T22:14:42Z
https://api.github.com/repos/huggingface/datasets/issues/comments/646332168
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
I look into `nlp-cli` and `user.py` to learn how to upload my own data.

It is supposed to work like this
- Register to get username, password at huggingface.co
- `nlp-cli login` and type username, passworld
- I have a single file to upload at `./ttc/ttc_freq_extra.csv`
- `nlp-cli upload ttc/ttc_freq_extra.csv`

But I got this error.

```
2020-05-21 16:33:52.722464: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
About to upload file /content/ttc/ttc_freq_extra.csv to S3 under filename ttc/ttc_freq_extra.csv and namespace korakot
Proceed? [Y/n] y
Uploading... This might take a while if files are large
Traceback (most recent call last):
  File "/usr/local/bin/nlp-cli", line 33, in <module>
    service.run()
  File "/usr/local/lib/python3.6/dist-packages/nlp/commands/user.py", line 234, in run
    token=token, filename=filename, filepath=filepath, organization=self.args.organization
  File "/usr/local/lib/python3.6/dist-packages/nlp/hf_api.py", line 141, in presign_and_upload
    urls = self.presign(token, filename=filename, organization=organization)
  File "/usr/local/lib/python3.6/dist-packages/nlp/hf_api.py", line 132, in presign
    return PresignedUrl(**d)
TypeError: __init__() got an unexpected keyword argument 'cdn'
```