loading private custom dataset script - authentication error
COLLABORATOR
This issue seems to have been resolved, so I'm closing it.
2023-06-15T14:49:20Z
https://github.com/huggingface/datasets/issues/5930#issuecomment-1593215945
https://api.github.com/repos/huggingface/datasets/issues/5930
IC_kwDODunzps5e9ovJ
https://api.github.com/repos/huggingface/datasets/issues/comments/1593215945/reactions
2023-06-15T14:49:20Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1593215945
https://avatars.githubusercontent.com/u/47462742?v=4
https://api.github.com/users/mariosasko/events{/privacy}
https://api.github.com/users/mariosasko/followers
https://api.github.com/users/mariosasko/following{/other_user}
https://api.github.com/users/mariosasko/gists{/gist_id}
https://github.com/mariosasko
mariosasko
MDQ6VXNlcjQ3NDYyNzQy
https://api.github.com/users/mariosasko/orgs
https://api.github.com/users/mariosasko/received_events
https://api.github.com/users/mariosasko/repos
https://api.github.com/users/mariosasko/starred{/owner}{/repo}
https://api.github.com/users/mariosasko/subscriptions
User
https://api.github.com/users/mariosasko
public
### Describe the bug

Train model with my custom dataset stored in HuggingFace and loaded with the loading script requires authentication but I am not sure how ?


I am logged in in the terminal, in the browser. I receive this error: 


/python3.8/site-packages/datasets/utils/file_utils.py", line 566, in get_from_cache
    raise ConnectionError(f"Couldn't reach {url} ({repr(head_error)})")
ConnectionError: Couldn't reach https://huggingface.co/datasets/fkov/s/blob/main/data/s/train/labels `(ConnectionError('Unauthorized for URL `https://huggingface.co/datasets/fkov/s/blob/main/data/s/train/labels. Please use the parameter `**`use_auth_token=True`**` after logging in with `**`huggingface-cli login`**`'))

when I added: `use_auth_token=True` and logged in via terminal then I received error:

or the same error in different format: 
raise ConnectionError(f"`Couldn't reach {url} (error {response.status_code}`)")
ConnectionError: Couldn't reach https://huggingface.co/datasets/fkov/s/blob/main/data/s/train/labels (`error 401`)




### Steps to reproduce the bug

1.  cloned transformers library locally:
https://huggingface.co/docs/transformers/v4.15.0/examples :

> git clone https://github.com/huggingface/transformers
> cd transformers
> pip install .
> cd /transformers/examples/pytorch/audio-classification
> pip install -r requirements.txt

2. created **loading script** 
> https://huggingface.co/docs/datasets/dataset_script  added next to dataset:

3. uploaded **private custom dataset** with loading script to HuggingFace
> https://huggingface.co/docs/datasets/dataset_script

4. added dataset loading script to **local directory** in the above cloned transformers library:
> cd /transformers/examples/pytorch/audio-classification

5. logged in to HuggingFace on local terminal with :
> **huggingface-cli login**

6.  run the model with the custom dataset stored on HuggingFace with code:  https://github.com/huggingface/transformers/blob/main/examples/pytorch/audio-classification/README.md

     cd /transformers/examples/pytorch/audio-classification
>  python run_audio_classification.py \
>     --model_name_or_path facebook/wav2vec2-base \
>     --output_dir l/users/flck/outputs/wav2vec2-base-s \
>     --overwrite_output_dir \
>     --dataset_name s \
>     --dataset_config_name s \
>     --remove_unused_columns False \
>     --do_train \
>     --do_eval \
>     --fp16 \
>     --learning_rate 3e-5 \
>     --max_length_seconds 1 \
>     --attention_mask False \
>     --warmup_ratio 0.1 \
>     --num_train_epochs 5 \
>     --per_device_train_batch_size 32 \
>     --gradient_accumulation_steps 4 \
>     --per_device_eval_batch_size 32 \
>     --dataloader_num_workers 4 \
>     --logging_strategy steps \
>     --logging_steps 10 \
>     --evaluation_strategy epoch \
>     --save_strategy epoch \
>     --load_best_model_at_end True \
>     --metric_for_best_model accuracy \
>     --save_total_limit 3 \
>     --seed 0 \
>     --push_to_hub \
>     **--use_auth_token=True**  



### Expected behavior

Be able to train a model the https://github.com/huggingface/transformers/blob/main/examples/pytorch/audio-classification/ run_audio_classification.py with private custom dataset stored on HuggingFace.

### Environment info

- datasets version:                2.12.0                  
- `transformers` version: 4.30.0.dev0
- Platform: Linux-5.4.204-ql-generic-12.0-19-x86_64-with-glibc2.17
- Python version: 3.8.12
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu117 (True)

Versions of relevant libraries:
[pip3] numpy==1.24.3
[pip3] torch==2.0.1
[pip3] torchaudio==2.0.2
[conda] numpy                     1.24.3                   pypi_0    pypi
[conda] torch                     2.0.1                    pypi_0    pypi
[conda] torchaudio                2.0.2                    pypi_0    pypi
