bug with sst2 in glue 
MEMBER
Maybe you can use nltk's treebank detokenizer ?
```python
from nltk.tokenize.treebank import TreebankWordDetokenizer

TreebankWordDetokenizer().detokenize("it 's a charming and often affecting journey . ".split())
# "it's a charming and often affecting journey."
```
2020-12-28T13:59:56Z
https://github.com/huggingface/datasets/issues/1639#issuecomment-751721651
https://api.github.com/repos/huggingface/datasets/issues/1639
MDEyOklzc3VlQ29tbWVudDc1MTcyMTY1MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/751721651/reactions
2020-12-28T13:59:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/751721651
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
NONE
I am looking for alternative file URL here instead of adding extra processing code: https://github.com/huggingface/datasets/blob/171f2bba9dd8b92006b13cf076a5bf31d67d3e69/datasets/glue/glue.py#L174
2021-08-05T09:36:56Z
https://github.com/huggingface/datasets/issues/1639#issuecomment-893314595
https://api.github.com/repos/huggingface/datasets/issues/1639
IC_kwDODunzps41PuYj
https://api.github.com/repos/huggingface/datasets/issues/comments/893314595/reactions
2021-08-05T09:39:13Z
https://api.github.com/repos/huggingface/datasets/issues/comments/893314595
https://avatars.githubusercontent.com/u/43598514?v=4
https://api.github.com/users/xinzhel/events{/privacy}
https://api.github.com/users/xinzhel/followers
https://api.github.com/users/xinzhel/following{/other_user}
https://api.github.com/users/xinzhel/gists{/gist_id}
https://github.com/xinzhel
xinzhel
MDQ6VXNlcjQzNTk4NTE0
https://api.github.com/users/xinzhel/orgs
https://api.github.com/users/xinzhel/received_events
https://api.github.com/users/xinzhel/repos
https://api.github.com/users/xinzhel/starred{/owner}{/repo}
https://api.github.com/users/xinzhel/subscriptions
User
https://api.github.com/users/xinzhel
public
MEMBER
I don't know if there exists a detokenized version somewhere. Even the version on kaggle is tokenized
2021-08-27T15:03:23Z
https://github.com/huggingface/datasets/issues/1639#issuecomment-907268781
https://api.github.com/repos/huggingface/datasets/issues/1639
IC_kwDODunzps42E9Kt
https://api.github.com/repos/huggingface/datasets/issues/comments/907268781/reactions
2021-08-27T15:03:23Z
https://api.github.com/repos/huggingface/datasets/issues/comments/907268781
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
Hi
I am getting very low accuracy on SST2 I investigate this and observe that for this dataset sentences are tokenized, while this is correct for the other datasets in GLUE, please see below.
Is there any alternatives I could get untokenized sentences? I am unfortunately under time pressure to report some results on this dataset. thank you for your help. @lhoestq 
 
```
>>> a =  datasets.load_dataset('glue', 'sst2', split="validation", script_version="master")
Reusing dataset glue (/julia/datasets/glue/sst2/1.0.0/7c99657241149a24692c402a5c3f34d4c9f1df5ac2e4c3759fadea38f6cb29c4)
>>> a[:10]
{'idx': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 'label': [1, 0, 1, 1, 0, 1, 0, 0, 1, 0], 'sentence': ["it 's a charming and often affecting journey . ", 'unflinchingly bleak and desperate ', 'allows us to hope that nolan is poised to embark a major career as a commercial yet inventive filmmaker . ', "the acting , costumes , music , cinematography and sound are all astounding given the production 's austere locales . ", "it 's slow -- very , very slow . ", 'although laced with humor and a few fanciful touches , the film is a refreshingly serious look at young women . ', 'a sometimes tedious film . ', "or doing last year 's taxes with your ex-wife . ", "you do n't have to know about music to appreciate the film 's easygoing blend of comedy and romance . ", "in exactly 89 minutes , most of which passed as slowly as if i 'd been sitting naked on an igloo , formula 51 sank from quirky to jerky to utter turkey . "]}

```