SST-2 test labels are all -1
CONTRIBUTOR
this also happened to me with `nlp.load_dataset('glue', 'mnli')`
2020-06-06T16:16:27Z
https://github.com/huggingface/datasets/issues/245#issuecomment-640083980
https://api.github.com/repos/huggingface/datasets/issues/245
MDEyOklzc3VlQ29tbWVudDY0MDA4Mzk4MA==
https://api.github.com/repos/huggingface/datasets/issues/comments/640083980/reactions
2020-06-06T16:16:52Z
https://api.github.com/repos/huggingface/datasets/issues/comments/640083980
https://avatars.githubusercontent.com/u/13238952?v=4
https://api.github.com/users/jxmorris12/events{/privacy}
https://api.github.com/users/jxmorris12/followers
https://api.github.com/users/jxmorris12/following{/other_user}
https://api.github.com/users/jxmorris12/gists{/gist_id}
https://github.com/jxmorris12
jxmorris12
MDQ6VXNlcjEzMjM4OTUy
https://api.github.com/users/jxmorris12/orgs
https://api.github.com/users/jxmorris12/received_events
https://api.github.com/users/jxmorris12/repos
https://api.github.com/users/jxmorris12/starred{/owner}{/repo}
https://api.github.com/users/jxmorris12/subscriptions
User
https://api.github.com/users/jxmorris12
public
MEMBER
Yes, this is because the test sets for glue are hidden so the labels are
not publicly available. You can read the glue paper for more details.

On Sat, 6 Jun 2020 at 18:16, Jack Morris <notifications@github.com> wrote:

> this also happened to me with nlp.load_datasets('glue', 'mnli')
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/nlp/issues/245#issuecomment-640083980>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABYDIHMVQD2EDX2HTZUXG5DRVJTWRANCNFSM4NVG3AKQ>
> .
>

2020-06-06T16:39:25Z
https://github.com/huggingface/datasets/issues/245#issuecomment-640087061
https://api.github.com/repos/huggingface/datasets/issues/245
MDEyOklzc3VlQ29tbWVudDY0MDA4NzA2MQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/640087061/reactions
2020-06-06T16:39:25Z
https://api.github.com/repos/huggingface/datasets/issues/comments/640087061
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
CONTRIBUTOR
Thanks @thomwolf!
2020-06-06T16:56:47Z
https://github.com/huggingface/datasets/issues/245#issuecomment-640089299
https://api.github.com/repos/huggingface/datasets/issues/245
MDEyOklzc3VlQ29tbWVudDY0MDA4OTI5OQ==
https://api.github.com/repos/huggingface/datasets/issues/comments/640089299/reactions
2020-06-06T16:56:47Z
https://api.github.com/repos/huggingface/datasets/issues/comments/640089299
https://avatars.githubusercontent.com/u/13238952?v=4
https://api.github.com/users/jxmorris12/events{/privacy}
https://api.github.com/users/jxmorris12/followers
https://api.github.com/users/jxmorris12/following{/other_user}
https://api.github.com/users/jxmorris12/gists{/gist_id}
https://github.com/jxmorris12
jxmorris12
MDQ6VXNlcjEzMjM4OTUy
https://api.github.com/users/jxmorris12/orgs
https://api.github.com/users/jxmorris12/received_events
https://api.github.com/users/jxmorris12/repos
https://api.github.com/users/jxmorris12/starred{/owner}{/repo}
https://api.github.com/users/jxmorris12/subscriptions
User
https://api.github.com/users/jxmorris12
public
CONTRIBUTOR
@thomwolf shouldn't this be visible in the .info and/or in the .features?
2021-01-09T09:50:10Z
https://github.com/huggingface/datasets/issues/245#issuecomment-757124613
https://api.github.com/repos/huggingface/datasets/issues/245
MDEyOklzc3VlQ29tbWVudDc1NzEyNDYxMw==
https://api.github.com/repos/huggingface/datasets/issues/comments/757124613/reactions
2021-01-09T09:50:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/757124613
https://avatars.githubusercontent.com/u/23721977?v=4
https://api.github.com/users/KennethEnevoldsen/events{/privacy}
https://api.github.com/users/KennethEnevoldsen/followers
https://api.github.com/users/KennethEnevoldsen/following{/other_user}
https://api.github.com/users/KennethEnevoldsen/gists{/gist_id}
https://github.com/KennethEnevoldsen
KennethEnevoldsen
MDQ6VXNlcjIzNzIxOTc3
https://api.github.com/users/KennethEnevoldsen/orgs
https://api.github.com/users/KennethEnevoldsen/received_events
https://api.github.com/users/KennethEnevoldsen/repos
https://api.github.com/users/KennethEnevoldsen/starred{/owner}{/repo}
https://api.github.com/users/KennethEnevoldsen/subscriptions
User
https://api.github.com/users/KennethEnevoldsen
public
MEMBER
It should be in the datasets card (the README.md and on the hub) in my opinion. What do you think @yjernite?
2021-01-09T10:48:41Z
https://github.com/huggingface/datasets/issues/245#issuecomment-757131224
https://api.github.com/repos/huggingface/datasets/issues/245
MDEyOklzc3VlQ29tbWVudDc1NzEzMTIyNA==
https://api.github.com/repos/huggingface/datasets/issues/comments/757131224/reactions
2021-01-09T10:48:41Z
https://api.github.com/repos/huggingface/datasets/issues/comments/757131224
https://avatars.githubusercontent.com/u/7353373?v=4
https://api.github.com/users/thomwolf/events{/privacy}
https://api.github.com/users/thomwolf/followers
https://api.github.com/users/thomwolf/following{/other_user}
https://api.github.com/users/thomwolf/gists{/gist_id}
https://github.com/thomwolf
thomwolf
MDQ6VXNlcjczNTMzNzM=
https://api.github.com/users/thomwolf/orgs
https://api.github.com/users/thomwolf/received_events
https://api.github.com/users/thomwolf/repos
https://api.github.com/users/thomwolf/starred{/owner}{/repo}
https://api.github.com/users/thomwolf/subscriptions
User
https://api.github.com/users/thomwolf
public
CONTRIBUTOR
I checked both before I got to looking at issues, so that would be fine as well.

Some additional thoughts on this: Is there a specific reason why the "test" split even has a "label" column if it isn't tagged. Shouldn't there just not be any. Seems more transparent
2021-01-09T12:05:08Z
https://github.com/huggingface/datasets/issues/245#issuecomment-757139712
https://api.github.com/repos/huggingface/datasets/issues/245
MDEyOklzc3VlQ29tbWVudDc1NzEzOTcxMg==
https://api.github.com/repos/huggingface/datasets/issues/comments/757139712/reactions
2021-01-09T12:07:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/757139712
https://avatars.githubusercontent.com/u/23721977?v=4
https://api.github.com/users/KennethEnevoldsen/events{/privacy}
https://api.github.com/users/KennethEnevoldsen/followers
https://api.github.com/users/KennethEnevoldsen/following{/other_user}
https://api.github.com/users/KennethEnevoldsen/gists{/gist_id}
https://github.com/KennethEnevoldsen
KennethEnevoldsen
MDQ6VXNlcjIzNzIxOTc3
https://api.github.com/users/KennethEnevoldsen/orgs
https://api.github.com/users/KennethEnevoldsen/received_events
https://api.github.com/users/KennethEnevoldsen/repos
https://api.github.com/users/KennethEnevoldsen/starred{/owner}{/repo}
https://api.github.com/users/KennethEnevoldsen/subscriptions
User
https://api.github.com/users/KennethEnevoldsen
public
NONE
I'm a little confused with the data size.
`sst2` dataset is referenced to `Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank` and the link of the dataset in the paper is https://nlp.stanford.edu/sentiment/index.html which is often shown in GLUE/SST2 reference.
From the original data, the standard train/dev/test splits split is 6920/872/1821 for binary classification. 
Why in GLUE/SST2 the train/dev/test split is 67,349/872/1,821 ? 


2021-01-11T15:15:56Z
https://github.com/huggingface/datasets/issues/245#issuecomment-758015788
https://api.github.com/repos/huggingface/datasets/issues/245
MDEyOklzc3VlQ29tbWVudDc1ODAxNTc4OA==
https://api.github.com/repos/huggingface/datasets/issues/comments/758015788/reactions
2021-01-11T15:15:56Z
https://api.github.com/repos/huggingface/datasets/issues/comments/758015788
https://avatars.githubusercontent.com/u/21202514?v=4
https://api.github.com/users/Opdoop/events{/privacy}
https://api.github.com/users/Opdoop/followers
https://api.github.com/users/Opdoop/following{/other_user}
https://api.github.com/users/Opdoop/gists{/gist_id}
https://github.com/Opdoop
Opdoop
MDQ6VXNlcjIxMjAyNTE0
https://api.github.com/users/Opdoop/orgs
https://api.github.com/users/Opdoop/received_events
https://api.github.com/users/Opdoop/repos
https://api.github.com/users/Opdoop/starred{/owner}{/repo}
https://api.github.com/users/Opdoop/subscriptions
User
https://api.github.com/users/Opdoop
public
NONE
> I'm a little confused with the data size.
> `sst2` dataset is referenced to `Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank` and the link of the dataset in the paper is https://nlp.stanford.edu/sentiment/index.html which is often shown in GLUE/SST2 reference.
> From the original data, the standard train/dev/test splits split is 6920/872/1821 for binary classification.
> Why in GLUE/SST2 the train/dev/test split is 67,349/872/1,821 ?

Have you figured out this problem? AFAIK, the original sst-2 dataset is totally different from the GLUE/sst-2. Do you think so?
2021-07-29T15:04:02Z
https://github.com/huggingface/datasets/issues/245#issuecomment-889222747
https://api.github.com/repos/huggingface/datasets/issues/245
IC_kwDODunzps41AHZb
https://api.github.com/repos/huggingface/datasets/issues/comments/889222747/reactions
2021-07-29T15:04:02Z
https://api.github.com/repos/huggingface/datasets/issues/comments/889222747
https://avatars.githubusercontent.com/u/37548571?v=4
https://api.github.com/users/yc1999/events{/privacy}
https://api.github.com/users/yc1999/followers
https://api.github.com/users/yc1999/following{/other_user}
https://api.github.com/users/yc1999/gists{/gist_id}
https://github.com/yc1999
yc1999
MDQ6VXNlcjM3NTQ4NTcx
https://api.github.com/users/yc1999/orgs
https://api.github.com/users/yc1999/received_events
https://api.github.com/users/yc1999/repos
https://api.github.com/users/yc1999/starred{/owner}{/repo}
https://api.github.com/users/yc1999/subscriptions
User
https://api.github.com/users/yc1999
public
NONE
@yc1999 Sorry, I didn't solve this conflict. In the end, I just use a local data file provided by the previous work I followed(for consistent comparison), not use `datasets` package.

Related information: https://github.com/thunlp/OpenAttack/issues/146#issuecomment-766323571
2021-08-03T07:42:44Z
https://github.com/huggingface/datasets/issues/245#issuecomment-891615363
https://api.github.com/repos/huggingface/datasets/issues/245
IC_kwDODunzps41JPiD
https://api.github.com/repos/huggingface/datasets/issues/comments/891615363/reactions
2021-08-03T07:42:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/891615363
https://avatars.githubusercontent.com/u/21202514?v=4
https://api.github.com/users/Opdoop/events{/privacy}
https://api.github.com/users/Opdoop/followers
https://api.github.com/users/Opdoop/following{/other_user}
https://api.github.com/users/Opdoop/gists{/gist_id}
https://github.com/Opdoop
Opdoop
MDQ6VXNlcjIxMjAyNTE0
https://api.github.com/users/Opdoop/orgs
https://api.github.com/users/Opdoop/received_events
https://api.github.com/users/Opdoop/repos
https://api.github.com/users/Opdoop/starred{/owner}{/repo}
https://api.github.com/users/Opdoop/subscriptions
User
https://api.github.com/users/Opdoop
public
NONE
@yc1999 I find that the original SST-2 dataset (6,920/872/1,821) can be loaded from https://huggingface.co/datasets/gpt3mix/sst2 or built with SST data and the scripts in https://github.com/prrao87/fine-grained-sentiment/tree/master/data/sst.
The GLUE/SST-2 dataset (67,349/872/1,821) should be a completely different version.

2021-12-08T00:47:32Z
https://github.com/huggingface/datasets/issues/245#issuecomment-988386743
https://api.github.com/repos/huggingface/datasets/issues/245
IC_kwDODunzps466ZW3
https://api.github.com/repos/huggingface/datasets/issues/comments/988386743/reactions
2021-12-08T00:47:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/988386743
https://avatars.githubusercontent.com/u/8981163?v=4
https://api.github.com/users/junyann/events{/privacy}
https://api.github.com/users/junyann/followers
https://api.github.com/users/junyann/following{/other_user}
https://api.github.com/users/junyann/gists{/gist_id}
https://github.com/junyann
junyann
MDQ6VXNlcjg5ODExNjM=
https://api.github.com/users/junyann/orgs
https://api.github.com/users/junyann/received_events
https://api.github.com/users/junyann/repos
https://api.github.com/users/junyann/starred{/owner}{/repo}
https://api.github.com/users/junyann/subscriptions
User
https://api.github.com/users/junyann
public
I'm trying to test a model on the SST-2 task, but all the labels I see in the test set are -1.
```
>>> import nlp
>>> glue = nlp.load_dataset('glue', 'sst2')
>>> glue
{'train': Dataset(schema: {'sentence': 'string', 'label': 'int64', 'idx': 'int32'}, num_rows: 67349), 'validation': Dataset(schema: {'sentence': 'string', 'label': 'int64', 'idx': 'int32'}, num_rows: 872), 'test': Dataset(schema: {'sentence': 'string', 'label': 'int64', 'idx': 'int32'}, num_rows: 1821)}
>>> list(l['label'] for l in glue['test'])
[-1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]
```