LexGLUE: A Benchmark Dataset for Legal Language Understanding in English.
CONTRIBUTOR
Please wait until Tuesday. Arxiv pre-print is pending. 🤗 
2021-10-03T13:29:53Z
https://github.com/huggingface/datasets/pull/3004#issuecomment-932951793
https://api.github.com/repos/huggingface/datasets/issues/3004
IC_kwDODunzps43m7bx
https://api.github.com/repos/huggingface/datasets/issues/comments/932951793/reactions
2021-10-03T13:29:53Z
https://api.github.com/repos/huggingface/datasets/issues/comments/932951793
https://avatars.githubusercontent.com/u/1626984?v=4
https://api.github.com/users/iliaschalkidis/events{/privacy}
https://api.github.com/users/iliaschalkidis/followers
https://api.github.com/users/iliaschalkidis/following{/other_user}
https://api.github.com/users/iliaschalkidis/gists{/gist_id}
https://github.com/iliaschalkidis
iliaschalkidis
MDQ6VXNlcjE2MjY5ODQ=
https://api.github.com/users/iliaschalkidis/orgs
https://api.github.com/users/iliaschalkidis/received_events
https://api.github.com/users/iliaschalkidis/repos
https://api.github.com/users/iliaschalkidis/starred{/owner}{/repo}
https://api.github.com/users/iliaschalkidis/subscriptions
User
https://api.github.com/users/iliaschalkidis
public
CONTRIBUTOR
Hi @lhoestq, I updated the README with the Arxiv publication info and now the tests are not passing.

It seems that the error is completely irrelevant to my code:

```
 Attempting uninstall: ruamel.yaml
    Found existing installation: ruamel-yaml 0.15.87
ERROR: Cannot uninstall 'ruamel-yaml'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
```
2021-10-05T08:06:05Z
https://github.com/huggingface/datasets/pull/3004#issuecomment-934167707
https://api.github.com/repos/huggingface/datasets/issues/3004
IC_kwDODunzps43rkSb
https://api.github.com/repos/huggingface/datasets/issues/comments/934167707/reactions
2021-10-05T08:06:05Z
https://api.github.com/repos/huggingface/datasets/issues/comments/934167707
https://avatars.githubusercontent.com/u/1626984?v=4
https://api.github.com/users/iliaschalkidis/events{/privacy}
https://api.github.com/users/iliaschalkidis/followers
https://api.github.com/users/iliaschalkidis/following{/other_user}
https://api.github.com/users/iliaschalkidis/gists{/gist_id}
https://github.com/iliaschalkidis
iliaschalkidis
MDQ6VXNlcjE2MjY5ODQ=
https://api.github.com/users/iliaschalkidis/orgs
https://api.github.com/users/iliaschalkidis/received_events
https://api.github.com/users/iliaschalkidis/repos
https://api.github.com/users/iliaschalkidis/starred{/owner}{/repo}
https://api.github.com/users/iliaschalkidis/subscriptions
User
https://api.github.com/users/iliaschalkidis
public
MEMBER
Hi ! Thanks for adding this one. Amazing work :o 

Yea you can ignore the `ruamel-yaml` error, it's unrelated to your PR and fixed on `master`
2021-10-08T16:20:40Z
https://github.com/huggingface/datasets/pull/3004#issuecomment-938776756
https://api.github.com/repos/huggingface/datasets/issues/3004
IC_kwDODunzps439Ji0
https://api.github.com/repos/huggingface/datasets/issues/comments/938776756/reactions
2021-10-08T16:20:40Z
https://api.github.com/repos/huggingface/datasets/issues/comments/938776756
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq, 

- I fetched and merged the upstream master to get the `ruamel-yaml` fix.
- I updated the README.md file including more information for the inputs and labels, while I also turned all tables in HTML format hoping that they will render nicely in the dataset card in the HF website.
- I also simplified the CaseHOLD input, excl. the unused `question` field and the `context` replicas, as suggested.

2021-10-09T10:56:32Z
https://github.com/huggingface/datasets/pull/3004#issuecomment-939276942
https://api.github.com/repos/huggingface/datasets/issues/3004
IC_kwDODunzps43_DqO
https://api.github.com/repos/huggingface/datasets/issues/comments/939276942/reactions
2021-10-09T10:56:32Z
https://api.github.com/repos/huggingface/datasets/issues/comments/939276942
https://avatars.githubusercontent.com/u/1626984?v=4
https://api.github.com/users/iliaschalkidis/events{/privacy}
https://api.github.com/users/iliaschalkidis/followers
https://api.github.com/users/iliaschalkidis/following{/other_user}
https://api.github.com/users/iliaschalkidis/gists{/gist_id}
https://github.com/iliaschalkidis
iliaschalkidis
MDQ6VXNlcjE2MjY5ODQ=
https://api.github.com/users/iliaschalkidis/orgs
https://api.github.com/users/iliaschalkidis/received_events
https://api.github.com/users/iliaschalkidis/repos
https://api.github.com/users/iliaschalkidis/starred{/owner}{/repo}
https://api.github.com/users/iliaschalkidis/subscriptions
User
https://api.github.com/users/iliaschalkidis
public
Inspired by the recent widespread use of the GLUE multi-task benchmark NLP dataset (Wang et al., 2018), the subsequent more difficult SuperGLUE (Wang et al., 2019), other previous multi-task NLP benchmarks (Conneau and Kiela, 2018; McCann et al., 2018), and similar initiatives in other domains (Peng et al., 2019), we introduce the Legal General Language Understanding Evaluation (LexGLUE) benchmark, a benchmark dataset to evaluate the performance of NLP methods in legal tasks. LexGLUE is based on seven existing legal NLP datasets, selected using criteria largely from SuperGLUE.

As in GLUE and SuperGLUE (Wang et al., 2019b,a), one of our goals is to push towards generic (or ‘foundation’) models that can cope with multiple NLP tasks, in our case legal NLP tasks possibly with limited task-specific fine-tuning. Another goal is to provide a convenient and informative entry point for NLP researchers and practitioners wishing to explore or develop methods for legalNLP. Having these goals in mind, the datasets we include in LexGLUE and the tasks they address have been simplified in several ways to make it easier for newcomers and generic models to address all tasks.

LexGLUE benchmark is accompanied by experimental infrastructure that relies on Hugging Face Transformers library and resides at: https://github.com/coastalcph/lex-glue.