Release of FairLex dataset
MEMBER
This is awesome ! The dataset card and the dataset script look amazing :)

I wanted to ask you if you'd be interested to have this dataset under the namespace of you research group at https://huggingface.co/coastalcph ? If yes, then you can actually create a dataset repository under your research group name and upload the files from this PR there
2022-03-02T14:42:19Z
https://github.com/huggingface/datasets/pull/3802#issuecomment-1057005816
https://api.github.com/repos/huggingface/datasets/issues/3802
IC_kwDODunzps4_AKD4
https://api.github.com/repos/huggingface/datasets/issues/comments/1057005816/reactions
2022-03-02T14:42:19Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1057005816
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Hi @lhoestq,

Yeah, I could do that. I see that people do that a lot of models, but not for datasets. 

Is there any good reason to have it under the organization domain instead of the general domain?

 Thanks!
2022-03-02T14:46:59Z
https://github.com/huggingface/datasets/pull/3802#issuecomment-1057010713
https://api.github.com/repos/huggingface/datasets/issues/3802
IC_kwDODunzps4_ALQZ
https://api.github.com/repos/huggingface/datasets/issues/comments/1057010713/reactions
2022-03-02T14:46:59Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1057010713
https://avatars.githubusercontent.com/u/1626984?v=4
https://api.github.com/users/iliaschalkidis/events{/privacy}
https://api.github.com/users/iliaschalkidis/followers
https://api.github.com/users/iliaschalkidis/following{/other_user}
https://api.github.com/users/iliaschalkidis/gists{/gist_id}
https://github.com/iliaschalkidis
iliaschalkidis
MDQ6VXNlcjE2MjY5ODQ=
https://api.github.com/users/iliaschalkidis/orgs
https://api.github.com/users/iliaschalkidis/received_events
https://api.github.com/users/iliaschalkidis/repos
https://api.github.com/users/iliaschalkidis/starred{/owner}{/repo}
https://api.github.com/users/iliaschalkidis/subscriptions
User
https://api.github.com/users/iliaschalkidis
public
MEMBER
It's nice to have it under your namespace:
- it will appear on your research group page, along with your models
- you can edit or create datasets at any time - you don't need to open PRs on GitHub

All the datasets that are not under a namespace are this way because we started adding datasets from GitHub. Now we encourage users to upload them directly to make things simpler, and aligned with the workflow for models

(the documentation will be updated in the following days)

Note that we will keep accepting PRs here though when there is no clear namespace under which a dataset should be, or for users that want a review from us
2022-03-02T14:54:39Z
https://github.com/huggingface/datasets/pull/3802#issuecomment-1057019647
https://api.github.com/repos/huggingface/datasets/issues/3802
IC_kwDODunzps4_ANb_
https://api.github.com/repos/huggingface/datasets/issues/comments/1057019647/reactions
2022-03-02T14:54:39Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1057019647
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Ok, I'll do that. So, I'll just have to upload all the files under the `/fairlex` directory in my PR, right?
2022-03-02T14:57:18Z
https://github.com/huggingface/datasets/pull/3802#issuecomment-1057023758
https://api.github.com/repos/huggingface/datasets/issues/3802
IC_kwDODunzps4_AOcO
https://api.github.com/repos/huggingface/datasets/issues/comments/1057023758/reactions
2022-03-02T14:57:18Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1057023758
https://avatars.githubusercontent.com/u/1626984?v=4
https://api.github.com/users/iliaschalkidis/events{/privacy}
https://api.github.com/users/iliaschalkidis/followers
https://api.github.com/users/iliaschalkidis/following{/other_user}
https://api.github.com/users/iliaschalkidis/gists{/gist_id}
https://github.com/iliaschalkidis
iliaschalkidis
MDQ6VXNlcjE2MjY5ODQ=
https://api.github.com/users/iliaschalkidis/orgs
https://api.github.com/users/iliaschalkidis/received_events
https://api.github.com/users/iliaschalkidis/repos
https://api.github.com/users/iliaschalkidis/starred{/owner}{/repo}
https://api.github.com/users/iliaschalkidis/subscriptions
User
https://api.github.com/users/iliaschalkidis
public
MEMBER
Yes exactly !
2022-03-02T15:01:44Z
https://github.com/huggingface/datasets/pull/3802#issuecomment-1057030094
https://api.github.com/repos/huggingface/datasets/issues/3802
IC_kwDODunzps4_AP_O
https://api.github.com/repos/huggingface/datasets/issues/comments/1057030094/reactions
2022-03-02T15:01:44Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1057030094
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Ok, I uploaded most of them from the UI environment (https://huggingface.co/datasets/coastalcph/fairlex). Can I possibly upload the dummy data as well from the UI environment. I really want to avoid the CLI right now 😄 
2022-03-02T15:05:51Z
https://github.com/huggingface/datasets/pull/3802#issuecomment-1057035280
https://api.github.com/repos/huggingface/datasets/issues/3802
IC_kwDODunzps4_ARQQ
https://api.github.com/repos/huggingface/datasets/issues/comments/1057035280/reactions
2022-03-02T15:05:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1057035280
https://avatars.githubusercontent.com/u/1626984?v=4
https://api.github.com/users/iliaschalkidis/events{/privacy}
https://api.github.com/users/iliaschalkidis/followers
https://api.github.com/users/iliaschalkidis/following{/other_user}
https://api.github.com/users/iliaschalkidis/gists{/gist_id}
https://github.com/iliaschalkidis
iliaschalkidis
MDQ6VXNlcjE2MjY5ODQ=
https://api.github.com/users/iliaschalkidis/orgs
https://api.github.com/users/iliaschalkidis/received_events
https://api.github.com/users/iliaschalkidis/repos
https://api.github.com/users/iliaschalkidis/starred{/owner}{/repo}
https://api.github.com/users/iliaschalkidis/subscriptions
User
https://api.github.com/users/iliaschalkidis
public
MEMBER
Yea sure, feel free to use the UI of the website, even for the dummy data ^^
2022-03-02T15:13:45Z
https://github.com/huggingface/datasets/pull/3802#issuecomment-1057043129
https://api.github.com/repos/huggingface/datasets/issues/3802
IC_kwDODunzps4_ATK5
https://api.github.com/repos/huggingface/datasets/issues/comments/1057043129/reactions
2022-03-02T15:13:45Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1057043129
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Did you upload them yourself? Because I see the data preview, and I'm pretty sure, I didn't do that 😄 
2022-03-02T15:15:33Z
https://github.com/huggingface/datasets/pull/3802#issuecomment-1057044905
https://api.github.com/repos/huggingface/datasets/issues/3802
IC_kwDODunzps4_ATmp
https://api.github.com/repos/huggingface/datasets/issues/comments/1057044905/reactions
2022-03-02T15:15:33Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1057044905
https://avatars.githubusercontent.com/u/1626984?v=4
https://api.github.com/users/iliaschalkidis/events{/privacy}
https://api.github.com/users/iliaschalkidis/followers
https://api.github.com/users/iliaschalkidis/following{/other_user}
https://api.github.com/users/iliaschalkidis/gists{/gist_id}
https://github.com/iliaschalkidis
iliaschalkidis
MDQ6VXNlcjE2MjY5ODQ=
https://api.github.com/users/iliaschalkidis/orgs
https://api.github.com/users/iliaschalkidis/received_events
https://api.github.com/users/iliaschalkidis/repos
https://api.github.com/users/iliaschalkidis/starred{/owner}{/repo}
https://api.github.com/users/iliaschalkidis/subscriptions
User
https://api.github.com/users/iliaschalkidis
public
MEMBER
The preview is computed from the real data ;)

The dummy data are used for testing only
2022-03-02T15:16:23Z
https://github.com/huggingface/datasets/pull/3802#issuecomment-1057045800
https://api.github.com/repos/huggingface/datasets/issues/3802
IC_kwDODunzps4_AT0o
https://api.github.com/repos/huggingface/datasets/issues/comments/1057045800/reactions
2022-03-02T15:16:35Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1057045800
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public
CONTRIBUTOR
Haha, ok I was shocked! Cool, I close this PR, then. Thanks, again! 
2022-03-02T15:18:51Z
https://github.com/huggingface/datasets/pull/3802#issuecomment-1057048290
https://api.github.com/repos/huggingface/datasets/issues/3802
IC_kwDODunzps4_AUbi
https://api.github.com/repos/huggingface/datasets/issues/comments/1057048290/reactions
2022-03-02T15:18:51Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1057048290
https://avatars.githubusercontent.com/u/1626984?v=4
https://api.github.com/users/iliaschalkidis/events{/privacy}
https://api.github.com/users/iliaschalkidis/followers
https://api.github.com/users/iliaschalkidis/following{/other_user}
https://api.github.com/users/iliaschalkidis/gists{/gist_id}
https://github.com/iliaschalkidis
iliaschalkidis
MDQ6VXNlcjE2MjY5ODQ=
https://api.github.com/users/iliaschalkidis/orgs
https://api.github.com/users/iliaschalkidis/received_events
https://api.github.com/users/iliaschalkidis/repos
https://api.github.com/users/iliaschalkidis/starred{/owner}{/repo}
https://api.github.com/users/iliaschalkidis/subscriptions
User
https://api.github.com/users/iliaschalkidis
public
MEMBER
Thank you 🤗
2022-03-02T15:21:10Z
https://github.com/huggingface/datasets/pull/3802#issuecomment-1057050650
https://api.github.com/repos/huggingface/datasets/issues/3802
IC_kwDODunzps4_AVAa
https://api.github.com/repos/huggingface/datasets/issues/comments/1057050650/reactions
2022-03-02T15:21:10Z
https://api.github.com/repos/huggingface/datasets/issues/comments/1057050650
https://avatars.githubusercontent.com/u/42851186?v=4
https://api.github.com/users/lhoestq/events{/privacy}
https://api.github.com/users/lhoestq/followers
https://api.github.com/users/lhoestq/following{/other_user}
https://api.github.com/users/lhoestq/gists{/gist_id}
https://github.com/lhoestq
lhoestq
MDQ6VXNlcjQyODUxMTg2
https://api.github.com/users/lhoestq/orgs
https://api.github.com/users/lhoestq/received_events
https://api.github.com/users/lhoestq/repos
https://api.github.com/users/lhoestq/starred{/owner}{/repo}
https://api.github.com/users/lhoestq/subscriptions
User
https://api.github.com/users/lhoestq
public

**FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing**

We present a benchmark suite of four datasets for evaluating the fairness of pre-trained legal language models and the techniques used to fine-tune them for downstream tasks. Our benchmarks cover four jurisdictions (European Council, USA, Swiss, and Chinese), five languages (English, German, French, Italian, and Chinese), and fairness across five attributes (gender, age, nationality/region, language, and legal area). In our experiments, we evaluate pre-trained language models using several group-robust fine-tuning techniques and show that performance group disparities are vibrant in many cases, while none of these techniques guarantee fairness, nor consistently mitigate group disparities. Furthermore, we provide a quantitative and qualitative analysis of our results, highlighting open challenges in the development of robustness methods in legal NLP.

*Ilias Chalkidis, Tommaso Pasini, Sheng Zhang, Letizia Tomada, Letizia, Sebastian Felix Schwemer, Anders Søgaard. FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing. 2022. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Dublin, Ireland.*

Note: Please review this initial commit, and I'll update the publication link, once I'll have the ArXived version. Thanks!
