Update model name
Browse files
README.md
CHANGED
|
@@ -15,18 +15,18 @@ We release pre-trained language models for Modern Standard Arabic (MSA), dialect
|
|
| 15 |
We also provide additional models that are pre-trained on a scaled-down set of the MSA variant (half, quarter, eighth, and sixteenth).
|
| 16 |
The details are described in the paper *"[The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://arxiv.org/abs/2103.06678)."*
|
| 17 |
|
| 18 |
-
This model card describes **CAMeLBERT-DA** (`bert-base-camelbert-da`), a model pre-trained on the DA (dialectal Arabic) dataset.
|
| 19 |
|
| 20 |
||Model|Variant|Size|#Word|
|
| 21 |
|-|-|:-:|-:|-:|
|
| 22 |
-
||`bert-base-camelbert-mix`|CA,DA,MSA|167GB|17.3B|
|
| 23 |
-
||`bert-base-camelbert-ca`|CA|6GB|847M|
|
| 24 |
-
|✔|`bert-base-camelbert-da`|DA|54GB|5.8B|
|
| 25 |
-
||`bert-base-camelbert-msa`|MSA|107GB|12.6B|
|
| 26 |
-
||`bert-base-camelbert-msa-half`|MSA|53GB|6.3B|
|
| 27 |
-
||`bert-base-camelbert-msa-quarter`|MSA|27GB|3.1B|
|
| 28 |
-
||`bert-base-camelbert-msa-eighth`|MSA|14GB|1.6B|
|
| 29 |
-
||`bert-base-camelbert-msa-sixteenth`|MSA|6GB|746M|
|
| 30 |
|
| 31 |
## Intended uses
|
| 32 |
You can use the released model for either masked language modeling or next sentence prediction.
|
|
@@ -37,7 +37,7 @@ We release our fine-tuninig code [here](https://github.com/CAMeL-Lab/CAMeLBERT).
|
|
| 37 |
You can use this model directly with a pipeline for masked language modeling:
|
| 38 |
```python
|
| 39 |
>>> from transformers import pipeline
|
| 40 |
-
>>> unmasker = pipeline('fill-mask', model='CAMeL-Lab/bert-base-camelbert-da')
|
| 41 |
>>> unmasker("الهدف من الحياة هو [MASK] .")
|
| 42 |
[{'sequence': '[CLS] الهدف من الحياة هو.. [SEP]',
|
| 43 |
'score': 0.062508225440979,
|
|
@@ -66,8 +66,8 @@ You can use this model directly with a pipeline for masked language modeling:
|
|
| 66 |
Here is how to use this model to get the features of a given text in PyTorch:
|
| 67 |
```python
|
| 68 |
from transformers import AutoTokenizer, AutoModel
|
| 69 |
-
tokenizer = AutoTokenizer.from_pretrained('CAMeL-Lab/bert-base-camelbert-da')
|
| 70 |
-
model = AutoModel.from_pretrained('CAMeL-Lab/bert-base-camelbert-da')
|
| 71 |
text = "مرحبا يا عالم."
|
| 72 |
encoded_input = tokenizer(text, return_tensors='pt')
|
| 73 |
output = model(**encoded_input)
|
|
@@ -76,8 +76,8 @@ output = model(**encoded_input)
|
|
| 76 |
and in TensorFlow:
|
| 77 |
```python
|
| 78 |
from transformers import AutoTokenizer, TFAutoModel
|
| 79 |
-
tokenizer = AutoTokenizer.from_pretrained('CAMeL-Lab/bert-base-camelbert-da')
|
| 80 |
-
model = TFAutoModel.from_pretrained('CAMeL-Lab/bert-base-camelbert-da')
|
| 81 |
text = "مرحبا يا عالم."
|
| 82 |
encoded_input = tokenizer(text, return_tensors='tf')
|
| 83 |
output = model(encoded_input)
|
|
|
|
| 15 |
We also provide additional models that are pre-trained on a scaled-down set of the MSA variant (half, quarter, eighth, and sixteenth).
|
| 16 |
The details are described in the paper *"[The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://arxiv.org/abs/2103.06678)."*
|
| 17 |
|
| 18 |
+
This model card describes **CAMeLBERT-DA** (`bert-base-arabic-camelbert-da`), a model pre-trained on the DA (dialectal Arabic) dataset.
|
| 19 |
|
| 20 |
||Model|Variant|Size|#Word|
|
| 21 |
|-|-|:-:|-:|-:|
|
| 22 |
+
||`bert-base-arabic-camelbert-mix`|CA,DA,MSA|167GB|17.3B|
|
| 23 |
+
||`bert-base-arabic-camelbert-ca`|CA|6GB|847M|
|
| 24 |
+
|✔|`bert-base-arabic-camelbert-da`|DA|54GB|5.8B|
|
| 25 |
+
||`bert-base-arabic-camelbert-msa`|MSA|107GB|12.6B|
|
| 26 |
+
||`bert-base-arabic-camelbert-msa-half`|MSA|53GB|6.3B|
|
| 27 |
+
||`bert-base-arabic-camelbert-msa-quarter`|MSA|27GB|3.1B|
|
| 28 |
+
||`bert-base-arabic-camelbert-msa-eighth`|MSA|14GB|1.6B|
|
| 29 |
+
||`bert-base-arabic-camelbert-msa-sixteenth`|MSA|6GB|746M|
|
| 30 |
|
| 31 |
## Intended uses
|
| 32 |
You can use the released model for either masked language modeling or next sentence prediction.
|
|
|
|
| 37 |
You can use this model directly with a pipeline for masked language modeling:
|
| 38 |
```python
|
| 39 |
>>> from transformers import pipeline
|
| 40 |
+
>>> unmasker = pipeline('fill-mask', model='CAMeL-Lab/bert-base-arabic-camelbert-da')
|
| 41 |
>>> unmasker("الهدف من الحياة هو [MASK] .")
|
| 42 |
[{'sequence': '[CLS] الهدف من الحياة هو.. [SEP]',
|
| 43 |
'score': 0.062508225440979,
|
|
|
|
| 66 |
Here is how to use this model to get the features of a given text in PyTorch:
|
| 67 |
```python
|
| 68 |
from transformers import AutoTokenizer, AutoModel
|
| 69 |
+
tokenizer = AutoTokenizer.from_pretrained('CAMeL-Lab/bert-base-arabic-camelbert-da')
|
| 70 |
+
model = AutoModel.from_pretrained('CAMeL-Lab/bert-base-arabic-camelbert-da')
|
| 71 |
text = "مرحبا يا عالم."
|
| 72 |
encoded_input = tokenizer(text, return_tensors='pt')
|
| 73 |
output = model(**encoded_input)
|
|
|
|
| 76 |
and in TensorFlow:
|
| 77 |
```python
|
| 78 |
from transformers import AutoTokenizer, TFAutoModel
|
| 79 |
+
tokenizer = AutoTokenizer.from_pretrained('CAMeL-Lab/bert-base-arabic-camelbert-da')
|
| 80 |
+
model = TFAutoModel.from_pretrained('CAMeL-Lab/bert-base-arabic-camelbert-da')
|
| 81 |
text = "مرحبا يا عالم."
|
| 82 |
encoded_input = tokenizer(text, return_tensors='tf')
|
| 83 |
output = model(encoded_input)
|